It is enough to show that the function $F$ is differentiable on every open subset of $\mathbb{R}$. So let $r>0$, and $ \phi: X\times(-r,r) \to [0,\infty],\ \phi(x,t)=s(f(x)+tg(x))=:\phi^x(t), $ where $ s: \mathbb{R} \to [0,\infty),\ s(t)=|t|^p. $ Since $s$ is differentiable, and $ s'(t)=\begin{cases} p|t|^{p-2}t &\text{ for } t \ne 0\\ 0 &\text{ for } t=0 \end{cases}, $ it follows that for every $x$ in $ \Omega:=\{x \in X:\ |f(x)|<\infty\}\cap\{x \in X:\ |g(x)|<\infty\} $ the function $\phi^x$ is differentiable and $ (\phi^x)'(t)=\partial_t\phi(x,t)=g(x)s'(f(x)+tg(x)) \quad \forall\ t \in (-r,r). $ Therefore $ |\partial_t\phi(x,t)| \le G_r(x):=\max(1,r^{p-1})|g(x)|(|f(x)|+|g(x)|)^{p-1} \quad \forall\ (x,t) \in \Omega\times(-r,r) $ Thanks to Hölder's inequality we have $ \int_XG_r\,d\mu=\max(1,r^{p-1})\int_X|g|(|f|+|g|)^{p-1}\le \max(1,r^{p-1})\|g\|_{L^p(X)}\|(|f|+|g|)\|^{p-1}_{L^p(X)}, $ i.e. $G_r \in L^1(X)$
Given $t_0 \in (-r,r)$ and a sequence $\{t_n\} \subset (-r,r)$ with $t_n \to t_0$ we set $ \tilde{\phi}_n(x,t_0)=\frac{\phi(x,t_0)-\phi(x,t_n)}{t_0-t_n} \quad \forall x \in \Omega, n \in \mathbb{N}. $ Then $ \lim_n\tilde{\phi}(x,t_0)=\partial_t\phi(x,t_0) \quad \forall\ x\in \Omega. $ Thanks to the MVT there is some $\alpha=\alpha(t_0,t_n) \in [0,1]$ such that $ |\tilde{\phi}_n(x,t_0)|=|\partial_t\phi(x,\alpha t_0+(1-\alpha)t_n)|\le G_r(x) \quad \forall\ x \in \Omega, n \in \mathbb{N}. $ Applying the dominated convergence theorem to the sequence $\{\tilde{\phi}_n\}$ we get for every $t_0 \in (-r,r)$: $ \int_X\partial_t\phi(x,t_0)d\mu(x)=\int_X\lim_n\tilde{\phi}_n(x,t_0)d\mu(x)=\lim_n\int_X\tilde{\phi}_n(x,t_0)=\lim_n\frac{F(t_0)-F(t_n)}{t_0-t_n}= F'(t_0). $ In particular we have $ F'(0)=\int_X g(x)s'(f(x))=\int_X fg|f|^{p-2}. $