Is it always true that -$\int_{\mathbb{R}} \log[p(Y|\theta)]v(\theta)d\theta \geq 0$ where $p(Y|\theta)$ is the likelihood for data $Y$ and $v(\theta)$ is any prior distribution? Thanks.
Is it always true that $-\int_{\mathbb{R}} \log p(Y|\theta)v(\theta)d\theta \geq 0$?
0
$\begingroup$
probability
probability-theory
2 Answers
1
For the case when $Y$ is not discrete, this expectation can be negative. Let for simplicity $Y$ is a single r.v. with PDF $p(y|\theta)=\theta e^{-\theta y}$ for $y>0$, $P(\theta=1)=P(\theta=4)=1/2$.
Let also observed $Y=1/4$. Then $-\log p(y|\theta)=-\log(\theta)+\theta y=-\log \theta+\dfrac{\theta}4$, $$ E\left(-\log\theta+\dfrac{\theta}4\right)=\dfrac12\left(-\log 1+\dfrac14\right)+\dfrac12\left(-\log 4+1\right)=\dfrac58-\log 2\approx -0.068 < 0. $$
0
If $Y$ is discrete, you can use Jensen's inequality and the fact that $P(Y) \le 1$: \begin{align*} \int_{\mathbb{R}} -\log[p(Y|\theta)]v(\theta)d\theta &\ge - \log \int_{\mathbb{R}}p(Y|\theta)\nu(\theta)d\theta \\ &= -\log p(Y) \\ &\ge 0. \end{align*}
Otherwise I'm not sure.