1
$\begingroup$

I was wondering how to proof this formula, commonly used in Bayesian Prediction:

$ \mathrm{P}(x|\alpha) = \int_\theta \mathrm{P}(x|\theta)\mathrm{P}(\theta|\alpha) \, \mathrm{d}\theta$

The left hand side can be expressed as the following, through marginalizing:

$ \mathrm{P}(x|\alpha) = \int_\theta \mathrm{P}(x, \theta | \alpha) \, \mathrm{d}\theta \quad \quad \ldots \text{(1)}$

Expanding the right hand side,

$ \int_\theta \mathrm{P}(x|\theta) \mathrm{P}(\theta|\alpha) \, \mathrm{d}\theta = \int_\theta \frac{\mathrm{P}(x,\theta)}{\mathrm{P}(\theta)} \frac{\mathrm{P}(\theta,\alpha)}{\mathrm{P}(\alpha)} \, \mathrm{d} \theta \quad \quad \ldots \text{(2)}$

Note that in equation (1), there will be a $\mathrm{P}(x,\theta,\alpha)$ term, but in equation (2), I can't see how that term will appear.

Thanks.

1 Answers 1

2

We have, in general

$ \mathrm{P}(x) = \int_\theta \mathrm{P}(x|\theta)\mathrm{P}(\theta) \, \mathrm{d}\theta$

and conditioning everything on $\alpha$ :

$ \mathrm{P}(x | \alpha) = \int_\theta \mathrm{P}(x|\theta \alpha) \mathrm{P}(\theta | \alpha) \, \mathrm{d}\theta$

In the Bayesian setting, $\mathrm{P}(x|\theta \alpha)=\mathrm{P}(x|\theta)$ because, if we are given the parameter $\theta$ we know the density of $x$, and the values of $\alpha$ adds nothing ($\alpha$ only gives us information about $\theta$ - once we know it, they contribute nothing).

  • 0
    It's an implicit assumption in the usual Bayesian setting, I'd say. Example: we are told that x follows a uniform distribution in $[0,\theta]$. Further, the (in principle unkown) parameter $\theta$ is assumed to be (a priori) a random variable with exponential distribution, with parameter $\lambda$ (parameter of our parameter=hyperparameter). Then, it's evident (isnt it?) that knowing $\theta$ we don't need $\lambda$, so that $\mathrm{P}(x|\theta, \lambda) = \mathrm{P}(x|\theta)$2012-08-24