1
$\begingroup$

I was wondering how to proof this formula, commonly used in Bayesian Prediction:

$$ \mathrm{P}(x|\alpha) = \int_\theta \mathrm{P}(x|\theta)\mathrm{P}(\theta|\alpha) \, \mathrm{d}\theta$$

The left hand side can be expressed as the following, through marginalizing:

$$ \mathrm{P}(x|\alpha) = \int_\theta \mathrm{P}(x, \theta | \alpha) \, \mathrm{d}\theta \quad \quad \ldots \text{(1)}$$

Expanding the right hand side,

$$ \int_\theta \mathrm{P}(x|\theta) \mathrm{P}(\theta|\alpha) \, \mathrm{d}\theta = \int_\theta \frac{\mathrm{P}(x,\theta)}{\mathrm{P}(\theta)} \frac{\mathrm{P}(\theta,\alpha)}{\mathrm{P}(\alpha)} \, \mathrm{d} \theta \quad \quad \ldots \text{(2)}$$

Note that in equation (1), there will be a $\mathrm{P}(x,\theta,\alpha)$ term, but in equation (2), I can't see how that term will appear.

Thanks.

1 Answers 1

2

We have, in general

$$ \mathrm{P}(x) = \int_\theta \mathrm{P}(x|\theta)\mathrm{P}(\theta) \, \mathrm{d}\theta$$

and conditioning everything on $\alpha$ :

$$ \mathrm{P}(x | \alpha) = \int_\theta \mathrm{P}(x|\theta \alpha) \mathrm{P}(\theta | \alpha) \, \mathrm{d}\theta$$

In the Bayesian setting, $\mathrm{P}(x|\theta \alpha)=\mathrm{P}(x|\theta)$ because, if we are given the parameter $\theta$ we know the density of $x$, and the values of $\alpha$ adds nothing ($\alpha$ only gives us information about $\theta$ - once we know it, they contribute nothing).

  • 0
    How do we know that $x$ depends on $\alpha$ only through $\theta$? In other words, where are we told that $\mathrm{P}(x|\theta, \alpha) = \mathrm{P}(x|\theta)$? Or is this an assumption?2012-08-24
  • 0
    It's an implicit assumption in the usual Bayesian setting, I'd say. Example: we are told that $x$ follows a uniform distribution in $[0,\theta]$. Further, the (in principle unkown) parameter $\theta$ is assumed to be (a priori) a random variable with exponential distribution, with parameter $\lambda$ (parameter of our parameter=hyperparameter). Then, it's evident (isnt it?) that knowing $\theta$ we don't need $\lambda$, so that $\mathrm{P}(x|\theta, \lambda) = \mathrm{P}(x|\theta)$2012-08-24