2
$\begingroup$

I'm confused by a bayesian way of calculating the probability $\text{Pr}(\theta \in (\theta_1, \theta_2) | y )$, where $\theta$ is assumed to have a (prior) uniform distribution on $[0,1]$, and the likelihood of the random variable $y$ is generated by a binomial sampling model (This example is taken from the textbook Bayesian Data Analysis). Using elementary rules of probability theory, we have

\begin{align} \text{Pr}(\theta \in (\theta_1, \theta_2) | y ) &= \frac{\text{Pr}(\theta \in (\theta_1, \theta_2), y )}{p(y)}\\ &= \frac{\int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)d\theta}{p(y)} \\ &= \frac{\int_{\theta_1}^{\theta_2}\binom{n}{y}\theta^y(1-\theta)^{n-y}d\theta}{p(y)}. \end{align}

I have two questions about the formula above:

  1. Why $\text{Pr}(\theta \in (\theta_1, \theta_2), y ) = \int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)d\theta$ ? What's wrong with this way: \begin{align} \text{Pr}(\theta \in (\theta_1, \theta_2), y ) &= \text{Pr}(y | \theta \in (\theta_1, \theta_2))\text{Pr}(\theta \in (\theta_1, \theta_2))\\ &= \int_{\theta_1}^{\theta_2}p(y|\theta)d\theta \int_{\theta_1}^{\theta_2}p(\theta)d\theta. \end{align}
  2. How to calculate the denominator, that is \begin{align} p(y) = \int_0^1 \binom{n}{y}\theta^y(1-\theta)^{n-y}d\theta. \end{align}

Thanks!

3 Answers 3

1

For the first question, be careful that in general $ \Pr(y|\theta \in (\theta_1,\theta_2)) \neq \int_{\theta_1}^{\theta_2}p(y|\theta)d \theta $ What is correct is $ \Pr(y,\theta \in (\theta_1,\theta_2)) = \int_{\theta_1}^{\theta_2}p(y,\theta)d \theta $

For the second question, you could use the beta function to compute that integral.

  • 0
    It is really nothing more than applying the law of total probability correctly http://en.wikipedia.org/wiki/Law_of_total_probability2012-12-31
1

For $1$: The first equality in your attempt is correct, but in the second one $\int_{\theta_1}^{\theta_2}p(y|\theta)\mathrm d\theta$ isn't the right way to calculate $\Pr(y|\theta\in(\theta_1,\theta_2))$; it should be

$ \Pr(y|\theta\in(\theta_1,\theta_2))=\frac{\int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)\mathrm d\theta}{\int_{\theta_1}^{\theta_2}p(\theta)\mathrm d\theta}\;, $

and if you use that you can cancel the denominator with the other factor and get $\int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)\mathrm d\theta$ after all.

For $2$, integrate by parts $y$ or $n-y$ times.

0
  1. In general $\text{Pr}(y | \theta \in (\theta_1, \theta_2)) \not = \int_{\theta_1}^{\theta_2}p(y|\theta)d\theta$: the right hand side increases as $\theta_2$ increases or as $\theta_1$ decreases but there is no reason to suppose the left hand side will. So your alternative will give the wrong answer.

  2. Your denominator should be $p(y) = {\int_{0}^{1}\binom{n}{y}\theta^y(1-\theta)^{n-y}d\theta} = \frac{1}{n+1}$ or whatever the actual limits on $\theta$ are (in other cases this may be $\pm \infty$), so that $\text{Pr}(\theta \in (\theta_1, \theta_2) | y )$ is actually a probability, something between $0$ and $1$ for all the possible values of $\theta_1$ and $\theta_2$. In this particular case it is a Beta function and with the prior distribution you were using the implication was that all integer values of $y$ from $0$ to $n$ were initially are equally likely.