2
$\begingroup$

I'm reading about Bayesian data analysis by Gelman et al. and I'm having big trouble interpreting the following part in the book (note, the rat tumor rate $\theta$ in the following text has: $\theta \sim Beta(\alpha, \beta)$):

Choosing a standard parameterization and setting up a ‘noninformative’ hyperprior dis- tribution. Because we have no immediately available information about the distribution of tumor rates in populations of rats, we seek a relatively diffuse hyperprior distribution for $(\alpha, \beta)$. Before assigning a hyperprior distribution, we reparameterize in terms of $\text{logit}(\frac{\alpha}{\alpha+\beta}) = \log(\frac{\alpha}{\beta})$ and $\log(\alpha+\beta)$, which are the logit of the mean and the logarithm of the ‘sample size’ in the beta population distribution for $θ$. It would seem reasonable to assign independent hyperprior distributions to the prior mean and ‘sample size,’ and we use the logistic and logarithmic transformations to put each on a $(-\infty, \infty)$ scale. Unfortunately, a uniform prior density on these newly transformed parameters yields an improper posterior density, with an infinite integral in the limit $(\alpha+\beta) \rightarrow \infty$, and so this particular prior density cannot be used here.

In a problem such as this with a reasonably large amount of data, it is possible to set up a ‘noninformative’ hyperprior density that is dominated by the likelihood and yields a proper posterior distribution. One reasonable choice of diffuse hyperprior density is uniform on $(\frac{\alpha}{\alpha+\beta}, (\alpha+\beta)^{-1/2})$, which when multiplied by the appropriate Jacobian yields the following densities on the original scale,

$$p(\alpha, \beta) \propto (\alpha+\beta)^{−5/2},$$

and on the natural transformed scale:

$$p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right)\propto \alpha\beta(\alpha+\beta)^{−5/2}.$$

My problem is especially the bolded parts in the text.

Question (1): What does the author explicitly mean by: "is uniform on $(\frac{\alpha}{\alpha+\beta}, (\alpha+\beta)^{-1/2})$"

Question (2): What is the appropriate Jacobian?

Question (3): How does the author arrive into the original and transformed scale priors?

To me the book hides many details under the hood and makes understanding difficult for a beginner on the subject due to seemingly ambiguous text.

P.S. if you need more information, or me to clarify my questions please let me know.

1 Answers 1

4

I figured the solution out myself so I'm gonna share it here if anyone is going to bump into the same part in Gelman's book (pages 110-111).

Answer (1):

The author simply means by this that $$p\left(\frac{\alpha}{\alpha+\beta}, \;(\alpha+\beta)^{-1/2}\right)=\text{constant}\propto 1.$$

Answer (2):

When the author talks about "appropriate Jacobian" he's talking about the determinant of the Jacobian matrix in the change of variables formula for density functions:

$$p(\phi)=p(\theta)\,{\color{blue}{\det\left(\frac{d\theta}{d\phi}\right)}}$$

Answer (3):

The author simply applies the change of variables formula two times. We know that $$p(\gamma, \delta) = p(\gamma(\alpha, \beta),\, \delta(\alpha, \beta))=p\left(\frac{\alpha}{\alpha+\beta}, \;(\alpha+\beta)^{-1/2}\right)=\text{constant}\propto 1.$$

If we denote $\theta=(\gamma, \delta)$ and $\phi=(\alpha, \beta)$, then:

$$\det\left(\frac{d\theta}{d\phi}\right)=\left|\begin{array}{cc} \frac{d\gamma}{d\alpha} & \frac{d\gamma}{d\beta} \\ \frac{d\delta}{d\alpha} & \frac{d\delta}{d\beta}\end{array}\right|=\left|\begin{array}{cc} \frac{\beta}{(\alpha+\beta)^2} & -\frac{\alpha}{(\alpha+\beta)^2} \\ -\frac{1}{2(\alpha+\beta)^{3/2}} & -\frac{1}{2(\alpha+\beta)^{3/2}}\end{array}\right|=-\frac{1}{2(\alpha+\beta)^{5/2}}.$$

From change of variables formula we get:

$$p(\alpha, \beta) = \underbrace{p\left(\frac{\alpha}{\alpha+\beta}, \;(\alpha+\beta)^{-1/2}\right)}_\text{= constant $\propto \;1$}\cdot \left(-\frac{1}{2(\alpha+\beta)^{5/2}}\right)\propto (\alpha+\beta)^{-5/2},$$

and there it is, i.e. the prior in original scale.

For the alternative scale, by using change of variables in exactly the same manner:

$$p(\alpha, \beta) = p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right)\,\det\left(\frac{d\theta}{d\phi}\right),$$

where this time $\gamma(\alpha, \beta) = \log\left(\frac{\alpha}{\beta}\right)$ and $\delta(\alpha, \beta) = \log(\alpha+\beta)$. For the Jacobian determinant we get:

$$\det\left(\frac{d\theta}{d\phi}\right)=\left|\begin{array}{cc} \frac{d\gamma}{d\alpha} & \frac{d\gamma}{d\beta} \\ \frac{d\delta}{d\alpha} & \frac{d\delta}{d\beta}\end{array}\right|=\left|\begin{array}{cc} 1/\alpha & -1/\beta \\ (\alpha+\beta)^{-1} & (\alpha+\beta)^{-1}\end{array}\right|=\frac{1}{\alpha\beta},$$

so we get:

$$\underbrace{p(\alpha, \beta)}_\text{$\propto\, (\alpha+\beta)^{-5/2}$} = p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right)\,\frac{1}{\alpha\beta},$$

or

$$ p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right) \propto \alpha\beta(\alpha+\beta)^{-5/2}.$$