3
$\begingroup$

Background

Suppose that the national mint mints coins with bias $p_i \sim \rm{Beta}(A,B)$ for some unknown constants $A$, $B$.

Given $n$ coins, you flip each coin a certain number of times. Coin $i$ comes up heads $a_i$ times and tails $b_i$ times. (The flips are clearly exchangeable, so we don't need the actual sequence of flips.)

If I did the math right, I figured out that the likelihood of $A, B$ given the statistics $(a_i, b_i)$ is:

$L(A,B \mid a_i, b_i) = \frac{\prod_i\rm{Beta}(A + a_i, B + b_i)}{\rm{Beta}(A, B)^n}.$

(Edit in 2012: This is the product of $n$ Pólya–Eggenberger urn schemes.)

I would like to summarize the statistics $(a_i, b_i)$ even further, into a finite set of numbers if possible.

Question

Can $\prod_i\Gamma(A + a_i)$ be written using sufficient statistics calculated from the $a_i$?

Example

For example, $\prod_i\exp(A + a_i)$ can be written $\exp(nA + S)$ where $n, S$ are sufficient statistics calculated from the $a_i$.

  • 0
    If this question interests you, please help me tag it.2011-06-28

1 Answers 1

2

I think I would agree with your expression for the likelihood, although this approach is not without problems (see below).

Notice, that since $a_i$ and $b_i$ are integers, the likelihood is just a rational function in $A$ and $B$: $ L(A,B \mid a,b) = \prod_{i=1}^n \left( \frac{ \prod\limits_{k=0}^{a_i-1} (A+k) \prod\limits_{m=0}^{b_i-1} (B+m) }{\prod\limits_{j=0}^{a_i+b_i-1} (A+B+j)}\right) $ The maximum likelihood equations are easily seen to be $ \begin{eqnarray} \sum_{i=1}^n \sum_{k=0}^{a_i-1} \frac{1}{A+k} &=& \sum_{i=1}^n \sum_{j=0}^{a_i+b_i-1} \frac{1}{A +B+j} \\ \sum_{i=1}^n \sum_{m=0}^{b_i-1} \frac{1}{B+m} &=& \sum_{i=1}^n \sum_{j=0}^{a_i+b_i-1} \frac{1}{A +B+j} \end{eqnarray} $ Given that $A>0$ and $B>0$ these equations become polynomial equations in $A$ and $B$ with, hopefully, a unique positive root. However, checks in Mathematica for small values of $n$ and $a_i$, $b_i$ indicate that the resulting system admits no positive finite solutions. enter image description here

It is easily seen, that infinite $A$ and $B$ solve MLE equations. Let's fix $r=A/B$. Then, in the limit of infinite $A$ and $B$, equations reduce to $ \begin{eqnarray} \sum_{i=1}^n a_i &=& \sum_{i=1}^n \frac{r}{r+1} (a_i+b_i) \\ \sum_{i=1}^n r b_i &=& \sum_{i=1}^n \frac{r}{r +1} \cdot (a_i+b_i) \end{eqnarray} $ which gives $ r = \frac{\sum_{i=1}^n a_i}{\sum_{i=1}^n b_i} $

This result is confirmed numerically by the following heuristic. Use $a_i$ and $b_i$ to estimate head probability $p_i$ for each coin using ML estimator $p_i = \frac{a_i}{a_i+b_i}$. Then use ML estimation for beta distribution to determine $\alpha$ and $\beta$ from the data $\{p_1, \ldots, p_n \}$.

enter image description here

I do not have a good explanation as to why ML estimation can not seem to determine both $\alpha$ and $\beta$. This may be worth a separate question.