MLE of probability distribution distributed on 3 points

Question

So this came up in class, as a "tough question for the more determined student" and I have no idea how to proceed.

the probability distribution is: $p(1; π_1, π_2) = π_1, p(2; π_1, π_2) = π_2$ and $p(3; π_1, π_2) = 1 − π_1 − π_2$ and 0 otherwise, where the parameters $π_1, π_2 \geq 0$ and $π_1 + π_2 \leq 1.$

The task is to find the MLE of the parameters $π_1, π_2.$

Any hints would be much appreciated - so far I have that the distribution can be written as $$0.5(X_i-2)(X_i-3)\pi_1 -(X_i-1)(X_i-3)\pi_2 +0.5(X_i-1)(X_i-2)(1-\pi_1-\pi_2)$$

Answer 1

I'm going to assume that you mean that the probability of your distribution being 1 given the values of $\pi_1$ and $\pi_2$ is $\pi_1$. Similar definitions for the distribution being 2 or 3. In addition, I will assume that there are $n_j$ observations of $j$, with $n_1+n_2+n_3=n$. Then we have the following:

\begin{align} L(\pi_1,\pi_2)&=\pi_1^{n_1}\pi_2^{n_2}(1-\pi_1-\pi_2)^{n_3}\\ l(\pi_1,\pi_2)&=n_1\ln\pi_1+n_2\ln\pi_2+n_3\ln(1-\pi_1-\pi_3)\\ l_1&=\frac{n_1}{\pi_1}-\frac{n_3}{1-\pi_1-\pi_2}=0\\ \frac{1}{1-\pi_1-\pi_2}&=\frac{n_1}{n_3\cdot\pi_1}\\ l_2&=\frac{n_1}{\pi_2}-\frac{n_3}{1-\pi_1-\pi_2}=0\\ \end{align} $l_1$ and $l_2$ are the partials with respect to $\pi_1$ and $\pi_2$, respectively. The last two lines yield a useful result.

\begin{align*} l_2&=\frac{n_1}{\pi_2}-\frac{n_1}{\pi_1}=0\\ \pi_1&=\frac{n_1}{n_2}\pi_2\\ \pi_2&=\frac{n_2}{n_1}\pi_1 \end{align*}

We use this substitution in our equation for $l_1=0$.

\begin{align*} \frac{n_1}{\pi_1}-\frac{n_3}{1-\pi_1-\frac{n_2}{n_1}\pi_1}&=\frac{n_1}{\pi_1}-\frac{n_1\cdot n_3}{n_1-(n_1+n_2)\pi_1}=0\\ \frac{1}{\pi}&=\frac{n_3}{n_1-(n_1+n_2)\pi_1}\\ n_1-(n_1+n_2)\pi_1&=n_3\pi_1\\ \hat{\pi}_1&=\frac{n_1}{n} \end{align*} Similarly, we can deduce that $\hat{\pi_2}=\frac{n_2}{n}$.