1
$\begingroup$

Assume $X_i\sim Exp(\lambda)$ are iid random variables. Also, for $1=1,\ldots,n$ let $Y_i=I\lbrace X_i>c_i\rbrace$ where the thresholds $c_1,\ldots,c_n$ are known constansts in $(0,\infty)$.

a) Derive the EM recursion to compute the MLE of $\lambda$ based on $Y_1,\ldots,Y_n$.

b) Give the first to iterates, $\hat{\lambda}_1$ and $\hat{\lambda}_2$, if the initial guess is $\hat{\lambda}_0=1$ and there are 3 observations: $Y_1=1$, $Y_2=1$ and $Y_3=0$ with $c_1=1$, $c_2=2$ and $c_3=3$.

I'm supposed to present this exersice in class. I read the chapter on the EM algorithm but there were no examples provided so I don't understand how to do the exercise. Any help is greatly appreciated :)

1 Answers 1

1

The EM algorithm has the two steps. First you find the expected log likelihood (where the expectation is taken under the current parameters and conditional on whatever data you can see) and then you adjust the parameters in the likelihood function to maximize this, and then iterate using these new parameter values.

You've probably read that much so I guess I'll just go through it slowly on this example.

Here your observed data is $Y_i = I(X_i>c_i),$ in other words you only know which variables were greater than their thresholds.

You are looking for the log-likelihood. For exponential data this is just $$ l(\lambda;X_i) = n\ln(\;\lambda) - \lambda\sum_i X_i$$

Now we need to take the expected value of this under the current parameter $\lambda_t$ and conditional on our observations $y_i$ of the $Y_i.$ The expected value is $$ E(l(\lambda;X_i)|Y_i=y_i,\lambda_t) = n\ln(\lambda)-\lambda\sum_iE(X_i|Y_i=y_i,\lambda_t).$$

This is maximized at $$ \lambda_{t+1} = \frac{n}{\sum_iE(X_i|Y_i=y_i,\lambda_t)}$$

So to finish, we need to compute the conditional expected values of the $X_i.$ If $y_i=0$ that means $X_i

We can write this succinctly as $$ E(X_i|Y_i=y_i,\lambda_t) = \frac{1}{\lambda_t}\left(1+ y_i\lambda_tc_i + (y_i-1)\left(\frac{\lambda_tc_i}{e^{\lambda_t c_i}+1}\right)\right).$$

So we sum this over $i$ and then plug into the expression $$ \lambda_{t+1} = \frac{n}{\sum_iE(X_i|Y_i=y_i,\lambda_t)}$$

to get the new value $\lambda_{t+1}.$ And then, we iterate. Do it all over again for using the starting value $\lambda_{t+1}$ rather than $\lambda_t.$

  • 0
    Makes sence. Does that mean – 2017-02-14
  • 0
    Makes sence. Does that mean the first "iterate" is $$\lambda_1=\frac{3}{(1+1(1+1*1)+0+1+1(1+1*2)+0+1-1*3/(e^{3}+1))}=\frac{3}{(3+4+1-3/(e^{3}+1)}\approx 0.382$$? – 2017-02-14
  • 0
    @NiclasEnglesson Uhh that seems a little low based on the observations, although the upshot of what you're doing is right. I'm not sure about why there's so many ones in the denominator. – 2017-02-14
  • 0
    @NiclasEnglesson Ahh I see what happened, I had a typo in the 'succinct version'. You shouldn't use this version for hand calculations like this anyway. – 2017-02-14