1
$\begingroup$

Essentially, my understanding of likelihood estimators is very weak, I've spent a while trying to get my head around it but the examples are all so different from each other that it's hard to find a clear method.

If someone could point me in the right direction for this question I'd very much appreciate it.

Question: Let $X$ be a random variable with parameter $\beta > 0$ and probability density function

$f_X(x)=\{ \frac{2^\beta\beta}{x^{\beta+1}},\quad x\geq2 \\ 0,\quad otherwise\\$

Let $x_1,x_2,...,x_n$ be a random sample from this distribution. Show that the log-likelihood for $\beta$ is

$ \ell(\beta,\textbf{x})=n\beta\log_e2+n\log_e\beta-(\beta+1)\sum\limits_{i=1}^n \log_ex_i $

and hence find the maximum likelihood estimator for $\beta$ and Fisher's information for $\beta$.

1 Answers 1

1

I just had to study that for an exam and I think I might be able to explain it to you, just the way I understood it. I am assuming you understand why people are interested in estimators;

You got yourself a sample $x = (x_1, \cdots, x_n) $. Note that each $x_i$ comes from the same distribution function, which is that $f_X(x) $. If you make the parameter $\beta $ change, the function $f_X(x) $ will start giving different values for the same input right? That makes sense: changing $f $'s formula would ought to make it a different function.

You want to find the value $\beta_0$ which is the most likely value for $\beta $ that would have produced your sample $x = (x_1, \cdots, x_n) $. That is, $\beta $ can have many values but some values are more likely to have produced your sample than others. Therefore, you want to maximize the probability that a given value for $\beta$ produced that sample!

An analogy: suppose there is some distribution function for a random variable $Y $, $f_Y(t) = k $ for some parameter $k $ you ate to estimate. Also, assume you have a sample which is $y = (3, 3, 3, 3) $. Well, you can pick any value for $k $; any number will do; but given that you know that sample $y $, it makes more sense to believe that $k = 3$ because then $f_Y(t) $ would produce your sample, than to believe that $k = -378$. If it were, how on earth did you get that sample? Thus it is not that likely that $k = -378$. That is why you write $\ell(\beta | x) $! "Find the most likely value of $\beta $ given that it must produce $x $".

So we want to maximize the probability of some $n $-sized sample coming out as the one you got. Let us say $s$ is an $n $-sized sample. You want to maximize

$$P(s = x) $$

But $P(s = x) = \prod_{i=1}^n P(s_i = x_i) $ given that all $x_i $ are independent. Now you rewrite those probabilities with the actual formula to yield

$$P(s = x) = \prod_{i=1}^n \frac{2^\beta\beta}{x_i^{\beta + 1}} $$

That product is simplifiable to

$$P(s = x) = \frac{\beta^n 2^{n\beta}}{(\prod x_i)^{\beta + 1}} $$

Now you can take the $\log$ of that, since the $\log$'s maximums will be the same as this function.

Can you then manipulate the $\log $ to achieve the desired result?

  • 0
    @N.K did this make sense? – 2017-01-13
  • 0
    It does make sense! Thank you. But how would the product fuction be turned in to a summation as the solution requires? Is there a rule of thumb when taking logs of products? – 2017-01-13
  • 0
    @N.K Yes there is! Remember that $\log{ab} = \log{a} + \log{b}, \log{a^k} = k\log{a}$. I though you were familiar with these? Try to use those two rules and turn that expression I got to into what you want. If you need further help, let us know! – 2017-01-13
  • 0
    Thank you so much! Just had a chance to attempt now, all makes sense, the product sign is what confused me initially but now I see I can still just treat it like any ordinary log product – 2017-01-16
  • 0
    I've been trying to finish off the question and I'm not sure if what I'm doing is correct or not. Essentially to get the maximum likelihood estimator I've found that you need to differentiate the log-likelihood, set to zero and then solve for the parameter. – 2017-01-16
  • 0
    After attempting I've come up with this: $\frac{d\ell(\beta,x)}{d\beta}=\frac{n}{\beta}+\frac{n\beta}{2}-\sum\limits_{i=1}^n ln(x_i)$ and after setting to equal $0$ and multiplying through I can solve $\hat \beta$ through the quadratic equation method. Would that then be acceptable to be considered as a maximum likelihood estimator for $\beta$? However, then you would get two possible estimators and I feel like that shouldn't happen... – 2017-01-16
  • 0
    @N.K I believe you got the derivative wrong! If you derive it correctly, you will get to an equation where there is only one possible value for $\beta$. After that, to prove that is indeed a maximum (because it could be a minimum as well) you can take the second derivative and show it is negative (if the first derivative is $0$ at the point $a$ and the second derivative is negative at that same point $a$, then $a$ is a point of a local maximum) – 2017-01-16
  • 0
    ah yes my mistake :) I believe the correct estimator should then be $\hat \beta = \frac{n}{\sum\limits_{i=1}^n \log_ex_i-n\log_e2}$? – 2017-01-16
  • 0
    @N.K yes indeed! – 2017-01-16