1
$\begingroup$

We are learning matrix calculus in class. This is really new for me and I am trying to get extra practice by going through old assignments. One of the questions asks to find the derivative of the function below with respect to $\beta$ and if possible to find the value of the elements of $\beta$ that maximize the function. After taking logs to simplify the original function, I took the partial with respect to $\beta$. I thought my derivative was correct, but when I set the partial equal to zero, I am unable to solve for $\beta$. So either my derivative is wrong or I don't know enough to solve for $\beta$, or it's a trick question and there is no analytical expression for $\beta$. A more advanced student suggested that it can probably only be solved by numerical methods but I was wondering if anyone here can tell me if my derivative is correct and whether or not it's possible to solve for $\beta$?

$ x_{1} = \begin{pmatrix} a\\ b\\ \end{pmatrix}\quad x_{2} = \begin{pmatrix} c\\ d\\ \end{pmatrix}\quad \beta = \begin{pmatrix} \beta_{1}\\ \beta_{2}\\ \end{pmatrix}\quad Y = \begin{pmatrix} y_{1}\\ y_{2}\\ \end{pmatrix}\quad $

f(x_{1},x_{2},y_{1},y_{2},\beta) = \frac{1}{x_{1}'\beta}\exp\left\{\frac{-y_{1}}{x_{1}'\beta}\right\}\frac{1}{x_{2}'\beta}\exp\left\{\frac{-y_{2}}{x_{2}'\beta}\right\}

My answer:

\frac{\partial \ln f}{\partial\beta}=-\frac{1}{x_{1}'\beta}x_{1}-\frac{1}{x_{2}'\beta}x_{2}+\frac{x_{1}y_{1}}{x_{1}'\beta.x_{1}'\beta}+\frac{x_{2}y_{2}}{x_{2}'\beta.x_{2}'\beta}=0

Is this correct and is it possible to solve for $\beta$? If so, how? Thanks!

  • 0
    Two vectors $x_1$ and $x_2$ are by definition linearly independent if $c_1x_1+c_2x_2=0$ implies $c_1=c_2=0$; that's what I meant by "the coefficients have to vanish individually". Since you can write your equation in that form, it follows that the coefficients multiplying $x_1$ and $x_2$ must both be zero; this is what gives you two equations; the fact that they're linear in $x_i'\beta$ will become apparent when you write them down and cancel common factors. Note, however, that this cancellation is only valid if $x_i'\beta\ne0$, and these are precisely the values where the function diverges.2012-01-18

1 Answers 1

1

This function doesn't have a global maximum, at least not if $x_1$ and $x_2$ are linearly independent: You can choose x_1'\beta such that the first exponential goes to $\infty$, and you can independently choose the sign of x_2'\beta such that the product is positive and thus goes to $\infty$.

However, you can still look for the stationary points and see whether they are local maxima. Taking the logarithm before taking the derivative is a good approach, and the resulting equation can be solved with nothing but linear algebra.

If $x_1$ and $x_2$ are linearly independent, their coefficients in this equation have to vanish individually. That gives you one linear equation for x_1'\beta and one for x_2'\beta. You can solve these, and then knowing the scalar products of $\beta$ with two linearly independent vectors, you can reconstruct $\beta$.

If $x_1$ and $x_2$ aren't linearly independent, you can substitute $x_2=\lambda x_1$ into the equation, and setting the coefficient of $x_1$ in the result to $0$ gives you a linear equation for x_1'\beta. In this case the stationarity condition only restricts $\beta$ in one direction, since the function is independent of the component of $\beta$ orthogonal to $x_1$ and $x_2$.