2
$\begingroup$

Let $X \in \mathbb{R}^{n×m}$ be a matrix such that $n \ge m$. Let $y \in \mathbb{R}^n$ be a vector such that $y \sim N (Xβ, σ^2I)$. Recall from the lectures that the MLE of $\beta$ is given by $\hat\beta = (X^TX)^{−1}X^TY$

(a) Explain why it is necessary for $n \ge m$. having trouble explaining this any help would be appreciated

1 Answers 1

1

You don't fit a line with only one data point nor a plane with only two. Fitting a plane when you have only two data points is the situation where $n=2$ and $m=3$ and you have $$ X\beta + (2\times 1\text{ error vector}) = \begin{bmatrix} 1 & x_1 & y_2 \\ 1 & x_2 & y_2 \end{bmatrix} \begin{bmatrix} \beta_1 \\ \beta_2 \\ \beta_3 \end{bmatrix} + (2\times 1\text{ error vector}) = \begin{bmatrix} y_1 \\ y_2 \end{bmatrix} = Y. $$ There is more than one vector $\beta$ that fits perfectly, since a plane through two points can rotate about an axis that is the line through those two points. The vector $Y$ can be written in more than one way as a linear combination of the columns of $X$.

The matrix $X^TX$ in such a case is a $3\times3$ matrix of rank $2$, so it cannot be inverted.