2
$\begingroup$

I need to show that $\bar h= \sum{h_{ii}/n} = \operatorname{Tr}[H]/n = (p+1)/n$

Using the fact that $\operatorname{Tr}[AB]=\operatorname{Tr}[BA]$ and $H=X(X^TX)^{-1}X^T$.

But I have no idea how to calculate $\bar h$, I'm betting the first equality works out because $H$ is a symmetric idempotent matrix. I also have no clue what $\operatorname{Tr}[H]$ means, I have never seen this notation before an cannot find it in my notes.

1 Answers 1

3

The trace of a matrix is the sum of its diagonal entries. The function $A\mapsto \text{Tr}(A)$ has the property that $\text{Tr}(AB)=\text{Tr}(BA)$ for any two matrices of compatible size.

So, if $X$ is of size $n\times(p+1)$, then $X^TX$ is a $(p+1)\times(p+1)$ matrix and $$ \text{Tr}(H)=\text{Tr}(X(X^TX)^{-1}X^T)=\text{Tr}((X^TX)^{-1}X^TX)=\text{Tr}(I_{p+1})=p+1. $$

So $\sum_{H_{jj}}/n=\text{Tr}(H)/n=(p+1)/n$.

  • 0
    Sorry!!! Typically in class we define P as the highest subscript of $\beta$. So $\beta_0, \beta_1, ..., \beta_p$2012-11-24
  • 1
    From what I recall, $X^T X$ has dimension $(p+1) \times (p+1)$, i.e. dimension of the vector $\beta$. So that $\operatorname{Tr}(H)$ should be equal to $(p+1)$, not $n$.2012-11-24
  • 1
    @MartinArgerami : It is conventional in dealing with certain design matrices in statistics to let $p$ be the number of columns other than one column whose every entry is $1$. Hence $X$ has $p+1$ columns, and typically a much larger number $n$ of rows than columns. So the "hat matrix" $H$ is an $n\times n$ matrix of rank $p+1$. If $Y\in\mathbb{R}^{n\times1}$ then $HY$ is the orthogonal projection of $Y$ onto the column space of $X$.2012-11-24
  • 1
    There is a grave error in this answer. If $X\in\mathbb R^{n\times(p+1)}$ then the identity matrix at the end is a $(p+1)\times(p+1)$ identity matrix. Its trace is $p+1$.2012-11-24
  • 0
    @Carly : Please: Don't write capital $P$ when you mean lower-case $p$. Mathematical notation is case-sensitive.2012-11-24
  • 0
    @Michael: I have modified the answer according to your comment. But note that nowhere in the question is anything suggested about the size of $X$.2012-11-24
  • 0
    It's not explicit in the question, but Carly's comment suggests that some usual conventions are followed.2012-11-25