0
$\begingroup$

If you have a matrix:

\begin{pmatrix} 2 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & 4 \\ \end{pmatrix}

The eigenvalues are $2,4,4$. The eigenvectors are $(0,0,1)$, $(0,1,0)$ and $(1,0,0)$. What is the first principal components, second principal components, and third principal components? Is it just $Y_1 = X_1$, $Y_2 = X_2$ and $Y_3 = X_3$?

1 Answers 1

1

Note that your matrix is $\begin{pmatrix} 2 & 0 & 0\\ 0 & 4 & 0\\ 0 & 0 & 4 \end{pmatrix} = 4 \begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix} \begin{pmatrix} 0 & 0 & 1 \end{pmatrix} + 4 \begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix} \begin{pmatrix} 0 & 1 & 0 \end{pmatrix} + 2 \begin{pmatrix} 1 \\ 0 \\ 0\end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \end{pmatrix}$ In general, the idea is to get the singular value decomposition of a matrix $A \in \mathbb{C}^{n \times n}$ as $A = \sigma_1 u_1 v_1^* + \sigma_2 u_2 v_2^* + \cdots + \sigma_n u_n v_n^*$ where $u_k,v_k \in \mathbb{C}^{n \times 1}$, $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_n \geq 0$, $u_k^* u_k = 1$ and $v_k^* v_k = 1$. The first $\ell$ principal components are the vectors $v_1,v_2,\ldots, v_{\ell}$.

(Typically when you talk of principal component analysis, the underlying matrix is Hermitian and hence $u_k = v_k$. So you could either consider $u_k$'s or $v_k$'s.)

In your case,

$\text{The first principal component is either $\begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix}$ or $\begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix}$}$

$\text{The first two principal component are $\begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix}$ and $\begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix}$}$

$\text{The three principal components are $\begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix}$, $\begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix}$ and $\begin{pmatrix} 1 \\ 0 \\ 0\end{pmatrix}$}$

EDIT You have the eigenvectors and eigenvalues paired up in a wrong order. The eigenvector corresponding to the eigenvalue $2$ is $\begin{pmatrix} 1 \\ 0 \\ 0\end{pmatrix}$ and similarly the others.