3
$\begingroup$

Let $u_1,\ldots,u_k$ be an orthonormal basis for the subspace $W \subset \mathbb{R}^m$. Let $A = (u_1u_2\ldots u_k)$ be the $m \times k$ matrix whose columns are the orthonormal basis vectors, and define $P=AA^T$ to be the corresponding projection matrix.

a.) Given $v \in \mathbb{R}^n$, prove that its orthogonal projection $w \in W$ is given by matrix multiplication $w=Pv$

I am trying to understand what exactly is going on in this question. Here is an illustration I found that I wish you guys can help me understand. Correct me if I am wrong in any of my analysis of this image.

enter image description here

On this image the subspace $W\subset \mathbb{R}^m$ is the plane. The orthogonal basis $u_1,\ldots,u_k$ is the vectors $u_1, u_2$. Given $v\in \mathbb{R}^n$ I am not sure what represents that in the image, will it be $h^*_N$? For $w\in W$, will it be the orthogonal vector $f$ in the image?

My attempt:

I notice that $P =AA^T$ is equivalent to the least square solution. Also, will $v\in \mathbb{R}^n$ be orthogonal to every vector in the subspace $W$, so then $v = w - u$, where $u = c_1u_1,\ldots,c_ku_k$?

I will have to prove that w (which will equal the vector $f$ in the image) was given by $w=Pv$? Or is $v\in \mathbb{R}^n$ just $A^T$ since it is in $\mathbb{R}^n$ and not in $\mathbb{R}^m$ (which also I am having trouble understanding what exactly does $\mathbb{R}^n$ and $\mathbb{R}^m$ mean?).

Any thorough explanation will be greatly appreciated!

  • 0
    @JoelCohen thanks for pointing that out. That helps a lot!2012-11-23

4 Answers 4

3

Note that $\vec{w} = \sum_{r=1}^{k} \alpha_r \vec{u}_r = A\begin{bmatrix} \alpha_1, \alpha_2, \ldots, \alpha_k\end{bmatrix}^T = A \vec{\alpha}$ Since $w$ is the orthogonal projection of $\vec{v}$, we have that $(\vec{v} - \vec{w}) \cdot \vec{u}_r = 0$ for all $\vec{u}_r$. Hence, we get that $(v - w)^T u_r = 0\implies (v-w)^T A = 0 \implies A^T(v-w) = 0 \implies A^T v = A^T w = A^T A \alpha$ Since the columns of $A$ are orthonormal, we have that $A^T A = I$. Hence, we get that $\alpha = A^Tv$. Hence, $w=A \alpha = A A^T v$

  • 0
    I understand now, thanks again!2012-11-28
1

Write $v$ as \begin{equation} v = w + w^{\perp}, \end{equation} so $w^{\perp}$ is orthogonal to $W$. Note that $w$ is in the column space of $A$, so $\exists \, c$ such that $w = Ac$. Hence \begin{equation} v = Ac + w^{\perp}. \end{equation} Now multiply both sides by $A^T$ to obtain \begin{align*} A^T v &= A^T A c + A^T w^{\perp}\\ &= c. \end{align*} It follows that $w = A A^T v$.

  • 0
    Thank you very much for explaining that, LittleO!2012-11-23
1

The natural way to project onto $W$ is by projecting into each of its "coordinates", which in this case are the $u_1,\ldots,u_k$. So $ Pw=\sum_{j=1}^k(u_j^Tw)\,u_j=\sum_{j=1}^k\,u_j(u_j^Tw)=\left(\sum_{j=1}^k\,u_ju_j^T\right)w=AA^Tw $

0

In terms of multi-linear algebra, think of the matrix $A$ as a direct sum of the $u_i$ and of its transpose as a direct sum of $\varphi_i$ (its duals), then that matrix multiplication is just a direct sum of the terms of the form $u_i \otimes \varphi_i$, each of which is a projection onto the subspace span($u_i$).