2
$\begingroup$

Related to my work is the concept of Singular Value Decomposition (SVD). Namely, given some matrix $B\in\mathbb{R}^{n\times m}$, $n\geq m$, SVD can be written as $B=U\Sigma V^T,$ where $U\in\mathbb{R}^{n\times n}$ ($n$ left-singular vectors), $\Sigma \in\mathbb{R}^{n\times m}$ (matrix with $m$ singular values on its diagonal), and $V\in\mathbb{R}^{m\times m}$ ($m$ right-singular vectors). So, how I see it, given that there are $m$ singular values, only right-singular vectors and $m$ first left-singular vectors have the asociated singular value? To which singular value do the rest of left-singular vectors correspond to?

In case one complements $\Sigma$ to contain zero singular values such that $\Sigma\in\mathbb{R}^{n\times n}$, then $V^T$ needs also to be complemented to be a $V^T\in\mathbb{R}^{n\times m}$ matrix. But, that would imply that a vector of all zeros is also a right-singular vector. Could someone clarify this?

Furthermore, suppose that the rank of matrix $B$ is $c$, $c. Does this imply that $B$ has only $d$ left- and right-singular vectors?

  • 0
    You have to also remember that there are two different versions of the SVD. The "thin"/"compact"/"economy" SVD is the version where the orthonormal matrix $\mathbf U$ has the same dimensions as the original matrix, and $\mathbf \Sigma$ is a square diagonal matrix. The one you are talking about in the OP is the "full" SVD, and chaohuang has already told you how the full SVD is related to the eigendecomposition of the cross-product matrix.2012-07-22

2 Answers 2

2

The left singular vectors of $B$ are eigenvectors of $BB^T$, and the right singular vectors of $B$ are eigenvectors of $B^TB$, so you can easily see why there are $n$ left singular vectors, and $m$ right singular vectors.

If $n \ge m$, then there are only $m$ singular values, which are the square root of eigenvalues of $BB^T$. If $m \ge n$, then there are only $n$ singular values, which are the square root of eigenvalues of $B^TB$. So the rest of left singular vectors do not correspond to singular values, there are the eigenvectors for the 0 eigenvalue of $BB^T$.

We always have $\Sigma \in\mathbb{R}^{n\times m}$, not $\Sigma \in\mathbb{R}^{n\times n}$ if $n > m$. Zero vectors are not singular vectors.

There are always $n$ left singular vectors, and $m$ right singular vectors.

  • 0
    @user506901 if $rank(A)=d$, then you are right.2012-07-23
0

In practice, when you want to find $U$ and $V$, you do the following,

Apply $B^{T}$ to both sides of the equqtuation $ B = U\Sigma V^{T} $. That gives you the equation $ B^{T} B = {(U\Sigma V^{T})}^{T} (U\Sigma V^{T}) \Rightarrow V\Sigma U^{T} U\Sigma V^{T}.$ Since $U$ is an orthogonal matrix then

$ U^{T}U=I \implies B^{T}B = V\Sigma^2 V^T \,.$

The last equation is nothing, but a diagonalization problem for the square matrix $B^T B$. You can do the same for $U$.

  • 0
    @Downvoter: What's the downvote for?2013-08-26