10
$\begingroup$

I have a positive definite matrix $A$. I am going to choose its Frobenius norm $\|A\|_F^2$ as a cost function and then minimize $\|A\|_F^2$. But I think I need to find a reason to convince people it is reasonable to choose $\|A\|_F^2$ as a cost function. So I'm wondering if there are some geometric meanings of the Frobenius norm. Thanks.

Edit: here $A$ is a 3 by 3 matrix. In the problem I'm working on, people usually choose $\det A$ as a cost function since $\det A$ has an obvious geometric interpretation: the volume of the parallelepiped determined by $A$. Now I want to choose $\|A\|_F^2$ as a cost function because of the good properties of $\|A\|_F^2$. That's why I am interested in the geometric meaning of $\|A\|_F^2$.

  • 2
    $\|A\|_F^2$ is equal to summation of the squre of all singular values of $A$. Perhaps you could start from the geometric meaning of singular values of a matrix.2011-07-28
  • 0
    Since all matrix norms are equivaluent, it doesn't matter which norm you choose as a cost function.2011-07-28
  • 1
    @Sunni: For the problem I'm working on, people usually choose $\det A$ as the cost function as $\det A$ has an obvious geometric meaning: the volume of the parallelepiped determined by $A$. I know $\|A\|_F^2=trace (AA^T)=\sum\sigma_i^2$. But what is the geometric meaning of the sum of singular values?2011-07-28

2 Answers 2

21

To give intuition, lets look at a particularly nice class of matrices. Suppose the matrix $A$ is symmetric. Then we can find a basis of $n$ orthonormal eigenvectors $v_1,\cdots v_n$ with eigenvalues $\lambda_1,\dots,\lambda_n$. Each eigenvalue geometrically represents the "stretching factor" in the direction of its associated eigenvector.

The Frobenius norm of $A$ in this case is $$\|A\|_F =\sqrt{\text{Tr}(AA^t)}=\sqrt{\text{Tr}(A^2)}=\sqrt{\lambda_1^2+\cdots+\lambda_n^2}.$$ Working in the basis $v_1,\cdots v_n$, consider the box which is the image of the unit cube under $A$. The Frobenius norm is the diagonal of that box, and the determinant is the volume.

The usual norm defined as $\sup_{\|x\|=1}\|Ax\|$ corresponds to the longest side of the box.

  • 0
    Very interesting. If not working in the basis $v_1,...,v_n$, $A$ is corresponding a parallelepiped. Then $\det A$ is still the volume of the parallelepiped, can we say $\|A\|_F$ is still the diagonal of the parallelepiped?2011-07-28
  • 0
    @Shiyu: By SVD, yes, but it will be the diagonal of a parallelepiped that's of smaller dimension than the ambient space.2011-07-28
  • 0
    @Eric this is quite nice explanation, however I am not sure what you mean by "consider the box which is the image of the unit cube under A"... What? :-) I _think_ I know what you mean... let us say you have a skewed box, so the Fro-norm is the longest diagonal within the box that touches the corners, while the determinant is the volume of this box?2012-08-30
  • 0
    @Mohammad: Well it is more like an $n$-dimensional parallelogram than a box if that helps.2012-08-30