1
$\begingroup$

A real matrix $X$ is semi-orthogonal if: \begin{align} X^TX = I \end{align} where $^T$ denotes the transpose operator, and $I$ is an identity matrix.

I am currently studying a system $y = Xg$, where $X$ is known, $y$ is known, and $g$ is unknown. We want to estimate $g$ by: \begin{align} \hat{g} = X^T y = X^T X g \end{align}

Notice that if $X^TX = I$, then the estimate is perfect: $\hat{g} = g$.

However, in general $X$ is not semi-orthogonal and the estimate is not perfect. I would like to find a metric that tells me how good the estimate is. I'm thinking of calculating a metric that describes the semi-orthogonality of $X^TX$.

There are all sorts of matrix norms I could apply to calculate a value for $\|X^T X - I \|$, and this would partially capture the extent to which $X$ is not semi-orthogonal. My question is: are many matrix norms equally appropriate in this scenario, or is there some "canonical" metric used for calculating the extent to which a matrix fails to be semi-orthogonal? If you suggest a metric or matrix norm, please explain why your choice is not arbitrary.

  • 0
    What you are doing is the classical least square estimate. – 2017-01-20
  • 0
    That is helpful to know. Any additional thoughts are appreciated. – 2017-01-20

1 Answers 1

2

You have to decide which vector norm you want to use in order to tell how good the estimate $\hat g$ is. (The statement "I would like to find a metric that tells me how good the estimate is" means that you have a concept of measuring the quality of $\hat g$, i.e. a vector norm for evaluating $\|g-\hat g\|$). Simply take the matrix norm that is induced by the vector norm that you had in mind to evaluate the quality of $\hat g$. With this matrix norm, you have $$\|\hat g-g\| = \|X^TXg-g\|=\|(X^TX-I)g\|\leq \|X^TX-I\|\cdot\|g\|$$ Therefore, $\|X^TX-I\|$ can indeed be used to get an upper bound for $\|\hat g-g\| / \|g\|$

As already mentioned by Artem, the problem is related to the least square approximation, i.e. the attempt to find $g$ that minimizes $\|Xg-y\|_2$. This problem can be solved by solving $X^TXg=X^Ty$, if the inverse matrix of $X^TX$ exists. Because of the relationship with the least square approximation, it might be appropriate to use the Euclidian norm as vector norm and the spectral norm as matrix norm for your attempt to find out how good the estimate is. The drawback is the effort for calculating the spectral norm. The spectral norm can be estimated by the Frobenius norm ($\|A\|_2 \leq \|A\|_F$), which can easily be evaluated. So we have $$\|\hat g-g\|_2 \leq \|X^TX-I\|_2\cdot\|g\|_2 \leq \|X^TX-I\|_F\cdot\|g\|_2$$