1
$\begingroup$

Original problem $Ax=b$ (let's say A is 400x5 matrix)

$x$ can be found through SVD. Now we can create a new problem that will provide us with the same solution for x':

I_5 x' = x

Where $I_5$ is a 5x5 identity matrix

Let's now extend the problem by adding more rows to $A$ and $b$. Calling the new variables A' and b'. Same logic can then be applied: A' x' = b'

However, at this point we can notice that since the new values were simply extensions of the previous problem, then we can write A' as

A' =[\begin{matrix} A \\ A_{added} \end{matrix}]


From here I'd like to substitude the original $A$ with an identity matrix, and the values in $b$ with the original $x$, creating

\begin{matrix} I_5 \\ A_{added} \end{matrix} x' = \begin{matrix} x \\ b_{added} \end{matrix}

Question: Why does this not provide me with the same x' as solving the original "extended" problem? How can I achieve this reduction/compression of the matrix?

I can only assume it's some sort of "weighting" issue, but I don't even know where to begin. Should the identity matrix be multiplied by a constant? Or not a constant?

Thanks

2 Answers 2

1

There are a few places where this goes wrong.

  1. There's indeed the weighing issue.

  2. When you try to add some more data to the system of equations, you merely use 'x' and 'I' of the first dataset. This makes the implicit assumption that the 5 columns of 'x' are perfectly uncorrelated, that is most likely not the case.

You might want to try another approach, see for example: this one

  • 0
    maybe some of the [other answers](http://stats.stackexchange.com/questions/6920/efficient-online-linear-regression/8681#8681) (apologies for linking to my own stuff) are easier to follow ?2011-03-31
0

If your original matrix $A$ is $400\times5$, then your original system is usually strongly overdetermined. When you say $x$ can be found through SVD, do you have reason to assume that the system is consistent and determined? If so, your replacement system should have the same solution set as the original system, since you've merely replaced a set of $400$ consistent equations determining $x$ by a more economic set of $5$ equations determining it. However, it's not clear to me a) why you might have $400$ equations $395$ of which are redundant, and b) what the added equations are supposed to mean, since x' is already determined. (If the original system is actually, despite appearances, under-determined, then indeed arbitrarily picking a solution from the solution space and fixing it in the augmented system will change the solution space, since the added equations would probably have picked (a) different solution(s).)

  • 0
    Yes, original system is grossly overdetermined. I'm looking for a best-fit solution. If I remember definition of consistent - then it's not consistent, since RMSD of $Ax-b$ is not 0.2011-03-28