least-squares estimate for simple 3x3 singular matrix with constraints

Question

I have some measurements $M_1, M_2, M_3$ and some parameters $a,b,c$ I want to estimate. They are related as follows:

$$\begin{bmatrix}M_1 \\ M_2 \\ M_3\end{bmatrix} = \begin{bmatrix}1 & -1 & 0 \\ 0 & 1 & -1 \\ -1 & 0 & 1\end{bmatrix} \begin{bmatrix}a \\ b \\ c\end{bmatrix} $$

where the 3x3 matrix is singular (no information about $a+b+c$), but I also know that $a \approx 1$ and $b \approx 0$ and $c \approx 0$, so it seems like I would want to minimize $(a-1)^2 + b^2 + c^2$.

How would I solve for $a,b,c$ in a least-squares sense, given this information? I know how to use least-squares estimates without constraints, in an overdetermined system, but not sure what to do here.

Answer 1

You can solve regularized Least Squares System:

$$ \arg \min \left\| A x - b \right\|_{2}^{2} + \lambda \left\| x - d \right\|_{2}^{2} $$

Where $ A $ is the matrix above, $ b $ is the measurements vector and $ d = \left[ 1, 0, 0 \right]^{T} $.
The parameter $ \lambda $ states you assurance in the model you supplied, namely how close $ x $ is to the vector $ d $.

The solution is given by:

$$ \hat{x} = \left( {A}^{T} A + \lambda I \right)^{-1} \left( {A}^{T} b + \lambda d \right) $$

As can be seen for $ \lambda = 0 $ the solution will be the least squares solution and for $ \lambda \to \infty $ you'll get $ \hat{x} = d $.