0
$\begingroup$

I have a function of the form $ E(\mathbf{x})=E_1(\mathbf{x})+E_2(\mathbf{x}) =\sum_i\|\ldots\|^2+\sum_j\|\ldots\|^2 $ and want to solve the optimization problem $ \text{argmin}_{\mathbf{x}} E(\mathbf{x}) $ using linear least-squares. So, in order to build the linear equation system $A\mathbf{x}=\mathbf{b}$ one could choose between two strategies

  1. $\frac{\partial E(\mathbf{x})}{\partial x_m}=0$ and add one equation to $A$
  2. $\frac{\partial E_1(\mathbf{x})}{\partial x_m}=0$ and $\frac{\partial E_2(\mathbf{x})}{\partial x_m}=0$ and add two equations to $A$

and do this for all partial derivatives $m$.

My question: When confronted with a situation like this, is it better/more efficient/simpler/$\lt$your adjective here$\gt$ doing it like in 1.? Or does it make sense to break a function down like in 2. and add more but simpler constraints when building $A\mathbf{x}=\mathbf{b}$?

1 Answers 1

3

The second alternative makes no sense. A smooth function $E(\mathbf x) = E_1(\mathbf x) + E_2(\mathbf x)$ is minimized when $\partial E(\mathbf x)/\partial x_m = 0$ for all $m$, or equivalently when $\partial E_1(\mathbf x)/\partial x_m + \partial E_2(\mathbf x)/\partial x_m = 0$. But of course the fact that $\partial E_1(\mathbf x)/\partial x_m$ and $\partial E_2(\mathbf x)/\partial x_m$ add up to zero does not mean that they are both zero!

For example, consider $E_1(x) = x^2$ and $E_2(x) = 2x$. The function $E(x) = E_1(x) + E_2(x) = x^2 + 2x$ is minimized at $x = -1$, even though $dE_2(x)/dx = 2$ is zero nowhere.

  • 0
    Alright, I guess that is enough to show that it cannot work the way I thought it would. Thanks for your help to clarify this!2012-02-18