0
$\begingroup$

I have read everywhere that the resolution of the linear system

$$Ax+b=0$$

where $A\in S_n(\mathbb R)$ and $b\in \mathbb R^n$ is equivalent to the resolution of the following optimization problem:

$$f(x)=\frac 12\langle Ax,x\rangle+\langle b,x\rangle.$$

Which means that if $x\in \mathbb R^n$ minimize the function $f(x)=\frac 12\langle Ax,x\rangle+\langle b,x\rangle$, then $x$ is solution of the linear sytem $Ax+b=0$ where $A\in S_n(\mathbb R)$.

I do not understand why this should be true... Can someone please explain it?

  • 1
    Did you try finding the gradient of $f$? – 2017-02-20
  • 0
    @5xum The gradient of $f$ seems to be $\nabla f(x)=Ax+b$, but I think there is still some arguments missing here... – 2017-02-20
  • 0
    It is not true. The sign is wrong. – 2017-02-20
  • 0
    @E.Joseph Apart from some sign mixup, yes that's the gradient. And since the function is a continuous and continuously differentiable function, you know that the gradient at its extreme value must be $0$. – 2017-02-20
  • 0
    @5xum I am starting to see it! But how do I know there is only one extreme value for $f$? – 2017-02-20
  • 1
    @E.Joseph Because there is only one zero for the gradient. The only other option would be for the function to go toward its minimum "at infinity", but that is also impossible. – 2017-02-20
  • 1
    @E.Joseph I see that you have created ([tag:linear-system]) tag. I just wanted to let you know that I have started a post on meta about this new tag: http://meta.math.stackexchange.com/questions/25694/tag-management-2017/25938#25938 – 2017-02-25

1 Answers 1

1

U cannot prove this by taking the gradient! This is not a function but a functional! You should variate it! U have the following functional $$F[x]=\langle{Ax}, x\rangle-\langle{b, x}\rangle$$ U variate the functional $$\delta{F[x]}=\langle{Ax}, \delta{x}\rangle-\langle{b, \delta{x}}\rangle=\langle{Ax-b}, \delta{x}\rangle$$ For the extremum, u require the functional to be stationary with respect to any variations, i.e. $$\delta{F[x]}=\langle{Ax-b}, \delta{x}\rangle=0$$ As the above equality holds for any $\delta{x}$, u better have $$x^{T}A^{T}-b^{T}=0$$ Taking the transpose of this equation, u arrive at $Ax=b$ !