6
$\begingroup$

In this paper by Prof Nesterov, First-order methods of smooth convex optimization with inexact oracle, proof of Theorem 2, there is the following very simple equation which I think it is wrong, \begin{align} \Vert x_{k+1}-x^*\Vert^2 = \Vert x_{k}-x^*\Vert^2 + 2\langle B(x_{k+1}-x_k),x_{k+1}-x^*\rangle -\Vert x_{k+1}-x_k\Vert^2 \end{align} (They defined norm as $\Vert x\Vert^2 =\langle Bx,x\rangle$) Obviously, it must be \begin{align} \Vert x_{k+1}-x^*\Vert^2 = \Vert x_k-x^*+x_{k+1}-x_{k}\Vert^2 = \Vert x_{k}-x^*\Vert^2 + 2\langle B(x_{k+1}-x_k),x_{k}-x^*\rangle \\ +\Vert x_{k+1}-x_k\Vert^2 \end{align} Am I missing something?

I also, checked journal version, Mathematical Programming.

  • 2
    You are not expanding it the way they have done. Use the fact that B is self adjoint linear operator. Notice the signs in the original statement, that should give you some hints. Alternatively, replace all the squared norms with inner product and simplify. – 2017-01-25

2 Answers 2

8

The author is correct. The norm in question is induced by the real inner product $(x,y):=\langle Bx,y\rangle$ where $B$ is positive definite with respect to $\langle\cdot,\cdot\rangle$. Let $u = x_{k+1} - x^\ast$ and $v = x_{k+1}-x_k$. Then the equation is simply saying that $$ \|u\|^2 = \|u-v\|^2 + 2(v,u) - \|v\|^2, $$ which is just a rearrangement of terms in the cosine law $$ \|u-v\|^2 = \|u\|^2 - 2(v,u) + \|v\|^2. $$

3

If it is bilinear, the derivation is correct.

$$2\langle B(x_{k+1}-x_k),x_k-x^*\rangle +\Vert x_{k+1}-x_k\Vert^2=$$

$$=2\langle B(x_{k+1}-x_k),x_k-x^*\rangle +2\Vert x_{k+1}-x_k\Vert ^2-\Vert x_{k+1}-x_k\Vert^2=$$

$$=2\left(\langle B(x_{k+1}-x_k),x_k-x^*\rangle +\langle B(x_{k+1}-x_k),x_{k+1}-x_k\rangle\right)-\Vert x_{k+1}-x_k\Vert^2=$$

$$=2\langle B(x_{k+1}-x_k),x_k-x^*+x_{k+1}-x_k\rangle -\Vert x_{k+1}-x_k\Vert^2=$$

$$=2\langle B(x_{k+1}-x_k),x_{k+1}-x^*\rangle -\Vert x_{k+1}-x_k\Vert^2$$