1
$\begingroup$

I have a function:

$$ f(\vec x) = \frac 12 \vec x \cdot (A \vec x) - \vec x \cdot \vec b $$

Now I have to find the gradient of the function: $\vec \nabla f(\vec x)$.

Is there some easy way to do this?

Currently I have this and it feels pretty cumbersome.

\begin{align*} f(\vec x) &= \frac 12 \left( x_i \left( A \vec x \right)^i \right) - x^i b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \frac{\partial}{\partial x_i} \left( x_i \left( A \vec x \right)^i\right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( \left( A \vec x \right)_i + x_i \frac{\partial}{\partial x_i} \left( A \vec x \right)_i\right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( a_i^j x_j + x_i \frac{\partial}{\partial x_i} a_i^j x_j \right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( a_i^j x_j + x_i a_{ii} \right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( a_i^j x_j + a_{ii} x_i \right) - b_i \\ \nabla f(\vec x) &= \frac 12 A \vec x + \frac 12 a_{ii} x_i \vec{e^i} - \vec b \\ \end{align*}


I now use a new index $z$ for the indexing and have now $i$ free for the dot products:

$\newcommand{\vnabla}{\vec{\nabla}} \newcommand{\half}{\frac{1}{2}}$

\begin{align*} f(\vec x) &= \half \left( x_i \left( A \vec x \right)^i \right) - x^i b_i \\ \left(\nabla f(\vec x) \right)_z &= \half \frac{\partial}{\partial x_z} \left( x_i \left( A \vec x \right)^i\right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + x_i \frac{\partial}{\partial x_z} \left( A \vec x \right)^i\right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + x^i \frac{\partial}{\partial x_z} a_i^j x_j \right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + x^i a_{iz} \right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + a_{iz} x^i \right) - b_z \\ \vnabla f(\vec x) &= \half A \vec x + \half A^t \vec x - \vec b \\ \vnabla f(\vec x) &= A \vec x - \vec b \\ \end{align*}

The last step can only be done if $A$ is symmetric, of course.

  • 2
    I don't get what you mean by $(f(x))_i$ - your $f$ is scalar... I suggest to write down the difference quotient $(f(x+th) - f(x))/t$ for some vector $h$ and positive $t$ and let $t\to 0$.2012-05-04
  • 0
    That indeed is nonsense. I fixed it.2012-05-04
  • 2
    The answer should just be $\nabla f(x) = Ax - b$. You've messed up somewhere; probably because you reused the index $i$ in $\partial/\partial x_i$ and $x_i (Ax)^i$.2012-05-04
  • 1
    Oh yes, sure enough. Rahul has it right. Your Einstein notation in the very first step has to use some index other than $i$ or you get into trouble trying to calculate the components of $\nabla f$ in the next step.2012-05-04
  • 0
    I tripped over there indexes a minute ago as well. Thanks for the final answer!2012-05-04
  • 0
    Okay, I think I got it. Maybe I almost got it since I need $A$ to be symmetric.2012-05-04
  • 1
    Yes, my mistake, the general answer is $\nabla f(x) = A_S x - b$ where $A_S = \frac12(A + A^T)$ is the symmetric part of $A$. Generally, when you have a term $x^TAx$, you morally ought to have $A$ symmetric, because the antisymmetric part does not affect the value at all.2012-05-04

0 Answers 0