2
$\begingroup$

Let $A$ be a $n\times n$ matrix, $\text{diag}(x)$ is the diagonal matrix with $x$ on the diagonal. How can I find $dF(x)$ for $F(x) = \text{diag}(x)Ax$?

Thank you very much in advance!

  • 0
    What have you tried? Also, please use MathJaX formatting so we can read more clearly! – 2017-02-04
  • 0
    Unfortunately I don't even know where to start. I'd really appreciate some help! Thank you! – 2017-02-04
  • 0
    You did well! I will make some changes for clarity. Once approved it will be visible. – 2017-02-04

2 Answers 2

1

If you use the Hadamard (aka elementwise) product, then you can get rid of the diag operation, which will make it easier to find the differential and jacobian of the function. $$\eqalign{ f &= x\circ(Ax) \cr\cr df &= (Ax)\circ dx + x\circ(A\,dx) \cr &= \operatorname{diag}(Ax)\,dx + (x1^T\circ A)\,dx \cr &= \big(\operatorname{diag}(Ax) + A\circ x1^T\big)\,dx \cr\cr \frac{\partial f}{\partial x} &= \operatorname{diag}(Ax) + A\circ x1^T \cr \cr}$$ In several steps, the commutivity of the hadamard product was used, i.e. $$\eqalign{ x\circ y & = y\circ x \cr &= \operatorname{diag}(x)\,y \cr &= \operatorname{diag}(y)\,x \cr }$$

0

I assume that $x = (x_1, \dots, x_n)^T \in \mathbb{R}^n$ is a column vector so $F \colon \mathbb{R}^n \rightarrow \mathbb{R}^n$. The differential $dF(x)$ must satisfy

$$ \lim_{h \to 0} \frac{F(x + h) - F(x) - dF(x)h}{\| h\|} = 0. $$

We have

$$ F(x + h) - F(x) = \operatorname{diag}(x + h)A(x + h) - \operatorname{diag}(x)Ax = \\ (\operatorname{diag}(x) + \operatorname{diag}(h))(Ax + Ah) - \operatorname{diag}(x)Ax = \\ \operatorname{diag}(x)Ah + \operatorname{diag}(h)Ax + \operatorname{diag}(h)Ah.$$

The term $\operatorname{diag}(h)Ah$ is quadratic in $h$ so we can guess that $dF(x)h = \operatorname{diag}(x)Ah + \operatorname{diag}(h)Ax$ and prove it:

$$ \frac{\| F(x+h) - F(x) - \operatorname{diag}(x)Ah + \operatorname{diag}(h)Ax \|}{\| h \|} = \frac{\| \operatorname{diag}(h)Ah \|}{\| h \|} \leq \frac{ \| \operatorname{diag}(h) \| \| A \| \| h \|}{\| h \|} = \\ \| \operatorname{diag}(h) \| \| A \| \xrightarrow[h \to 0]{} 0.$$