3
$\begingroup$

My question is just how to derive this equality?

$$\frac{d}{dt}\|A-X(t)\|^2=2\operatorname{tr}\big((X-A)\dot{X}\big)$$
where

  1. $A, X\in \mathbb{R}^{n\times n}$

In particular, how to obtain the matrices trace product term?
It seems there are three levels chain rule, one for square, one for norm and then $X(t)$

I just know the following: $$\nabla_x \|Ax-b\|^2=2A^T(Ax-b)$$

  • 1
    I always find it easier to write it out using the inner product notation $\langle \cdot, \cdot \rangle$ and then take derivatives. For example, as $A$ and $X$ are real, we get $$\| A - X(t) \|^{2} = \langle A, A \rangle - 2 \langle A, X \rangle + \langle X, X \rangle$$ and differentiation just becomes an application of the product rule. – 2017-02-25

2 Answers 2

2

Let's use the $L^2$ norm as an example. Then, we have

$$\begin{align} ||A-X(t)||^2&=\sum_{i,j}|A_{ij}-X_{ij}(t)|^2\\\\ &=\sum_{i,j}(A^2_{ij}-2A_{ij}X_{ij}(t)+X^2_{ij}(t)) \end{align}$$

Differentiating $(1)$ we obtain

$$\begin{align} \frac{d}{dt}||A-X(t)||^2&=\sum_{i,j}(-2A_{ij}X'_{ij}(t)+2X_{ij}X_{ij}'(t))\\\\ &=2\sum_{ij}(X_{ij}(t)-A_{ij})X'_{ij}(t)\\\\ &=2\sum_{ij}(X_{ij}(t)-A_{ij})(X'_{ji})^T(t)\\\\ &=2\,\text{tr}\left((X(t)-A)X'(t)\right) \end{align}$$

as was to be shown!

Note: As elucidated in the comment by @mattos, the result can be obtained using inner product notatation. We could have begun by writing $||A-X(t)||^2=\langle A-X(t),A-X(t)\rangle$. Upon differentiating, we obtain $$\frac{d}{dt}||A-X(t)||^2=\langle X(t)-A,X'(t)\rangle +\langle X'(t),X(t)-A\rangle=2\text{tr}\left(A-X(t),X'(t)\right)$$as expected!

  • 0
    $2\sum_{ij}(X_{ij}(t)-A_{ij})(X'_{ji})^T(t)$ this step is for trace product? – 2017-02-25
  • 0
    Yes, that is the trace of $X-A$ times $X'$ – 2017-02-25
  • 0
    Isn't the result very dependent on the norm? The norm is required to be the Frobenius / L^2 / Schatten-2 norm, that's where the trace term is coming from. – 2017-02-25
  • 0
    @BadamBaplan I've edited. Thanks. – 2017-02-25
  • 0
    Would the down voter care to comment? – 2017-02-26
2

For a moment, consider the case where $X,\dot{X},A\in{\mathbb R}^{m\times n}$

Then your assumed answer cannot possibly be correct since the product $$(X-A)\dot{X}$$ has incompatible dimensions.

Several posters have suggested working with the inner product (which I'll denote by a colon).
If you follow that suggestion you obtain $$\eqalign{ f &= (X-A):(X-A) \cr\cr df &= 2(X-A):dX \cr &= 2(X-A):\dot{X} dt \cr\cr \frac{df}{dt} &= 2(X-A):\dot{X} \cr &= 2\operatorname{tr}\big((X-A)^T\dot{X}\big) \cr\cr }$$ This result is correct for the rectangular $(m\times n)$ case and also when $(m=n)$.

If you're sure that your assumed answer is correct, then perhaps you forgot to mention that $X$ is symmetric?

  • 0
    Yes, you are correct! They are symmetric! I forgot it. – 2017-02-25