1
$\begingroup$

What is the derivative of ${\lVert{ABC-D}\rVert}_F^2$ with respect to $B$?

where:

${\lVert{.}\rVert}_F^2$ represents the power 2 of the frobenius norm

$A$ is an $n \times m$ matrix

$B$ is an $m \times m$ matrix

$C$ is an $m \times l$ matrix

$D$ is an $n \times l$ matrix

1 Answers 1

1

I used the Matrix Cookbook for reference.

Let $E=ABC$. First, $$\frac{\partial}{\partial E} \|E-D\|_F^2 = 2 (E-D) \in \mathbb{R}^{n \times l}.$$

Next, if $a_i^\top$ denotes the $i$th row of $A$, and $c_j$ denotes the $j$th column of $C$, then $$\frac{\partial}{\partial B} (ABC)_{ij}= \frac{\partial}{\partial B} a_i^\top B c_j = a_ic_j^\top \in \mathbb{R}^{m \times m}.$$ So $\frac{\partial}{\partial B}(ABC)$ can be viewed as a big $n \times l$ matrix where the $(i,j)$ entry is itself the $m \times m$ matrix $a_i c_j^\top$.

Applying chain rule gives $$\frac{\partial}{\partial B} \|ABC-D\|_F^2 = 2(ABC-D) \circ \frac{\partial}{B} (ABC) = 2\sum_{i=1}^n \sum_{j=1}^l (ABC-D)_{ij} a_i c_j^\top \in \mathbb{R}^{m \times m},$$ where "$\circ$" denotes some sort of Hadamard product (just refer to the double sum).

Edit: as pointed out in the comments, the final answer can be written as $$2A^\top (ABC-D) C^\top.$$

  • 0
    Thanks :) Is there a matrix form for your answer? for example: $A^T(ABC-D)C^T$2017-01-03