3
$\begingroup$

I am trying to find the following derivative:

$$ \frac{\partial}{\partial W}W^TW $$

where $W \in \mathbb{R}^{n\times m}$ is a matrix. Also I am interested in finding the associated

$$ \frac{\partial}{\partial W}\|W^TW\|_\mathcal{F}^2 $$

I am aware of the fact that

$$ \frac{\partial}{\partial X}\|X\|_\mathcal{F}^2 = \frac{\partial}{\partial X}Tr(XX^T) = 2X $$

But I am not sure if the derivation in terms of $W$ is possible at all. Please advise.

Thank you very much.

  • 2
    What kind of objects are $W$ and $X$? Matrices, operators? What is $\mathcal F$? How comes $\text{Tr}(XX^T)=2X$ if $X$ is an operator or a matrix?2011-11-10
  • 0
    Sorry, yes, I forgot to say that both $W$ and $X$ are matrices and that $\mathcal{F}$ denotes the Frobenius norm.2011-11-10
  • 0
    What is $\frac{\partial}{\partial W}$?2011-11-10
  • 0
    Then $\mathrm{Tr}(XX^T)$ is not $2X$.2011-11-10
  • 0
    @AlexeiAverchenko - Maybe this notation would be better?$\frac{\partial W^TW}{\partial W}$, it should denote the partial derivative of $W^TW$ with respect to $W$2011-11-10
  • 0
    @Alexei: It's the [Fréchet derivative](http://en.wikipedia.org/wiki/Fr%C3%A9chet_derivative).2011-11-10
  • 0
    @DidierPiau - According to the Matrix cookbook $\frac{\partial Tr(X^TX)}{\partial X} = 2X$, if however we replace $X$ with $W^TW$, then the derivative is different and that's where I have trouble.2011-11-10
  • 0
    Until about 20 minutes ago, there was no differential sign in front of $\mathrm{Tr}(XX^T)$ so you were in effect equating a **number** with a **matrix**.2011-11-10
  • 0
    @DidierPiau - oops, sorry, I didn't realize that. I don't even remember correcting it, so I guess someone else just did. Thanks for pointing it out.2011-11-10

3 Answers 3