Can someone give me the answer to the following expression?
$\frac{\partial}{\partial X}\|Xa\|_2 =?$
$X$ is a square matrix and $a$ is a vektor of the apropriate size. $\|\cdot\|_2$ is the euclidean norm. Thanks
Can someone give me the answer to the following expression?
$\frac{\partial}{\partial X}\|Xa\|_2 =?$
$X$ is a square matrix and $a$ is a vektor of the apropriate size. $\|\cdot\|_2$ is the euclidean norm. Thanks
If the following is correct, then how are the above answers related to it?
$ \frac{\partial}{\partial X}\left[\|Xa\|_2\right] = \frac{\partial}{\partial X}\left[(a^TX^TXa)^{\frac{1}{2}}\right] = \frac{1}{2}(a^TX^TXa)^{-\frac{1}{2}}\cdot\frac{\partial}{\partial X}\left[(a^TX^TXa)\right] =\frac{1}{2}(a^TX^TXa)^{-\frac{1}{2}}X(aa^T + aa^T) = (a^TX^TXa)^{-\frac{1}{2}}Xaa^T = \frac{Xaa^T}{\|Xa\|_2} $
Take any matrix $B$ and expand $ \|(X+tB)a\|_2^2 = \|Xa\|_2^2 + 2 t \langle Xa , Ba \rangle + t^2 \|Ba\|_2^2. $ Therefore the directional derivative of the square of your map in the direction $B$ is $ 2 B^t Xa $ where $B^t$ is the transposed matrix of $B$.
Recall that, when it exists, the differential $DF(X)$ at $X$ of a real-valued function $F$ defined on the space $M_n$ of the square matrices of size $n$ is the unique linear functional $L_X$ defined on $M_n$ such that $F(X+Y)=F(X)+L_X(Y)+\|Y\|_2\cdot\varepsilon_X(Y)$, where $\varepsilon_X(Y)$ has limit $0$ when $Y$ converges to the zero matrix.
Let $G:X\mapsto\|Xa\|_2^2$. Then $G(X)=a^tX^tXa$ and $ G(X+Y)=G(X)+2a^tX^tYa+a^tY^tYa, $ hence $DG(X)$ exists and is given by $ DG(X)(Y)=2a^tX^tYa. $ Let $F:X\mapsto\|Xa\|_2=\sqrt{G(X)}$. Assume first that $Xa\ne0$. Then, $ G(X+Y)=G(X)\cdot(1+F(X)^{-2}\cdot DG(X)(Y)+o(\|Y\|_2)). $ Together with the fact that $\sqrt{1+t}=1+\frac12t+o(t)$ when $t\to0$, this yields $ F(X+Y)=F(X)\cdot(1+\tfrac12F(X)^{-2}\cdot DG(X)(Y)+o(\|Y\|_2)), $ hence $ F(X+Y)=F(X)+\tfrac12F(X)^{-1}\cdot DG(X)(Y)+o(\|Y\|_2). $ Thus, $DF(X)$ exists and is given by $ DF(X)(Y)=\tfrac12F(X)^{-1}\cdot DG(X)(Y)=\frac{a^tX^tYa}{\|Xa\|_2}. $ The case $Xa=0$ is different. Now, $F(X+Y)=F(X)+\|Ya\|_2$.
Given the function \begin{eqnarray*} f\left(X\right) & = & \left\Vert X\cdot a\right\Vert \\ & = & \sqrt{a^{T}\cdot X^{T}\cdot X\cdot a}\\ & = & \left(a^{T}\cdot X^{T}\cdot X\cdot a\right)^{1/2} \end{eqnarray*} where $X$ is a matrix and $a$ is a vector. The gradient is \begin{eqnarray*} \frac{\partial f}{\partial X} & = & \frac{1}{2}\cdot\left(a^{T}\cdot X^{T}\cdot X\cdot a\right)^{-1/2}\cdot\frac{\partial}{\partial X}a^{T}\cdot X^{T}\cdot X\cdot a\\ & = & \frac{1}{2\cdot\left\Vert X\cdot a\right\Vert }\cdot\frac{\partial}{\partial X}a^{T}\cdot X^{T}\cdot X\cdot a \end{eqnarray*} where \begin{eqnarray*} & & \frac{\partial}{\partial X}a^{T}\cdot X^{T}\cdot X\cdot a=\\ & = & \lim_{t\rightarrow0}\frac{a^{T}\cdot\left(X+t\cdot D\right)^{T}\cdot\left(X+t\cdot D\right)\cdot a-a^{T}\cdot X^{T}\cdot X\cdot a}{t}\\ & = & \lim_{t\rightarrow0}\frac{a^{T}\cdot\left(X^{T}+t\cdot D^{T}\right)\cdot\left(X+t\cdot D\right)\cdot a-a^{T}\cdot X^{T}\cdot X\cdot a}{t}\\ & = & \lim_{t\rightarrow0}\frac{t\cdot\left(a^{T}\cdot X^{T}\cdot D\cdot a+a^{T}\cdot D^{T}\cdot X\cdot a+t\cdot a^{T}\cdot D^{T}\cdot D\cdot a\right)}{t}\\ & = & \lim_{t\rightarrow0}a^{T}\cdot X^{T}\cdot D\cdot a+a^{T}\cdot D^{T}\cdot X\cdot a+t\cdot a^{T}\cdot D^{T}\cdot D\cdot a\\ & = & a^{T}\cdot X^{T}\cdot D\cdot a+a^{T}\cdot D^{T}\cdot X\cdot a\\ & = & 2\cdot\left\langle X\cdot a,D\cdot a\right\rangle \\ & = & 2\cdot Tr\left(a^{T}\cdot X^{T}\cdot D\cdot a\right)\\ & = & 2\cdot Tr\left(a\cdot a^{T}\cdot X^{T}\cdot D\right)\\ & = & 2\cdot\left\langle \left(a\cdot a^{T}\cdot X^{T}\right)^{T},D\right\rangle \\ & = & 2\cdot\left\langle X\cdot a\cdot a^{T},D\right\rangle \\ & = & \left\langle 2\cdot X\cdot a\cdot a^{T},D\right\rangle \end{eqnarray*} and so you have $\frac{\partial}{\partial X}a^{T}\cdot X^{T}\cdot X\cdot a=2\cdot X\cdot a\cdot a^{T}$. Therefore your solution is the same as my solution: \begin{eqnarray*} \frac{\partial}{\partial X}\left\Vert X\cdot a\right\Vert & = & \frac{X\cdot a\cdot a^{T}}{\left\Vert X\cdot a\right\Vert } \end{eqnarray*} By the way: The solution of the mathematicians I cannot understand, too.