1
$\begingroup$

Am learning about Fréchet differentials and was wondering if for a real matrix $X$ and positive semidefinite real matrices $A,B$ the function $f(X)=TrX^TAX-X^TBX$ is twice Fréchet differentiable or not?

I believe that the gradient is $2AX-2BX$ but how do I prove Fréchet differentiability?

  • 1
    If all 1st order partial derivatives are continuous, the function is Fréchet differentiable.2012-09-14

1 Answers 1

1

$\def\Mat{\mathop{\mathrm{Mat}}\nolimits}\def\R{\mathbb R}\def\tr{\mathop{\mathrm{tr}}}\def\abs#1{\left|#1\right|}\def\norm#1{\left\|#1\right\|}$ In order for $f\colon \Mat_n(\R) \to \R$ to be differentiable at $X \in \Mat_n(\R)$, it must hold \[ f(X + H) = f(X) + f'(X)H + o(H), \quad H \to 0 \] for some linear map $f'(X)\colon \Mat_n(\R) \to \R$. Just computing, we get (writing $C := A -B$ for brevity and using $C^T = C$) \begin{align*} f(X + H) &= \tr(X^TCX + H^TCX + X^TCH + H^TCH)\\ &= \tr(X^TCX) + 2\tr(H^TCX) + \tr(H^TCH) \end{align*} We have \begin{align*} \abs{\tr H^TCH} &\le n\norm{H^TCH}_2\\ &\le n\norm{C}_2\norm{H}^2_2 \end{align*} so $\tr(H^TCH) = o(H), H \to 0$ and $f'(X)H = 2\tr(H^TCX)$.

For showing that $f$ is twice differentiable, we have to prove that the map \[ f'\colon \Mat_n(\R) \to \Mat_n(\R)^*, \quad$X$\mapsto 2\tr({-}^TCX) \] is differentiable at each $X$, that is, there is a linear $f''(X)\colon \Mat_n(\R) \to \Mat_n(\R)^*$ with \[ f'(X+K) = f'(X) + f''(X)K + o(K), \quad K \to 0 \] But this is easy, as $f'$ is linear, so if we define $f''(X) = f'$ for each $X$, we have \[ f'(X+K) = f'(X) + f'(K) = f'(X) + f''(X)K. \] So, $f$ is twice (Frechet) differentiable.


Another way to show your claim is to follow LVKs comment and show that all partial derivatives of $f$ up to order 2 exist and are continuous (which implies that $f$ is twice differentiable by a well known theorem), we have, writing $[M]_{ij}$ for the $i,j$-th coordinate of a matrix $M$ and $X = (x_{ij})$: \begin{align*} f(X) &= \sum_i [X^TCX]_{ii}\\ &= \sum_{i,j,k} [X^T]_{ij}[C]_{jk}[X]_{ki}\\ &= \sum_{i,j,k} [C]_{jk}x_{ji}x_{ki} \end{align*} As this is a polynomial in the $x_{ij}$, its partial derivatives are polynomials themselves, so your claim follows again.

  • 1
    $\mathrm{Mat}_n(\mathbb R)^*$ denotes the dual space of $\mathrm{Mat}_n(\mathbb R)$, i. e. the linear functionals on the matrices.2012-09-17