2
$\begingroup$

Let $U \in \mathbb{R}^{n}$ be open, and let $f: U \mapsto \mathbb{R}$ be a $C^{1}$ function. We obviously have $\nabla f$ = $(Df)^{T}$ where $T$ denotes the transpose of the matrix.

When are the norms of the gradient of $f$ and the derivative of $f$ equivalent? In other words, is the norm of the gradient is $||\nabla f||$ $\textbf{always}$ equal to the norm of the derivative $||Df$|| ?

My question is motivated by the Mean Value Inequality, i.e.

$$||f(b) - f(a)|| \leq M ||b - a||$$

where $a, b \in U$ with the segment $\{a+t(b-a): 0 \leq t \leq 1\} \subset U$ and $M = \mbox{max}_{0 \leq t \leq 1} ||Df(a+t(b-a)||$.

Can we replace $Df$ with $\nabla f$ in $M$ and write $M = \mbox{max}_{0 \leq t \leq 1} ||\nabla f(a+t(b-a)||$ ?

Another question. In the $||\nabla f||$ formulation, the norm is just the standard Euclidean one. For the Mean Value Inequality, what would be the standard norm used for the computation of $||Df||$ ?

Thanks.

  • 0
    Yes, you can replace $||\text{D}f||$ by $||(\text{D}f)^T||$ with a factor (not necessary equal), this essentially leads back to the question that the norm of the transposed is equivalent to the original one. Have a look at http://math.stackexchange.com/questions/628847/norm-of-matrix-transpose – 2017-01-07
  • 0
    Thanks. What would be the "name" of the norm used in the mean value inequality when we write $||Df||$? Given that we have the Euclidean norm for $||\nabla f||$ - see my edit @MarvinF – 2017-01-07
  • 0
    Should be the supremum/infinity-norm, one writes $$\max_{0\leq t \leq 1} ||\text{D}f(a+t(b-a))||=||\text{D}f(a+\bullet(b-a))||_\infty .$$ Note that you get something like $$||f(b)-f(a)|| \leq M ||b-a|| \leq C ||\nabla f(a+\bullet(b-a))||_\infty ||b-a||$$ – 2017-01-07

1 Answers 1

3

Yeah, if you measure things correctly. You have made the observation that the differential is often written as a row vector and then the gradient is just the transpose of this vector. If you measure the norm of both row/columns vectors in the same way (say, using the standard Euclidean norm) which is a reasonable thing to do then clearly the result will be the same.

A more abstract and coordinate free approach would be to treat the differential $Df(p)$ as a linear functional. That is, let us fix an inner product on $\mathbb{R}^n$ (say the standard Euclidean inner product) and denote by $\| \cdot \|$ the induced norm on $\mathbb{R}^n$. Now it makes more sense to measure the norm of $Df(p)$ using the operator norm

$$ \| Df(p) \| := \sup_{||x|| = 1} | Df(p)x |. $$

In turns out that also in this case, if you define the gradient of $f$ correctly as vector (by the formula $Df(p)x = \left< (\nabla f)(p), x \right>$), you will also have $\| Df(p) \| = \| (\nabla f)(p) \|$ where on the left hand side we use the operator norm.

  • 0
    Very nice explanation! @levap – 2017-01-07