0
$\begingroup$

For my class, I am trying to understand why the norm of a matrix, is bounded in such a way. Specifically, I am trying to understand the rationale behind this equation. We are given the jacobian matrix $\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}}$ being defined as:

$$ \frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}} = \prod_{i = k + 1}^{t}\mathbf{\theta}^T \text{diag}[\mathbf{\phi'}(\mathbf{h}_{i-1})] $$

Now given this, we are told that the (2-norm?) of the above jacobian is bounded as so:

$$ \lVert{\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}}} \lVert \leq \lVert \mathbf{\theta}^T \lVert \lVert \text{diag}[\mathbf{\phi'}(\mathbf{h}_{i-1})]\lVert $$

Why is this true?

For a broader context, it is taken from those slides.

I can see and understand why the jacobian matrix $\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}}$ is the way it is, but what I do not understand is where this equation bounding it's (I'm guessing 2-norm) comes from.

Thanks.

  • 0
    Can you clearly and rigorously define all the objects that appear in the image? The slides are unreadable. – 2017-01-17
  • 0
    @AlexProvost Here you go. – 2017-01-17

1 Answers 1

1

From the previous line of the slide,

$\frac{\partial h_{i}}{\partial h_{i-1}}=\Theta^{T}\mbox{diag}[\phi'(h_{i-1}]$

A basic property of matrix norms is that for any matrices $A$ and $B$ of compatible sizes,

$\| AB \| \leq \| A \| \| B \|$

Thus

$\left\| \frac{\partial h_{i}}{\partial h_{i-1}} \right\| \leq \| \Theta^{T} \| \| \mbox{diag}[\phi'(h_{i-1}] \|$

  • 0
    Ah! Thanks @Brian - does this property have a name? Also, why is this true? – 2017-01-17
  • 0
    @Spacey This is called submultiplicativity of the operator norm. It is proved by noting that $\| Av \| \leq \| A \| \|v \|$ holds for any vector $v$ and then invoking the definition of the operator norm. – 2017-01-17