3
$\begingroup$

Is there a "simple" solution to

$\bf \frac{\partial}{\partial w}\big(w \odot f(w)\big)$

assuming the matrix $\bf \frac{\partial f}{\partial w}$ is known? With simple I mean something like in the normal vector multiplication case

$\bf \frac{\partial}{\partial w}\big(w^Tf(w)\big) = f(w) + \big[\frac{\partial f(w)}{\partial w}\big]^T w$

such that no other knowledge of $\bf f(w)$ is required.

2 Answers 2

3

Let Diag denote the function which creates a diagonal matrix from a vector.

Define some variables for convenience $$\eqalign{ F &= \operatorname{Diag}(f) \cr W &= \operatorname{Diag}(w) \cr h &= w\odot f = f\odot w \cr J &= \frac{\partial f}{\partial w} \cr \cr }$$ Now find the differential and gradient of $h$ $$\eqalign{ dh &= f\odot dw + w\odot df \cr &= F\,dw + W\,(J\,dw) \cr \cr \frac{\partial h}{\partial w} &= F + WJ \cr }$$

2

Given $\mathrm f : \mathbb R^n \to \mathbb R^n$, let vector field $\mathrm g : \mathbb R^n \to \mathbb R^n$ be defined by

$$\mathrm g (\mathrm w) : = \begin{bmatrix} w_1 \, f_1 (\mathrm w)\\ w_2 \, f_2 (\mathrm w)\\ \vdots\\ w_n \, f_n (\mathrm w)\end{bmatrix}$$

Differentiating $g_i (\mathrm w)$ with respect to $w_j$,

$$\frac{\partial g_i}{\partial w_j} = \delta_{ij} \, f_i + w_i \, \frac{\partial f_i}{\partial w_j}$$

Hence, the Jacobian of $\mathrm g$ is

$$\frac{\partial \mathrm g}{\partial \mathrm w} = \mbox{diag} (\mathrm f (\mathrm w)) + \mbox{diag} (\mathrm w) \, \frac{\partial \mathrm f}{\partial \mathrm w}$$