What is the result of this type of derivative?

Question

What is the results of

$$ \frac{\partial}{\partial y}\left(x^{\mathsf T}A(y)z\right) $$

where $x\in\mathbb R^m$, $y\in\mathbb R^n$, $z\in\mathbb R^\ell$, and $A\colon=\mathbb R^n\to\mathbb R^{m\times\ell}$.

Thanks in advance!

Answer 1

Since the derivative is linear you can factor out the products, i.e., the derivative with respect to each element of $y$ is equal to

$$\frac{\partial}{\partial y_i}\left(x^{\rm T} \cdot A(y) \cdot z\right) = x^{\rm T} \cdot \left( \frac{\partial}{\partial y_i}A(y)\right) \cdot z.$$

for $i=1, 2, ..., n$. If you want to stack these into a vector, you can do it like this:

$$x^{\rm T} \cdot \left( \frac{\partial}{\partial y_i}A(y)\right) \cdot z = (z \otimes x)^{\rm T} \cdot {\rm vec}\left\{\frac{\partial}{\partial y_i}A(y)\right\} = {\rm vec}\left\{\frac{\partial}{\partial y_i}A(y)\right\}^{\rm T} \cdot (z \otimes x),$$

where $\otimes$ is the Kronecker product and ${\rm vec}\{\cdot\}$ is the vectorization operator. Therefore,

$$\frac{\partial}{\partial y}\left(x^{\rm T} \cdot A(y) \cdot z\right) = D^{\rm T} \cdot (z \otimes x),$$

where $D \in \mathbb{R}^{m \cdot \ell \times n}$ and the $i$-th column of $D$ is the derivative of ${\rm vec}\{A\}$ with respect to $y_i$ for $i=1, 2, ..., n$.