Gradient and transpose

Question

I have simple question, when taking the Jacobian of a linear vector function, do you have to transpose the matrix $A$?

$b\in\mathbb{R}^{n},x\in\mathbb{R}^{m}\\J(Ax+b) = A^T, A\in\mathbb{R}^{n\times m}$

Answer 1

Your function $x\to Ax+b$ maps $\Bbb R^m$ to $\Bbb R^n$, hence its gradient maps $\Bbb R^m$ to the space of linear operators "from $\Bbb R^m$ to $\Bbb R^n$", or, after a slight notation abuse, to the space of matrices with $n$ lines and $m$ columns.

In other words, $J\in M_{n\times m}(\Bbb R)$, and you can even say that $\nabla J = A$.

Answer 2

$J_x(f)$ is the, in case of existence, uniquely determined matrix with the property

$$f(x+h)=f(x)+J_x(f) h+\mathrm o(\Vert h\Vert).$$

When plugging in $f(x)=Ax+b$, you are looking for a $J$ with

$$ A(x+h)+b=Ax + b + J_x(f)h+\mathrm o(\Vert h\Vert).$$

Since the left side is equal to $Ax + Ah + b$ (because of linearity), you see, that the unique solution is $J_x(f)=A$, and you can even drop the $\mathrm o$-term.