Recently I came across the following expression:
$$ \langle \nabla f(x_1),x_2 \rangle$$
I do understand how to calculate the value of the expression. You take the derivatives of each of the entry in the function and then substitute $x_1$ in it and then take the dot product between the resulting value and $x_2$. But is there any meaning for this expression? What does this expression represent?
Please help.
The inner product $\langle \nabla f(\mathrm x_1), \mathrm x_2 \rangle$ is the directional derivative of $f$ in the direction of $\mathrm x_2$ at $\mathrm x_1$.
$$\lim_{h \to 0} \frac{f (\mathrm x_1 + h \mathrm x_2) - f (\mathrm x_1)}{h} = \langle \nabla f(\mathrm x_1), \mathrm x_2 \rangle$$
$\nabla f(x_1)$ is the jacobian matrix of the scalar-valued function $f$ at $x_1$. Now the differential $Df(x_1)$ is a linear map, in this case simply given by $\langle \nabla f(x_1),x_2 \rangle$. So the answer to your question is: The differential of a scalar-valued function $f$ in $x_1$ evaluated in $x_2$. To make this more visible, we have $$\begin{pmatrix} \partial_1f(x_1) & \dots \partial_nf(x_1)\end{pmatrix} \begin{pmatrix} x^1_2\\ \vdots\\ x_2^n\end{pmatrix} = \langle \nabla f(x_1),x_2 \rangle$$
$\langle \nabla f(x_1),x_2 \rangle$ is also reffered as directional derivative along $x_2$.
Note that the following identity holds: $$ \frac{\partial f}{\partial v}=\lim_{h \to 0}=\frac{f(x_1+hv)-f(x_1)}{h}=<\nabla f(x_1),v> $$ In other words, it is the derivative of $f$ in $x_1$ in the direction $v$
Consider the following curve: $$\alpha: \mathbb{R} \to \mathbb{R}^n : t \mapsto x_1 + tx_2$$ We can compute $(f \circ \alpha)'(0)$, that is the derivative with respect to $t$ and evaluate it in $0$. Using the chain rule, we find that $$(f \circ \alpha)'(0) = Df(\alpha(0)) \alpha'(0) = Df(x_1) (x_2)$$ where $Df$ is the derivative of $f$, or as you say it: the gradient of $f$. This is a linear map: $$Df(x_1): \mathbb{R}^n \to \mathbb{R}^n,$$ so it makes sense to apply this map to a vector $x_2$.
In the case where $f : \mathbb{R} \to \mathbb{R}^n$, we find that it corresponds to the inproduct you wrote down. But you can apply this for any function $f: \mathbb{R}^m \to \mathbb{R}^n$.