It seems a quite simple problem if we use Lagrange multiplier. But I have some other methods which seem very weird. Can anybody help me to fix them?
Weird solution 1: Using a rotation matrix.~~~~~~~~~~~~~~~~~~~~~~~~~
Since the length of $x$ is fixed, $x$ can vary on a sphere. That means any variance of $x$ is caused by a rotation matrix $R$. $$\delta x = Rx-x=(R-I)x$$ According to rotation axis-angle formula, a rotation matrix can be written as $$R=e^{[w]_{\times}}$$ where $w$ is the product of the rotation axis and rotation angle. $[w]_{\times}$ is the associating skew-symmetric matrix of the vector $w$. The first order Taylor expansion of $R$ is $$R=I+[w]_{\times}$$ Hence we have $$dx=(R-I)x=[dw]_{\times}x=[x]_{\times}dw$$ Now $$df(x)=g(x)^Tdx=g(x)^T[x]_{\times}dw$$ Since $dw$ has no constraints, solving the following equation can give the extrema. $$g(x)^T [x]_{\times}=0$$ That is the cross product $$g(x) \times x=0$$
Weird solution 2:~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$||x||=1$, $dx$ is not free. But we can find a free vector $y$ such that $x=\frac{y}{||y||}$. Then $dy$ should be free. And we have $$dx=\frac{dy||y||-yd||y||}{||y||^2}=\frac{1}{||y||}\left( I-xx^T \right) dy$$ Therefore $$df(x)=g(x)^Tdx=\frac{1}{||y||}g(x)^T\left( I-xx^T \right) dy$$ Since $dy$ has no constraints, solving the following equation can give the extrema. $$\frac{1}{||y||}g(x)^T\left( I-xx^T \right)=0 $$
Normal Solution 3: using Lagrange multiplier.~~~~~~~~~~~~~~~~~~~~~~~~~ $$d[f(x)+\lambda (x^T x-1)]=\left( g(x)^T+2\lambda x^T \right)dx$$ So we have $$g(x)+2\lambda x=0$$
Remark: Note the solution 1 is the same as the solution 3, because both of them just mean $g(x)$ is collinear with $x$.