How to find the conjugate function of exponential $f(x) = \exp\langle x, c\rangle$?
By the definition: $$f^*(y) = \sup_x \big\{\langle x, y\rangle-\exp\langle x, c\rangle\big\}$$
So take derivative of $h(x) =\langle x, y\rangle-\exp\langle x, c\rangle\ $ and let it be $0$, we have
$$\nabla h(x) = y-\exp\langle x, c\rangle c=0$$ So we get $$y = \exp\langle x, c\rangle c$$
However, $c$ is a vector, how to obtain $x^*$, the optimal solution?
If $y = \exp\langle x,c \rangle c$, then either $y$ and $c$ are proportional and $\exp\langle x,c \rangle$ is their ratio (and this only determines the value of $\langle x, c \rangle$; there are lots of $x$ that have the same inner product), or $y$ and $c$ are not proportional and there does not exist any solution for $x$.
Don't forget that the equation you are solving is merely giving you the stationary points; there is no guarantee that the function has a maximum that occurs at a stationary point.
In fact, if $y$ and $c$ are not parallel, it's actually pretty easy to see that the supremum is infinite; just consider $x$ perpendicular to $c$.