A classical theorem of Rockafellar characterizes the gradients of convex functions.
Theorem B. Let $T:E\to E^*$ be a multivalued mapping. In order that there exist a lower semicontίnuous proper convex function $f$ on $E$ such that $T=\partial f$, it is necessary and sufficient that $T$ be a maximal cyclically monotone operator. Moreover, in this case $T$ determines $f$ uniquely up to an additive constant.
Since your $f$ is continuous, you can ignore "multivalued" and "maximal". As a corollary, if $f$ can be written as the difference of two cyclically monotone maps then it is the gradient (of the difference of two corresponding functions). A function that can be written as the difference of two convex functions is called delta-convex (or d.c.). This is a pretty large class; in finite dimensions in includes all $C^2$ functions. There is a fair amount of recent literature on d.c. functions, but I don't remember seeing any better characterization of their gradients.