As you said,
$$ f(x) = f(a)+ \underbrace{{\frac{1}{4!}f^{(4)}(a)(x-a)^4 + o((x-a)^4)}}_{g(x)}. $$
If you are able to show that $g(x)$ has the same sign in a neighbourhood $V$ of $a$, that implies that $f(x)$ attends a relative extreme point at $a$. So you need to prove that $g(x)$ has the same sign in a neighbourhood $V$ of $a$. Without loss of generality, let us assume $f^{(4)}(a) >0$.
Proof that $g(x)$ is has the same sign in a neighbourhood $V$ of $a$
First, let's remember the definition of the $o$ notion. We say that $r(x) = o(f(x))$ at the point $a$ if
$$ \lim_{x \rightarrow a}\frac{r(x)}{f(x)} = 0. $$
Using the $\epsilon$-$\delta$ definition of limit, we have that, for every $\epsilon>0$, there exist a $\delta>0$ such that
$$ \bigg|\frac{r(x)}{f(x)}\bigg| < \epsilon~\,~\text{if $x$ is in the neighbourhood $V_{\delta} = \{x : |x-a|<\delta \}$}. $$
In other words, for every $\epsilon>0$, there is a neighbourhood $V$ of $a$ such that $$ |r(x)| < \epsilon |f(x)|~\,~\text{if $x\in V$}.$$
Second, let's apply this definition on $g(x)$.
$$g(x) = \frac{1}{4!}f^{(4)}(a)(x-a)^4 + \underbrace{r(x)}_{= o((x-a)^4)}.$$
Hence, if we take $\epsilon^* = \frac{1}{2}\frac{1}{4!}f^{(4)}(a) < \frac{1}{4!}f^{(4)}(a) $, there exist a neighbourhood $V$ of $a$ such that
$$ |r(x)| < \epsilon^*(x-a)^4 ~\,~ \text{if}~\,~x \in V. $$
Hence
$$ \begin{array}[rcl]~ g(x) &=& \frac{1}{4!}f^{(4)}(a)(x-a)^4 + r(x) \\ & \geq & \frac{1}{4!}f^{(4)}(a)(x-a)^4 - |r(x)| \\
& >& \frac{1}{4!}f^{(4)}(a)(x-a)^4 - \epsilon^*(x-a)^4 \\
& =& \underbrace{(\frac{1}{4!}f^{(4)}(a) - \epsilon^*)}_{> 0}(x-a)^4 \geq 0,~\text{for every }~x\in V\end{array} $$
which proves that $g(x)$ has the same sign in a neighbourhood $V$ of $a$.