First of all: if the order of $g$ is $k$, then we have that $g^k = e$, so that your group actually is $H = \{e, g, g^2, \ldots, g^{k-1}\}$. Are you aware of the subgroup criterion? (If no, it states that if $H$ is a subset of a group $G$ and $H$ is not empty, then it suffices to prove that for any $x, y \in H$ we also have $x \ast y^{-1} \in H$ in order to prove that $H$ is a subgroup. You can easily prove this yourself).
Now clearly $H$ is not empty, since $e \in H$. Now suppose $x^l, x^m \in G$, where both $l,m \in \{0, 1, \ldots k-1\}$. Consider $x^l \ast (x^m)^{-1}$. Note that $(x^m)^{-1} = x^{-m}$ (you can prove this, but I guess this is clear).
Now we distinguish two cases: either $l \geq m$ or $l < m$. In the first case we have that $x^l \ast x^{-m} = x^{l - m} \in G$ since $0 \leq l - m \leq k-1$.
If $l < m$, then we would have that $x^{l-m}$ has a negative exponent. However, we have that $x^k = e$, so we find that $x^l \ast x^{-m} = x^{l-m} = x^{l-m} \ast x^k = x^{l-m + k}$ and since $l
By the subgroup criterion, we have that $H = \{e, g, g^2, \ldots, g^{k-1}\}$ is a subgroup.