Let $G$ be a group. By definition $\text{Aut}(G)=\left\{f:G\rightarrow G\mid f \mbox{ is an isomorphism of }G\right\}$. Given two automorphisms $f,g\in \text{Aut}(G)$, we can consider the composition $g\circ f$. We claim that $\text{Aut}(G),\circ$ is a group. We have to check all axioms.
First of all we need to show that $g\circ f$ is again an automorphism, i.e. a homomorphism that is bijective. Now since $g$ and $f$ are bijective, $g\circ f$ is bijective. Moreover, $$(g\circ f)(ab)=g(f(ab))=g(f(a)f(b))=g(f(a))g(f(b))=(g\circ f)(a)(g\circ f)(b)$$
for all $a,b\in G$. Hence $g\circ f$ is a group homomorphism.
Secondly we need to show that $\circ$ is associative, i.e. $(h\circ g)\circ f=h\circ (g\circ f)$. Just evaluate both morphisms at $a\in G$ and see that both expressions coincide due to the associativity of $G$.
Thirdly we need to check that there is a neutral element for $\circ$. Clearly $Id_G:G\rightarrow G:a\mapsto a$ is an automorphism. Since $f\circ Id_G=Id_G\circ f$ for all $f\in \text{Aut}(G)$, $Id_G$ is the neutral element.
Last but not least, we have to check that each $f\in \text{Aut}(G)$ has an inverse for $\circ$. Consider the inverse function $f^{-1}$. Clearly $f^{-1}\circ f=Id_G=f\circ f^{-1}$. So it remains to show that $f^{-1}$ is a group morphism. Now it's a very good exercise to prove this last statement.
EDIT: Let's prove the last statement. Suppose that $f:G\rightarrow G$ is a group isomorphism. We need to show that $f^{-1}$ is a group morphism. Let $a,b\in G$. By definition there exist a unique $x,y\in G$ such that $f(x)=a$ and $f(y)=b$. Hence $$f^{-1}(ab)=f^{-1}(f(x)f(y))=f^{-1}(f(xy))=xy.$$
Similarly $$f^{-1}(a)f^{-1}(b)=f^{-1}(f(x))f^{-1}(f(y))=xy.$$
Hence $f^{-1}(ab)=f^{-1}(a)f^{-1}(b).$