Here is a semi-formal proof of this statement in PA. By semi-formal, I mean that I will use terms like "is prime" or "dividing" which are not part of the base language, but which are easily expressed in it, and I will assume you have already built up a library of basic number theoretic facts.
Lemma 1: For all $n >1$, there is a prime dividing $n$.
Proof: By induction on $n$. Base case $n=2$ is trivial. Assume the statement for $n. If $m$ is prime, then it is also true for $n=m$. If $k$ is a nontrivial divisor of $m$ then, by induction, $k$ has a prime divisor so $n$ does as well. $\square$
Lemma 2: For all $n \geq 1$, there exists $N \geq 1$ such that, for all $k$ with $1 \leq k \leq n$, we have $k|N$.
Proof: By induction on $n$. Base case $n=1$ is trivial. If $M$ is divisible by all $k$ with $1 \leq k < m$, then $Mm$ is divisible by all $k$ with $1 \leq k \leq m$. $\square$
Theorem: For all $n$, there is a prime greater than $n$.
Proof: By Lemma 2, there is an $N$ such that $k|N$ for $1 \leq k \leq n$. By Lemma 1, there is a prime $p$ which divides $N+1$. If $p \leq n$, then $p|N$ and $p|N+1$, a contradiction. So $p>n$, as desired. $\square$.
It is interesting to note that Lemma 2 asserts a relation of exponential growth. (That is to say, it is a theorem of the form $\forall n \exists N : \cdots$ where the smallest $N$ satisfying the $\cdots$ is exponential in $n$.) It is known that certain very weak fragments of PA can't prove the existence of a relation of exponential growth; it would be interesting to know whether they can prove infinitude of primes.