Theorem. Let $p$ be a prime, and let $G$ be a group of order $p^n$. Then $G$ has subgroups of order $p^i$ for each $i$, $0\leq i\leq n$.
Lemma. Let $G$ be a group of order $p^n$, where $p$ is a prime. Then $Z(G)\neq\{1\}$.
Proof of Lemma. This follows form the class formula: define an equivalence relation on $G$ letting $a\sim b$ if and only if there exists $g\in G$ such that $gag^{-1}=b$. Since $gag^{-1}=hah^{-1}$ if and only if $h^{-1}ga = ah^{-1}g$, if and only if $h^{-1}g\in C_G(a)$, if and only if $h$ and $g$ represent the same coset modulo $C_G(a)$, it follows that the equivalence class of $a$ has as many elements as the index of $C_G(a)$.
Now let $a_1,\ldots,a_k$ be representatives from the equivalence classes of $G$. Then $p^n = |G| = \sum_{i=1}^k|\{b\in G\mid a_i\sim b\}| = \sum_{i=1}^k[G:C_G(a_i)].$ Now, there is at least one element whose centralizer is all of $G$: the identity; and since only the identity is related to $e$, one of the $a_i$ must equal $e$. Thus, the sum on the right contains terms that are not congruent to $0$ modulo $p$. But the sum is $0$ modulo $p$, so there is more than one $i$ for which $[G:C_G(a_i)] = 1$. In particular, there is an $a_i\neq e$ such that $C_G(a_i)=G$, which means $a_i\in Z(G)$. $\Box$
Proof of Theorem. Induction on $n$. If $|G|=p$, then $G$ is cyclic of order $p$, and has subgroups of orders $1$ and $p$.
Assume by induction the result holds for groups of order $p^n$, and let $G$ be a group of order $p^{n+1}$. Let $a\in Z(G)$ be an element of order $p$ (it exists, since $Z(G)$ is not trivial), and let $N=\langle a\rangle$. Then $N\triangleleft G$, since for all $g\in G$, $gag^{-1}=a$; so $K=G/N$ is a group of order $p^n$. By the induction hypothesis, $K$ has subgroups $K_0,\ldots,K_n$ of order $p^0,p^1,\ldots,p^n$, respectively. For each $i$, let $H_i = \{g\in G\mid gN\in K_i\}$. By the isomorphism theorems, $H_i$ are subgroups of $G$, and $H_i/N\cong K_i$; therefore, $|H_i|=|K_i||N| = p^i\times p = p^{i+1}$, so $G$ has subgroups $H_0,\ldots,H_n$ of orders $p^1,\ldots,p^{n+1}$. Together with the identity, we get that $G$ has subgroups of order $p^i$ for $i=0,\ldots,n+1$, as desired. $\Box$
Corollary. Let $G$ be a group of order $p^n$, $p$ a prime, and $n\gt 1$, in which every element is exponent $p$. Then $G$ contains a subgroup isomorphic to $C_p\times C_p$.
Proof. Let $H$ be a subgroup of $G$ of order $p^2$ (which exists by the Theorem). Then $H$ is of order $p^2$, so the center has order at least $p$; but $H/Z(H)$ cannot be cyclic and nontrivial, so $Z(H)$ cannot be of order exactly $p$, hence $Z(H)=H$ and $H$ is abelian. Thus, $H$ is an abelian group of order $p^2$ in which every element is order $p$; the only possibility is $H\cong C_p\times C_p$. $\Box$