Here's another proof, not quite as elegant, but with a different flavor that I feel is also worth seeing. In this proof I use a few of the theorems you may have seen in the "group actions" section of the book:
Since conjugate elements have the same order, all nonidentity elements have the same order. Thus only one prime number, $p$, divides the order of the group, since for every prime dividing $|G|$ we have an element whose order is that prime (This follows from Cauchy's Theorem and the Sylow Theorems, which I expect you'll encounter soon). But we know $p$-groups have nontrivial centers (by the class equation, one of the most important elementary results using group actions), and elements of the center are their own conjugacy classes. Since the order of the center is at least $p$ and the center has at most one nonidentity element, $p=2$, and $|Z(G)|=2$. If $|G|$ is 4 or more, then we have elements not in the center, which cannot be conjugate to elements of the center. Thus $|G|=2$.