Another collection of examples, not yet mentioned, are the Prüfer groups (also known as quasicyclic groups); it is sometimes denoted $\mathbb{Z}_{p^{\infty}}$. They are infinite, but every proper subgroup is finite (and every nontrivial quotient is isomorphic to the original group). Here are three descriptions:
For a fixed prime $p$, let $\mathbb{Z}_{p^{\infty}}$ be the group of all $p^k$-th complex roots of unity for all $k\geq 0$, with the group operation being multiplication. It is not hard to show that every proper subgroup is generated by a $p^n$th primitive complex root of unity for some $n$, hence is cyclic.
As an alternative description, consider subgroup of the additive group $\mathbb{Q}/\mathbb{Z}$ that consists of all classes represented by a fraction whose denominator is a power of $p$.
A final description: consider the collection of groups $\{\mathbb{Z}/p^n\mathbb{Z}\}_{n\in\mathbb{N}}$, with injections $i_n\colon\mathbb{Z}/p^n\mathbb{Z}\to\mathbb{Z}/p^{n+1}\mathbb{Z}$ given by $i_n(a+p^n\mathbb{Z}) = pa+p^{n+1}\mathbb{Z}$. Then $\mathbb{Z}_{p^{\infty}}$ is the direct limit $\displaystyle\lim_{\longrightarrow}\mathbb{Z}/p^n\mathbb{Z}$ of this direct system.
The additive group $\mathbb{Q}$ is almost such a group, in that every finitely generated subgroup is cyclic, but of course it contains noncyclic proper subgroups (e.g., the pre-image of the Prüfer group for some $p$).