If they go to 0 really slowly, their sum can still be infinite. Here's an easy example, much like the harmonic series, but perhaps clearer in how slow the terms go to 0.
Let $a_1 = 1$, $a_2 = a_3 = \frac{1}{2}$, $a_4 = a_5 = a_6 = \frac{1}{3}$, $a_7 = a_8 = a_9 = a_{10} = \frac{1}{4}$, and so on. So, the first term adds to 1, the next 2 terms add to 1, the next 3 terms add to 1, the next 4 terms add to 1, the next 5 terms add to 1, the next 6 terms add to 1, ..., the next $n$ terms add to 1, ... . So, no matter how far out we, the leftover terms still add up to $\infty$.
Remember, the definition of a convergent series is that the sequence of partial sums converges. Just because the terms themselves go to 0 does not imply that the sequence of partial sums eventually converges to something. If the terms of the series go to 0 slowly enough, the partial sums will grow without bound, even though the growth might be very slow, and thus the series will diverge.