What follows is a variant of the method suggested by Fahad Sperinck, which almost gave the desired bound. Although we obtain a pretty short proof of the inequality, I think that the "right" proof is the one in the post by David Speyer. (A proof based on geometry is "right," as is a combinatorial proof.)
Let us start as Fahad Sperinck did, from $ \int_n^{n+1} \frac{x-[x]}{x^2}\: dx = \log\Big(\frac{n+1}{n}\Big) - \frac{1}{n+1} < \frac{1}{n} - \frac{1}{n+1} -\frac{1}{2n^2} +\frac{1}{3n^3}. $
Ultimately, we will summing from $N$ to infinity. If we keep this fact in mind, the chunk $ \frac{1}{n}-\frac{1}{n+1} $ sums beautifully to $1/N$, and should be left as is. If we could show that the part that is taken away, namely $\frac{1}{2n^2}-\frac{1}{3n^3}$ is bigger than $\frac{1}{2}\left(\frac{1}{n}-\frac{1}{n+1}\right),$ we would be finished.
Now I will do some unofficial scribbling, don't look. I want to show that $1/2n^2-1/3n^3 \ge 1/2(n)(n+1)$, so I want to show that $(3n-2)/6n^3\ge 1/2n(n+1)$, so I want to show that $(3n-2)/3n^2 \ge 1/(n+1)$, so I want to show that $(3n-2)(n+1) \ge 3n^2$, and this is clearly true if $n \ge 2$, just multiply out the stuff on the left.
Now if I had the energy I would hide my tracks, and have the desired inequality drop out as if by magic.
Comment: Somehow, one acquires the habit of thinking of $n^2$ and $1/n^2$ as "nice" and of $n(n+1)$ and $1/n(n+1)$ as not so nice. In many ways, the opposite is true. Certainly that is the case from the combinatorial point of view.
The calculations in the post were fine, the problem was that of giving away a tiny bit too much. That was, maybe, because the strategy was directed at getting to something that looks like $1/n^2$, which was viewed as tractable and desirable. But $1/n(n+1)$, aka $1/n-1/(n+1)$, arises naturally in the problem, and is much more tractable.