The following is an alternate answer, based on telescoping series, not geometric series. The procedure makes it easy to show that the partial sums are Cauchy, by making it easy to find an explicit bound.
We present first something the OP did not ask for: a proof that the series
$$\frac{1}{1^2}+\frac{1}{2^2} +\frac{1}{3^2}+ \cdots $$
converges.
Note that
$$\frac{1}{(n+1)^2} <\frac{1}{n(n+1)}= \frac{1}{n}-\frac{1}{n+1}$$
It follows that
$$\sum_{k=m+1}^{n} \frac{1}{k^2} <\frac{1}{m}$$
by the usual "telescoping series" argument. We can then use the "Cauchy sequence" criterion to conclude that our original series converges. And then, if $p > 2$, we can use the obvious inequality $1/n^{p} \le 1/n^2$ to deal with $p$-series with $p>2$. In order to do this, we need to know what $n^p$ means. But there is no difficulty if $p$ is an integer, or even just rational.
So far, simple and familiar. We now ask: can the same telescoping series argument be used to deal with the case $1
yes.
We will need to work with inequalities involving $1/n^{p}$. This is fine if we have already introduced the general exponential function, but it becomes problematical otherwise. So we will start with the $p$-series with $p=1+1/2$. Then we will look at the $p$-series with $p=1+1/4$.
We will find, for $p=3/2$, an inequality analogous to the one we used in the case $p=2$. So look at $1/(n+1)^{3/2}$. It would be nice if this were less than $1/n^{1/2} -1/(n+1)^{1/2}$, for then nothing much would change.
That is almost what happens.
$$\frac{1}{n^{1/2}} -\frac{1}{(n+1)^{1/2}}=\frac{(n+1)^{1/2}-n^{1/2}} {n^{1/2}(n+1)^{1/2}}$$
Multiply "top" and "bottom" by $(n+1)^{1/2}+n^{1/2}$. We obtain
$$\frac{1}{n^{1/2}(n+1)^{1/2}((n+1)^{1/2} +n^{1/2})}$$
and this is clearly bigger than
$$\frac{1}{2(n+1)^{3/2}}$$
We conclude that
$$\frac{1}{(n+1)^{3/2}}<\frac{2}{n^{1/2}} -\frac{2}{(n+1)^{1/2}}$$
and now the telescoping series argument works.
We deal next with $p=5/4=1+1/4$.
The natural candidate to compare with $1/(n+1)^{5/4}$ is $1/n^{1/4}-1/(n+1)^{1/4}$. Bring this expression to a common denominator. We obtain
$$\frac{(n+1)^{1/4}-n^{1/4}}{n^{1/4}(n+1)^{1/4}}$$
Rationalize the numerator by multiplying top and bottom by
$(n+1)^{1/4}+n^{1/4}$, and then by $(n+1)^{1/2}+n^{1/2}$. We obtain
$$\frac{1}{n^{1/4}(n+1)^{1/4}((n+1)^{1/4}+n^{1/4})((n+1)^{1/2}+n^{1/2})}$$
This is clearly bigger than
$$\frac{1}{4(n+1)^{5/4}}$$
We conclude that
$$\frac{1}{(n+1)^{5/4}}<\frac{4}{n^{1/4}} -\frac{4}{(n+1)^{1/4}}$$
and again the telescoping argument works.
For $p=9/8=1+1/8$, the same idea works, except that we will need to multiply by $3$ terms to rationalize the numerator. We can continue in this way forever.
So the conclusion is that if $p=1+1/2^k$, the $p$-series converges. For more general $p>1$, just find an integer $k$ such that $1+1/2^k
Alternately, we could use standard inequalities involving the exponential function to show that (for $p>1$), $1/(n+1)^p