There are many ways to find the power series that corresponds to a function. As is often the case in mathematics, you try to reduce the problem at hand to a problem you already know how to solve. So if you know the geometric power series you try to reduce new problems to that, which the examples do. An example would be if you want to do $\sum_{n=0}^{\infty} n x^n$. Since you know $\sum_0^{\infty}x^n=\frac{1}{1-x}, \sum_{n=0}^{\infty} nx^n=\sum_0^{\infty}x\frac{d}{dx}x^n$ Now you need to justify taking the derivative outside the sum. There are beautiful theorems that tell you when this is justified. We physicists ignore them, as all of our functions satisfy them.
Another example would be the series for $\sin(x)$ near 0 (the Maclaurin series). Taylor's theorem says $\sin(x)=\sum_{n=0}^{\infty}\frac{\sin^{(n)}(0)}{n!}x^n$ where $\sin^{(n)}(0)$ is the $n^{th}$ derivative of $\sin(x)$ evaluated at 0. If you know that the $n^{th}$ derivative of $\sin (x)$ at 0 is 0 if $n$ is even and $-1^{(n-1)/2}$ if $n$ is odd this becomes the familiar $\sin(x)=\sum_{n=1,n \text{ odd}}^{\infty}\frac{-1^{\left(\frac{n-1}{2}\right)}x^n}{n!}$
Once you have this (and the corresponding ones for other functions like e^x, cos(x), etc) you can reduce problems to these as well. Then, as Arjang commented, if you try to operate with differential and integral operators, you need to worry about whether this is justified.