You have to understand that the notion of function as it is used nowadays is quite recent. During a long time, analysts were perfectly happy to work with so-called multiform functions. For example, $\log : \mathbb C \setminus \{0\} \to \mathbb C$ was a perfectly fine function to work with, even if you had to be careful : if the argument of this “function” makes a turn around 0, the image of $\log$ changes (something like $2i\pi$ is added).
One of the great things Riemann did is to notice that this creates a true (“uniform”) function on something slightly more complicated. In modern language, there is a Riemann surface $X$, a function $p : X \to \mathbb C \setminus \{0\}$ (actually, $X = \mathbb C$ and $p=\exp$, but allow me to forget that) and a true function $f : X \to \mathbb C$ such that the various “values” of $\log(z)$ are just the $f(\tilde z)$ for $\tilde z \in p^{-1}\{z\}$. And the fact that the “value” of $\log$ changes when you make a turn around 0 is just the fact that your loop around 0 is the image of no loop in $X$ : when you try to lift that loop to $X$, you get a non closed path. The difference in value for $\log$ (the “monodromy”) is just the consequence of the topology of $X$. (Of course, that quickly leads to covering spaces, etc.) Somehow, the main change of focus is here: to understand functions, you have to understand Riemann surfaces.
Now, for what kind of functions do you want to play that game? If I remember correctly, Riemann quotes the example of the logarithm in his article, but he is mainly interested in (inverses of) algebraic functions, which give birth to compact Riemann surfaces and finite (branched) coverings over the Riemann sphere. A goal was to understand elliptic and Abelian integrals, functions that were becoming more and more important in physics, analysis and number theory...
A later source of examples was the theory of differential equations. We now tend to look at these from a real point of view (that is, with a real time $t \in \mathbb R$) but the complex point of view used to be more important. Differential equations, even if they are linear and of order two, also define “multiform functions” that you want to understand better. This was for example the main motivation of Poincaré, whose work (in that subject) culminated with the uniformisation theorem. (Note: ”uniformisation”, because you want uniform functions!)
Of course, everything I say is grossly simplified. For a start, every word here is an anachronism, because a lot of complex analysis, algebraic geometry, even surface topology was developed precisely during that period between 1851 (Riemann's first article) and 1907 (Poincaré and Koebe's first convincing proofs of the uniformisation theorem). But it is certainly true that a huge part of the second half of the mathematical nineteenth century revolved around such notions...
If you want to read more about that, McKean and Moll's Elliptic Curves is a great book to understand the relation between those innocent-looking elliptic integrals and that somehow fearsome theory of Riemann surfaces. And if you read French, there are two relevant historical/mathematical books: Dieudonné's Cours de géométrie algébrique (first volume) and Saint-Gervais's Uniformisation des surfaces de Riemann. Normally, a pdf version of the latter exists somewhere on the internet, but I'm afraid you'll need a good library to find the former. (How can such a wonderful book not to be reedited?)