Here is a different (much more imprecise and intuitive, but hopefully illuminating, and I believe along the lines of what you were asking) angle on it.
The theorems of complex analysis are so powerful (for computing integrals in terms of residues etc.) that I sometimes forget integrals are really limits of sums. Thinking of the complex integral in these terms:
As in @Did's answer, take $f=1$; everything you need is contained in this example. To get a general feel for what happens when $z$ is inside the circle, take the case that $z=0$. Then the integral is $\int_C \frac{1}{x}dx$ where $C$ is the unit circle, oriented counterclockwise. Use the standard calculus fudge of imagining this as the sum of infinitely many infinitesimal complex numbers. Each infinitesimal summand is $dx/x$ where $x$ lies on the unit circle. $dx$ is a little bit of circle, i.e. an infinitesimal vector tangent to the circle. $x$ is the radius of the circle that ends at this tangent. They are at right angles, so the ratio $dx/x$ is a multiple of $i$. Since the circle is oriented ccw, it is a positive multiple of $i$ and because $x$ has unit length, it has the magnitude of $dx$. So each element $dx/x$ is really an arc length element times $i$. Adding all these up we get the full arc length times $i$, i.e. $2\pi i$.
If $z$ is not zero but still inside the circle, the calculation is less nice, but an essentially similar thing happens: $z$ is always to the left of you as you walk ccw along the circle, so the angle between $dx$ and $x-z$ will stay in a confined range (less than 180 degrees). Then the sum of the elements $dx/(x-z)$ is nonzero since the arguments being close together prevents them from cancelling each other. (In fact it is always $2\pi i$ for this function, but the point is that it is not zero.)
If $z$ is outside the circle, the key difference is that now as you travel along the circle you can't keep $z$ on your left the whole time, so the elements $dx/(x-z)$ will be forced to point in all directions. For example take $z=2$. Then the integral is $\int_C \frac{1}{x-2}dx$. Imagining $x$ somewhere on the circle now, $dx$ is again a little bit of circle, and now $x-2$ is the vector from $2$ to $x$. $\frac{dx}{x-2}$ has a magnitude based on the (changing) magnitude of $x-2$ and its argument is given by the angle between $dx$ and $x-2$. As $x$ goes around the circle, you can see that $dx/(x-2)$ ends up pointing in all directions. This is how the sum of all of them can cancel out. The details of how the cancellation takes place aren't what I'm stressing - you can write out $x-2$ and $dx$ in terms of real and imaginary parts based on a parametrization like $x=\cos\theta + i\sin\theta$ and crunch out the integral to get zero, but I didn't find that very illuminating, and couldn't see a simpler geometric argument that they cancel out perfectly. The main point is just that because the circle does not enclose $2$, as $x$ goes along the circle, $2$ is not strictly to your left as you walk around the circle, so the value of the elements $dx/(x-2)$ have to double back on themselves.
A good deal of this discussion has been specific to the function $f=1$, but the main point is actually extremely general: If $z$ inside the loop, then going around the loop you can keep $z$ on your left the whole time, and an integral of $f/(x-z)$ can exploit this to accumulate some nonzero value. If $z$ is outside the loop, then as you go around the loop, $z$ will be on your left sometimes and your right sometimes, and under the right geometric conditions on $f$ (specifically, holomorphic in a region containing the unit disc), this forces all the elements $fdx/(x-z)$ to cancel.