Imagine a probability distribution. Now imagine its density or mass function. Imagine that the graph of the function is actually printed out on a piece of paper (possible an 'infinitely big' one) - or actually, on a thin wooden board. Now imagine cutting out the bit of the wooden board that is inside the function, leaving a thin sliver along the x-axis wherever needed in order to connect the whole function together into one cutout.
Lastly, imagine trying to balance this wooden cutout of the p{m,d}f on a point. Is the point on which it would balance always the expectation of the distribution?
I think of course you'd always say that the 'along the x-axis' bits have weight 0, even though in real life they would of course have a weight.