I would not differentiate twice, only because of the simple reason that you don't even know that $f$ is differentiable. However, there is a geometric way of looking at this. A convex function $f$ is defined by the following property: the graph of $f(x)$ between $x = a$ and $x = b$ lies below the line connecting the points $(a,f(a))$, $(b, f(b))$. I would like to say that pictorially I can think of the averages of the endpoints as always being larger than the full average over the whole interval $(a,b)$. So when you're averaging the average, this should remain the case. But maybe that isn't really precise enough...
Sean Eberhard's change of variables is very useful for formalizing this; for $r \in [0,1]$, $ rg(a) + (1-r)g(b) = \int_0^1 rf(as) + (1 -r)f(bs) \, ds \geq \int_0^1 f(ras + (1-r)bs) \, ds = g(ra + (1 - r)b). $