Yes indeed Lagrange MVT will work ,
Let $a<\delta_1
If we start with the following expression,
$\displaystyle \frac{1}{b-a}\left(\frac{f(c)-f(a)}{c-a}-\frac{f(c)-f(b)}{c-b}\right)$
Now, There must exist $\delta_1\in(a,c)$ & $\delta_2\in(c,b)$ such that $\displaystyle f'(\delta_1)=\frac{f(c)-f(a)}{c-a}$ & $\displaystyle f'(\delta_2)=\frac{f(c)-f(b)}{c-b}$ by Lagrange MVT
Now suppose $\delta_1<\xi<\delta_2$ then since $f'(x)$ exists in $(a,b)$ and continuous on $[\delta_1,\delta_2]$ so again by Lagrange MVT we have ,
$\displaystyle f''(\xi)=\frac{f'(\delta_2)-f'(\delta_1)}{\delta_2-\delta_1}$
So the very first expression turns,
$\displaystyle \frac{1}{b-a}\left(\frac{f(c)-f(a)}{c-a}-\frac{f(c)-f(b)}{c-b}\right) = \frac{f'(\delta_2)-f'(\delta_1)}{b-a}=\frac{f'(\delta_2)-f'(\delta_1)}{2(\delta_2-\delta_1)}=\frac{1}{2}f''(\xi)$
where $\displaystyle \left(\frac{b-a}{2}=\delta_2-\delta_1\right)$
Since $f(a)=f(b)=0$ this is equivalent to $\displaystyle f(c)=\frac{1}{2}(c-a)(c-b)f''(\xi)$