This is the only result where I have seen the power of Cauchy's criterion of Riemann-Stieltjes integrability. The result you seek to prove can be established using Darboux sums (the traditional proof in Apostol's Mathematical Analysis) but this works only when integrator function $\alpha$ is of bounded variation. For a general bounded integrator function $\alpha$ there appears to be no other option apart from Cauchy's criterion.
Let us then suppose that $f\in R(\alpha) $ and then by Cauchy's criterion for Riemann-Stieltjes integrability for every $\epsilon>0$ there is a partition $P_{\epsilon} $ of $[a, b] $ such that for any partitions $P, Q$ of $[a, b] $ which are finer than $P_{\epsilon} $ we have $$|S(f, P, \alpha) - S(f, Q, \alpha) |<\epsilon $$ We need to prove that $f\in R(\alpha) $ over $[a, c] $ (proof for $f\in R(\alpha) $ over $[c, b] $ is similar). Thus let $\epsilon >0$ be given and let $P_{\epsilon} $ be the partition of $[a, b] $ such that Cauchy's criterion for $f\in R(\alpha) $ over $[a, b] $ holds. If $c\notin P_{\epsilon} $ then we can add point $c$ in $P_{\epsilon} $ and hence it is OK to assume that $c\in P_{\epsilon} $ and note that this does not in anyway affect Cauchy's criterion for $f\in R(\alpha) $ over $[a, b] $.
Let $P_{\epsilon} '=P_{\epsilon} \cap [a, c] $ then we can see that $P_{\epsilon}' $ is a partition of $[a, c] $. We will show that this particular partition of $[a, c] $ is the one which satisfies the Cauchy's criterion for $f\in R(\alpha) $ over $[a, c] $. Let us suppose that $P', Q'$ are arbitrary partitions of $[a, c] $ which are finer than $P_{\epsilon} '$. Let $P_{\epsilon}''=P_{\epsilon} \cap [c, b] $ and $P=P' \cup P_{\epsilon}'', Q=P'\cup P_{\epsilon}''$. Then it can be easily seen that $P, Q$ are partitions of $[a, b] $ which are both finer than $P_{\epsilon} $.
Now we choose a set of tags for partitions $P', Q'$ of $[a, c] $ in arbitrary manner and choose another set of tags for partition $P_{\epsilon} '' $ of $[c, b]$. Combining these sets of tags we get one set of tags for partition $P$ and another set of tags for partition $Q$ in such a manner that the tags for $P, Q$ which lie in interval $[c, b] $ are exactly the same. Consider the Riemann-Stieltjes sums for $f$ over partitions $P, Q$ with these tags. Then it is easily seen that $$S(f, P, \alpha) = S(f, P',\alpha)+S(f,P_{\epsilon}'',\alpha),S(f,Q,\alpha)=S(f,Q',\alpha)+S(f,P_{\epsilon}'',\alpha)$$ We can now see that $$|S(f, P',\alpha)-S(f,Q',\alpha)|=|S(f,P,\alpha)-S(f,Q,\alpha)|<\epsilon $$ and therefore $f\in R(\alpha) $ over $[a, c]$.