Here is an unnecessarily slick answer:
There is a famous Lebesgue criterion for Riemann integrability of a function $f: [a,b] \rightarrow \mathbb{R}$ (my colleague Roy Smith informs me that it can actually be found already in the work of Riemann!): it is necessary and sufficient that $f$ be bounded and that its set of discontinuities have (Lebesgue!) measure zero.
Given this: it is an easy exercise to show that modifying a function by changing its values at any finite set $S$ does not change its boundedness/unboundedness, and similarly could only create or destroy discontinuities at $x$ for $x \in S$. So the Lebesgue criterion applies here. (Beware: changing a function at a countable set of values only can change the continuity at every point: I leave it to the reader to supply the canonical example of this.)
Of course one can -- and should -- also show this directly from the definition of Riemann integrability.