Yes, the direct sum of free modules is free. One way to show it is to show it has a basis. Let $\beta_i$ be a basis for $F_i$; then $\cup \beta_i$ is a basis for the direct sum (interpret an element of $\beta_i$ as tuple that has $0$ $j$th entry for every $j\neq i$).
Another is note that a module $M$ is free if and only if there exists a set $X$ such that $M\cong R^{(X)}$, where $R^{(X)}$ is the set of functions $f\colon X\to R$ with finite support, with coordinatewise addition and scalar multiplication; then show that $\bigoplus_{i\in I} R^{(X_i)} \cong R^{(\mathop{\amalg}\limits_{i\in I}X_i)}$ where $\amalg$ is the disjoint union.
Yet a third way is to remember the "free module" is the left adjoint of the underlying set functor, and therefore respects coproducts; so the direct sum (coproduct) of free modules is the free module on the coproduct of their bases (this is just the abstract nonsense way of justifying the second proof mentioned above).