I'm not sure what part you are trying to be rigorous with but here is another approach:
Let $\epsilon >0$. Since $f$ in integrable, we can find $M$ such that $\int_{[-M,M]^C} |f| < \frac{1}{2}\epsilon$. Then we have (using the fact that $\cos$ is Lipschitz of rank 1): \begin{eqnarray} |F(t)-F(s)| & = & |\int_{[-M,M]} f(x)(\cos(tx)-\cos(sx)) dx| \\ &\leq& \int_{[-M,M]} |f(x)||(\cos(tx)-\cos(sx))| dx \\ &\leq& \int_{[-M,M]} |f(x)||t-s||x| dx \\ &\leq& M |t-s| \int_{[-M,M]} |f(x)| dx \\ &\leq& M |t-s| \|f\|_1 \end{eqnarray} Now choose $|t-s| \leq \frac{1}{2 M \|f\|_1}\epsilon$, then $|F(t)-F(s)| < \epsilon$.
Addendum: (I missed the Riemann Lebesgue lemma part of the problem originally.)
We use the fact that smooth compactly supported functions $C_c^\infty(\mathbb{R})$ are dense in $L^1(\mathbb{R})$. If $f \in C_c^\infty(\mathbb{R})$, fix $t\neq 0$ and then let $\phi_t(x) = f(x) \frac{\sin(xt)}{t}$. Note that $\|\phi_t\| \leq \frac{1}{|t|} \|f\|_1$. We have $\phi_t'(x) = f'(x)\frac{\sin(xt)}{t} + f(x) \cos(xt)$, and for $|x|$ sufficiently large, $\phi_t(x) =0$. Hence for any $M$ sufficiently large, $0 = \phi_t(M)-\phi_t(-M) = \int_{-M}^M \phi_t'(x) dx = \int_{-M}^M (f'(x)\frac{\sin(xt)}{t} + f(x) \cos(xt)) dx$. Hence $\int f(x) \cos(xt) dx = -\int f'(x)\frac{\sin(xt)}{t} dx$, and so $| \int f(x) \cos(xt) dx | \leq \frac{1}{|t|} \|f'\|_1$. It follows that $\lim_{t \to \infty} \int f(x) \cos(xt) dx = 0$.
Now let $f \in L^1(\mathbb{R})$, $\epsilon > 0$ and choose $g \in C_c^\infty(\mathbb{R})$ such that $\|f-g\|_1 < \frac{1}{2} \epsilon$. Then \begin{eqnarray} |F(t)| &\leq &|\int (f(x) -g(x))\cos(xt) dx + \int g(x)\cos(xt) dx| \\ &\leq & \|f-g\|_1 + |\int g(x)\cos(xt) dx| \\ &\leq & \frac{1}{2} \epsilon + \frac{1}{|t|} \|g'\|_1 \end{eqnarray} Hence for $|t| $ sufficiently large, we have $|F(t)|< \epsilon$. The desired result follows.