First, use the substitution $t = x^2$ to obtain
$ I := \int_{0}^{\infty}\sin\left(x^2\right)\;dx=\int_{0}^{\infty}\frac{\sin t}{2\sqrt{t}}\;dt. $
This integral converges conditionally, thus we make an integration by parts to obtain an absolutely convergent integral as follows:
$I = \left[\frac{1-\cos t}{2\sqrt{t}}\right]_{0}^{\infty}-\int_{0}^{\infty}\left(\frac{d}{dt}\frac{1}{2\sqrt{t}}\right)(1 - \cos t)\;dt =\int_{0}^{\infty}\frac{1 - \cos t}{4t^{3/2}}\;dt.$
Now, from gamma integral,
$ \begin{align*} I &=\frac{1}{4\Gamma(3/2)}\int_{0}^{\infty}\left(\frac{\Gamma(3/2)}{t^{3/2}}\right)(1 - \cos t)\;dt \\ &=\frac{1}{4\Gamma(3/2)}\int_{0}^{\infty}\left(\int_{0}^{\infty}u^{1/2}e^{-tu}\;du\right)(1 - \cos t)\;dt\\ &=\frac{1}{4\Gamma(3/2)}\int_{0}^{\infty}\int_{0}^{\infty}u^{1/2}e^{-tu}(1 - \cos t)\;dudt \\ &=\frac{1}{2\sqrt{\pi}}\int_{0}^{\infty}\int_{0}^{\infty}u^{1/2}e^{-tu}(1 - \cos t)\;dudt, \end{align*} $
where in the last line we have used the fact that $\Gamma(3/2) = \frac{1}{2}\sqrt{\pi}$, which is a direct consequence of the Gaussian integral. (Of course, this integral can be evaluated by a famous real analysis technique.) By Tonelli's theorem we can change the order of integration, and with the substitution $u = v^2$ we obtain
$\begin{align*}I &=\frac{1}{2\sqrt{\pi}}\int_{0}^{\infty}\int_{0}^{\infty}u^{1/2}e^{-tu}(1 - \cos t)\;dtdu \\ &= \frac{1}{2\sqrt{\pi}} \int_{0}^{\infty}\frac{u^{1/2}}{u(1+u^2)}\;du = \frac{1}{2\sqrt{\pi}} \int_{0}^{\infty} \frac{2}{1+v^4} \; dv. \end{align*}$
To evaluate the last integral, we use the following decomposition
$ \frac{2}{1+v^4} = \frac{1+v^{-2}}{(v-v^{-1})^2-2} - \frac{1-v^{-2}}{(v+v^{-1})^2-2}. $
Thus with the substitution $z = v - v^{-1}$ and $w = v + v^{-1}$, the integral becmes
$\begin{align*}I &= \frac{1}{2\sqrt{\pi}} \left( \int_{0}^{\infty} \frac{d(v-v^{-1})}{(v-v^{-1})^2+2} - \int_{0}^{\infty} \frac{d(v+v^{-1})}{(v+v^{-1})^2-2} \right) \\ &= \frac{1}{2\sqrt{\pi}} \left( \int_{-\infty}^{\infty} \frac{dz}{z^2+2} - \int_{\infty}^{\infty} \frac{dw}{w^2-2} \right) = \frac{1}{2\sqrt{\pi}} \left( \frac{\pi}{\sqrt{2}} - 0 \right) = \sqrt{\frac{\pi}{8}}. \end{align*}$
Only a slight modification of this argument immediately yields
$ \int_{0}^{\infty} \cos\left(x^2\right) \; dx = \sqrt{\frac{\pi}{8}}. $