The WolframAlpha answer suggests using a substitution of the form $x = \sqrt{3} \sin^2 u$, but I'd suggest using $x = \sqrt{3} \csc^2 u$, if you want to avoid complex-valued results. You'll get an integrand with a factor of $\sqrt{1+\sin^2u}$ in it. To get an answer, you're going to have to integrate by parts (and use an identity) a few times to reduce the order/degree, until you get down to integrands of the form $1/\sqrt{1+\sin^2u}$ and $\sqrt{1+\sin^2u}$, whose antiderivatives are elliptic functions of the first kind and the second kind, $F(u\,|\,{-1})$ and $E(u\,|\,{-1})$, respectively. If you have the patience to figure that all out, you'll realize the WA answer is not uglier than one should expect, except perhaps that it gives complex values instead of real ones.
(You can also integrate your original integral by parts first, using the identity $-3 = (x^2-3)+x^2$, until you reach an integrand of the form $x^{1/2}\sqrt{x^2-3}$; and then do the trig. sub.)