You’re on the right track. By definition $G_x=\{g\in G:g\cdot x=x\}$. Suppose that $g,h\in G_x$; then $(gh)\cdot x=g\cdot(h\cdot x)=g\cdot x=x\;,$ so $gh\in G_x$, and $G_x$ is closed under the group operation. Moreover, $g^{-1}\cdot x=g^{-1}\cdot(g\cdot x)=(g^{-1}g)\cdot x=1_G\cdot x=x\;,$ so $g^{-1}\in G_x$, and $G_x$ is closed under taking inverses. Thus, $G_x$ is indeed a subgroup of $G$. To finish the proof, you need only verify that there is a bijection between left cosets of $G_x$ in $G$ and the orbit of $x$.
Added: The idea is to show that just as all elements of $G_x$ act identically on $x$ (by not moving it at all), so all elements of a left coset of $G_x$ act identically on $x$. If we can also show that each coset acts differently on $x$, we’ll have established a bijection between left cosets of $G_x$ and members of the orbit of $x$.
Let $h\in G$ be arbitrary, and suppose that $g\in hG_x$. Then $g=hk$ for some $k\in G_x$, and $g\cdot x=(hk)\cdot x=h\cdot(k\cdot x)=h\cdot x\;.$ In other words, every $g\in hG_x$ acts on $x$ the same way $h$ does. Let $\mathscr{G}_x=\{hG_x:h\in G\}$, the set of left cosets of $G_x$, and let
$\varphi:\mathscr{G}_x\to\operatorname{Orb}(x):hG_x\mapsto h\cdot x\;.$
The function $\varphi$ is well-defined: if $gG_x=hG_x$, then $g\in hG_x$, and we just showed that in that case $g\cdot x=h\cdot x$.
It’s clear that $\varphi$ is a surjection: if $y\in\operatorname{Orb}x$, then $y=h\cdot x=\varphi(hG_x)$ for some $h\in G$. To complete the argument you need only show that $\varphi$ is injective: if $h_1G_x\ne h_2G_x$, then $\varphi(h_1G_x)\ne\varphi(h_2G_x)$. This is perhaps most easily done by proving the contrapositive: suppose that $\varphi(h_1G_x)=\varphi(h_2G_x)$, and show that $h_1G_x=h_2G_x$.