First of all, get all the stuff you know on one side, and all the stuff you don't on the other: $\dfrac{\sin(180^\circ/n)}{1-\sin(180^\circ/n)} = \dfrac{r}{R}$
Now, for convenience I'm going to introduce a new variable, $t$, just to hold on to $\sin(180^\circ/n)$ - once we've got the expression written down in terms of $t$ we can 'unpack' it again. This gives us $\dfrac{t}{1-t} = \dfrac{r}{R}$. We can multiply both sides by $(1-t)$ and expand: $t = \dfrac{r}{R}(1-t) = \dfrac{r}{R}-t\dfrac{r}{R}$. Now, add $t\dfrac{r}{R}$ to both sides, and factor out the factor of $t$ on the left: $t+t\dfrac{r}{R} = \dfrac{r}{R}$, or $t(1+\dfrac{r}{R}) = \dfrac{r}{R}$. Divide out by the factor on the left, and then finally multiply numberator and denominator on the right by R to clear the fractions: $t=\dfrac{\frac{r}{R}}{1+\frac{r}{R}} = \dfrac{r}{R+r}$. Now that we have an expression in terms of $t$, we can reintroduce our $n$: $\sin(180^\circ/n) = \dfrac{r}{R+r}$. Now just apply arcsin to both sides and divide $180^\circ$ into both sides: $\dfrac{180^\circ}{n} = \arcsin\left(\dfrac{r}{R+r}\right)$, or finally:
$n=\dfrac{180^\circ}{\arcsin\left(\dfrac{r}{R+r}\right)}$
Also, one thing I strongly encourage doing whenever you're solving a problem like this: once you have what you think is the final formula, test it! In this case, we know that six equally-sized circles will fit around a circle, so if we plug in $R=r$ we should find $n=6$: $\begin{align} n &= \dfrac{180^\circ}{\arcsin\left(\dfrac{r}{R+r}\right)} \\ &= \dfrac{180^\circ}{\arcsin\left(\dfrac{r}{r+r}\right)} \\ &= \dfrac{180^\circ}{\arcsin\left(\dfrac{r}{2r}\right)} \\ &= \dfrac{180^\circ}{\arcsin(\frac{1}{2})} \\ &= \dfrac{180^\circ}{30^\circ} \\ &= 6 \end{align} $