By the way, the formula holds in more generality. Using the commutator identities: $[x,yz] = [x,z]z^{-1}[x,y]z,\qquad [xy,z] = y^{-1}[x,z]y[y,z],$ which can be verified by direct computation.
Theorem. If $a$ and $b$ commute with $[b,a]$, then:
- For all integers $n$, $[a^n,b] = [a,b]^n = [a,b^n]$.
- For all integers $n$, $\displaystyle (ab)^n = a^nb^n[b,a]^{n(n-1)/2}$.
Proof.
- If $n=0$ or $n=1$, the identity is trivial. Assuming it holds for $k$, we have $ \begin{align*} [a^{k+1},b] &= [aa^k,b]\\ &=a^{-k}[a,b]a^k[a^k,b]\\ &= [a,b][a^k,b]\\ &= [a,b][a,b]^k = [a,b]^{k+1}. \end{align*}$ So the identity $[a^n,b]=[a,b]^n$ holds for all nonnegative integers. If $k\gt 0$, then $\begin{align*} 1 &= [a^{k}a^{-k},b] = a^{k}[a^{k},b]a^{-k}[a^{-k},b]\\ &= a^k [a,b]^k a^{-k}[a^{-k},b]\\ &= a^ka^{-k}[a,b]^k[a^{-k},b]\\ &= [a,b]^k[a^{-k},b]. \end{align*}$ So $[a^{-k},b] = ([a,b]^k)^{-1} = [a,b]^{-k}$, hence the identity holds for all integers.
Finally, $[a,b^n] = [b^n,a]^{-1} = ([b,a]^n)^{-1} = ([b,a]^{-1})^{n} = [a,b]^n.$ This proves 1.
To prove 2, the result holds for $n=0$ and $n=1$. If the result holds for $k$, then $\begin{align*} (ab)^{k+1} &= (ab)(ab)^k\\ &= aba^kb^k[b,a]^{k(k-1)/2}\\ &=aa^kb[b,a^k]b^k[b,a]^{k(k-1)/2}\\ &=a^{k+1}bb^k[b,a^k][b,a]^{k(k-1)/2}\\ &= a^{k+1}b^{k+1}[b,a]^k[b,a]^{1+2+\cdots+(k-1)}\\ &= a^{k+1}b^{k+1}[b,a]^{1+2+\cdots+k}\\ &= a^{k+1}b^{k+1}[b,a]^{(k+1)k/2}. \end{align*}$ And if $k\gt 0$, then $\begin{align*} (ab)^{-k} &= ((ab)^k)^{-1}\\ &= \left(a^kb^k[b,a]^{k(k-1)/2}\right)^{-1}\\ &= [b,a]^{-k(k-1)/2}b^{-k}a^{-k}\\ &= b^{-k}a^{-k}[b,a]^{-k(k-1)/2}\\ &= a^{-k}b^{-k}[b^{-k},a^{-k}][b,a]^{-k(k-1)/2}\\ &= a^{-k}b^{-k}[b,a]^{(-k)(-k)}[b,a]^{-k(k-1)/2}\\ &= a^{-k}b^{-k}[b,a]^{-k(-k + (k/2)- (1/2)}\\ &= a^{-k}b^{-k}[b,a]^{-k(-k-1)/2}, \end{align*}$ proving the formula for all integers. $\Box$