As Jim Belk has quite rightly pointed out, there do exist infinite non-abelian examples. However, there do not exist any infinite finitely generated examples of non-abelian balanced groups.
Proof:
Suppose $G$ is non-abelian, balanced, infinite, and finitely generated. Group elements are either central or non-central (obviously), and the non-central ones square to some fixed element, $u$ say. As groups do not contain non-trivial idempotents (elements such that $e^2=e$), one has that $u \in Z(G)$.
Noting that for $x$ some non-central generator, $xu\not\in Z(G)$ so $u=(xu)^2=u^3$ and so $u^2=1$. Further, $uw^2=u$ for all $w\in Z(G)$, by the same logic. Thus, $W^2=u$ for all $W\in G\setminus Z(G)$ while $W^2=1$ for all $W \in Z(G)$.
Therefore, every element has order either 2 or 4. One can either plow on ahead (I don't think it is too hard, but it is a tad tedious) to prove the problem outright, or one can apply an early result about Burnsides Problem: a finitely generated group in which each element has order a divisor of 4 is finite*. Ta-da!
*here is a link to a MathOverflow question about where to find a proof of this.