Let $k$ be a commutative ring and $V$ be a $k$-module (possibly graded). Let $TV$ denote the free (graded) algebra on $V$. The following question could just as well be applied to the free (graded) commutative algebra on $V$, but since it is a quotient of the former it suffices to consider the non-commutative case.
For a free $V$, what $TV$ is is easy to describe: by choosing a basis, we get an isomorphism $V\cong k^{(I)}$, and then $TV\cong K\langle x_i:i\in I\rangle$ where this denotes polynomials in non-commuting variables.
But what about non-free $V$? Then not every module is free, but at least it is a quotient of a free module. So we have an exact sequence $0\to E\to F\to V\to 0$ where $F$ is free; does this help in describing $TV$ as a quotient of $TF$ (which is easy since $F$ is free)? I'm unsure because the tensor product being a right exact functor, I don't expect there to be an associated short exact sequence of tensor algebras.
I would ideally like to see some concrete examples as well... I get the impression that $T(\mathbb{Z}/2)$ (over $\mathbb{Z}$) is $\mathbb{Z}\oplus \mathbb{Z}/2[X]^+$ where $+$ denotes the augmentation ideal, and similarly for the other cyclic rings, for example.