I had little success in searching on the web for a rigorous construction in the ring ($A = \mathbb{Z}$) case some time ago and gave it up in the end. However this proved useful so I am going to settle this and write the full solution in the general case of algebras over a (commutative) ring $k$. The following construction was communicated to me by Prof. Gigel Militaru; as far as I understood the original source seems obscure.
So, start with a ground ring $k$ and a (possibly infinite) family of $k$-algebras $(A_i)_{i \in I}$ and let $T$ be the tensor algebra of the direct sum $M$ of all $A_i$ regarded as $k$-modules.
We will use the fact that the tensor algebra is the "smallest" algebra containing a module, in the sense that it comes equipped with a $k$-linear morphism $\alpha : M \to T$ such that for all $k$-algebras $A$ and $k$-linear $f : M \to A$, there is an unique $k$-algebra morphism $g : T \to A$ such that $g \circ \alpha = f$.
Now, let $\alpha_i = \alpha \circ j_i$, where $j_i : A_i \to M$ are the canonical monomorphisms and $\alpha$ is the inclusion just mentioned. Of course, $\alpha_i$ are not yet $k$-algebra maps, since $j_i$ are not multiplicative, so we need to make them so.
Let $I$ be the two-sided ideal generated by all the relations $\alpha_i(ab) - \alpha_i(a)\alpha_i(b)$ and $1_T - \alpha_i(1_{A_i})$. Let $A = T/I$ and $\gamma_i = \pi \circ \alpha_i$, where $\pi : T \to T/I$ is the canonical map. The claim is that $A$ together with all $\gamma_i$ is the coproduct.
It is obvious that $\gamma_i$ are $k$-algebra maps. Let $(B, \eta_i : A_i \to B)$ be a pair consisting of a $k$-algebra $B$ and $k$-algebra morphisms. These extend to a $k$-linear map $\eta : M \to B$, and by the indicated property of $T$, to a $k$-algebra map $\eta' : T \to B$. Since $\eta_i = \eta' \circ \alpha_i$, we can factor through $T/I$ to find the desired morphism for the universal property. Uniqueness follows from uniqueness in the three universal properties.