You will need to show that for every element $x$ in the product and every box topology neighborhood $U$ of $x$, there is an element of $B$ such that $x\in B\subseteq U$.
Edit: Once you've done that, you need only confirm that every element of $B$ is open in the box topology on $X$--which should be a fairly simple task, as elements of a basis $B_a$ of $X_a$ are necessarily open in $X_a$. Then by Lemma 13.2 of Munkres, $B$ will be a basis for the box topology on $X$, as desired.
Note that this last detail is actually essential to include! (I neglected to mention it because it should be clear that the elements of $B$ are open in the box topology on $X$, but in hindsight, I shouldn't have omitted that detail. Apologies for any confusion!) For example, if we take $B'$ to be the set of all singletons $\{x\}$ (for $x\in X$), the set $B'$ rather trivially satisfies the condition I mentioned in the first place, but while it is a basis, it is a basis for the discrete topology--strictly a finer topology (in general) than the box topology.
All the condition I mentioned at first really shows is that if $B$ is a basis for some topology, then it generates a topology finer than the box topology on $X$ (possibly strictly finer). Knowing that the elements of $B$ are open in the box topology allows us to conclude that (1) $B$ is a basis for a topology on $X$, and that (2) the generated topology is coarser than (so equivalent to, since also finer than) the box topology on $X$.