Going to definitions: let's prove two inclusions.
Suppose $x = (x_{\alpha})_{\alpha \in \Lambda}$ is in $\overline{\prod_{\alpha\in\Lambda} A_\alpha}$ and let's show it is in $\prod_{\alpha\in\Lambda}\overline{A_\alpha}$.
Fix $\alpha \in \Lambda$, and an open set $O \subset X_\alpha$ which contains $x_\alpha$. Then ${\pi_\alpha}^{-1}[O]$ ($O$ in coordinate $\alpha$ and the whole space in all other coordinates) is open the product space and contains $x$ and so intersects $\prod_{\alpha \in \Lambda} A_\alpha$, which means that $O$ in particular intersects $A_\alpha$. As $O$ was arbitrary this means that $x_\alpha$ is in $\overline{A_\alpha}$, and as this holds for all $\alpha$, this shows the first inclusion.
Now pick a point in $x= (x_{\alpha})_{\alpha \in \Lambda}$ in $\prod_{\alpha\in\Lambda}\overline{A_\alpha}$, and let $O$ be a basic open neighborhood of $x$ in the product space.
This means that for some finite subset $F = \{ \alpha_1,\ldots,\alpha_n\}$ of $\Lambda$, we have $O_{\alpha_1},\ldots,O_{\alpha_n}$ open subsets of $X_{\alpha_i}$ resp., such that $O = \prod_{\alpha\in\Lambda} O_\alpha$, where $O_\alpha = X_\alpha$ for all $\alpha \in \Lambda\setminus F$.
Every $O_\alpha$ intersects $A_\alpha$, as all $O_\alpha$ are open sets containing $x_\alpha$ and $x_\alpha$ is in $\overline{A_\alpha}$, and so $O$ intersects $\prod_{\alpha\in\Lambda} A_\alpha$. As $O$ was an arbritary basic open subset of $x$, every open neighbourhood around $x$ will intersect $\prod_{\alpha\in\Lambda} A_\alpha$, and so $x$ is in the closure of this set, as needed.
This concludes the proof. Note that almost the exact same proof will also show the same identity when we give the product space the so-called box topology.