'Weak topology' has more than one meaning; is your $\mathbb R^\infty$ the space of sequences in $\mathbb R^\omega$ that are eventually $0$, topologized so that $U \subseteq \mathbb R^\infty$ is open iff each $U \cap \mathbb R^n$ is open in $\mathbb R^n$, where $\mathbb R^n$ is identified with $\{(x_i:i \in \omega) : \forall i \ge n (x_i = 0)\}$? If so, I can interpret $(0,\dots, a_n,\dots,0)$ as a sloppy notation for a sequence $(a_0,a_1,\dots)$ in which at most the term $a_n \ne 0$. I still don't see any reasonable interpretation of the original doubly-indexed version, but the singly-indexed version can be understood in several ways. Perhaps the most straightforward is as follows.
For $n \in \omega$ let $\alpha_n = (a^n_i:i \in \omega) \in \mathbb R^\infty$. Suppose further that for each $n \in \omega$ there is an index $\iota(n) \in \omega$ such that $a^n_i = 0$ whenever $i \neq \iota(n)$. The claim would then be that if $(\alpha_n:n \in \omega)$ converges, then $a^n_{\iota(n)} = 0$ for all sufficiently large $n$. On this interpretation the claim is false: just take $\alpha_n = (1,0,0,\dots)$ for all $n$.
In the special case in which the range of the function $\iota$ is infinite, however, the result is correct. Suppose that $(\alpha_n:n \in \omega)$ converges; clearly the limit must be the zero sequence $\zeta = (0,0,\dots)$. If the $a^n_{\iota(n)}$ are not eventually $0$, there is an infinite $I \subseteq \omega$ such that $\iota \restriction I$ is 1-1 and $\forall n \in I (a^n_{\iota(n)} \ne 0)$. Let $U = \{(x_n:n \in \omega) \in \mathbb R^\infty: \forall n \in I(|x_n| < |a^n_{\iota(n)}|)\}$; $U$ is an open nbhd of $\zeta$, but $\forall n \in I(\alpha_n \notin U)$, $\rightarrow\leftarrow$.