Consider an integer $N\geq 2$, the polynomial ring in infinitely many variables $\mathbb Q[T_1,T_2,T_3,\ldots, T_n,...]$ and its quotient $R=\mathbb Q[T_1,T_2,T_3,\ldots, T_n,...]/\langle T_1^N,T_2^N,T_3^N,\ldots, T_n^N,...\rangle=\mathbb Q[t_1,t_2,t_3,\ldots, t_n,...]$ The formal power series series $p(x)=t_1x+t_2x^2+t_3x^3+\ldots+t_nx^n+\ldots \in R[[x]]$ clearly has all its coefficients nilpotent but is nevertheless not nilpotent: this is not trivial but proved in this article by Fields (Proc.AMS,Vol. 27, Number 3, March 1971).
However, he proves that if $R$ is a ring of characteristic $p\gt 0$, then a power series $f(X)=\sum a_ix^i\in R[[x]]$ all of whose coefficients $a_i$ are nilpotent is itself nilpotent iff the orders of nilpotence of the $a_i$'s are bounded : all $a_i^N=0$ for some integer $N$.
Hence if you replace $\mathbb Q$ by $\mathbb F_p$ in the above example, the resulting formal series $p(x)$ is nilpotent.
Fields's article is interesting throughout and extremely elementary: the level is that of the first few chapters of Atiyah-Macdonald's Commutative Algebra.