I am wondering if there is a general formula for doing this.
Consider my favorite simple example: Consider stochastic matrix $P$ with entities $p_{11}=1/3$, $p_{22}=5/6$, $p_{12}=2/3$, $p_{21}=1/6$.
Is there a way to find out what are the recurrent and transient states without having to find a general formula for $P^n$?