Recently I've found lecture notes on "Analysis on Graphs" where the potential theory methods were used to study discrete-time, time-reversible Markov chains (i.e. the state space is countable).
Among such methods there was the Dirichlet problem for Laplace equations; Cheeger and isoperimetric inequalities were used to lower-bound second eigenvalue of the Laplace operator. On of the main tool used was discrete Green's formula: $ \int\limits_A\Delta f(x)g(x)\pi(dx) = -\frac12\int\limits_{A}\nabla_{xy}f\cdot\nabla_{xy}g\;\pi(dx)+\int\limits_{A}\int\limits_{A^c}(\nabla_{xy}f)g(x)P(x,dy)\pi(dx). $ where the set $A$ is measurable, $\Delta = I-P$ is a Laplace operator ($P$ is a Markov operator, $I = P^0$ is the identity one) and $\nabla_{xy}f = f(x) - f(y)$ is the discrete gradient.
Clearly, for the uncountable state space the same formula holds (for any measurable $A$) if the Markov process is time-reversible w.r.t. measure $\pi$. I am interested if in the general case (not time-reversible) Green's formula is still so powerful tool, and which measure $\pi$ should be taken.
With this thoughts I started looking for the book on potential theory for discrete-time, general-space Markov processes. Currently I am reading 'Markov Chains' by D. Revuz, there is a chapter on potential theory - but methods used there are different and based mostly on potential operator $ G = I + P + P^2+\dots $
My questions are the following:
I am looking for the reference which treats potential theory for Markov processes in a discrete time with the use of Dirichlet forms (even if it is fruitful only for time-reversible processes). Especially I am interested in the functions which are minimums for these forms. Again, in the general case, which measure $\pi$ should be used?
Maybe, there is book devoted to potential theory of discrete-time Markov processes in general, because usually I saw only chapters in other books.
In his book, Revuz considers only non-negative superharmonic functions. This does not seem to be a restriction (w.r.t. considering bounded from below superharmonic functions) since e.g. Riesz decomposition holds also in the latter case - it just brings a bit of an inconvenience, since for the supmarkovian kernels constants are not always harmonic, so simple shift does not help. Is there any reason to consider non-negative superharmonic functions rather than bounded from below?
The part of this question I asked on MO but didn't get an answer.