-1
$\begingroup$

If the problem of optimal stopping for finite state discrete time Markov Chains is solved on the infinite horizon explicitly?

Edited: This means if for a given MC $X(n)$ with a state space $x_1,...,x_n$, transition matrix $P$; and a vector $g_1 = g(x_1),...,g_n = g(X_n)$ there is an explicit solution to the problem $ v_i = \sup\limits_{\tau<\infty}\mathsf{E}[g(X(\tau))|X_0 = x_i]. $

  • 0
    @Didier Piau, thanks for redirecting ) I was thinking that everyone see new messages in his inbox regardless of @ thing - just because he already wrote a comment on this question. Now it seems more clear.2011-06-08

1 Answers 1

2

Yes, it is solved in a finite number of steps by so called the State Elimination algorithm (for Markov chains), see the papers of Isaac M. Sonin.

  • 0
    That's yours? Ok, thanks )2011-06-07