2
$\begingroup$

Suppose Markov Chain has two states $1,2$ and transition probability $$P_{1,2}=q,\;P_{1,1}=1-q,\;P_{2,1}=p,\;P_{2,2}=1-p$$

Hence we have transition Matrix $$M=\begin{pmatrix}1-p & p\\q & 1-q\end{pmatrix}$$ At state $1$ payoff $2000$ and $\dfrac{-200}{1-p}$ at state 2

$$E[\text{payoff}]=\sum_{j=1}^\infty[1,0]M^j\left[2000,\frac{-200}{1-p}\right]^T=[1,0](I-M)^{-1}\left[2000,\frac{-200}{1-p}\right]^T$$

But $$M=\begin{pmatrix}p & p\\q & q\end{pmatrix}$$

which is not invertible?

  • 0
    Can you please explain why is $\sum_j M^j =(I-M)^{-1}$? – 2017-01-13
  • 0
    Geometric series? – 2017-01-13

0 Answers 0