3
$\begingroup$

There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined.

$T_0 = \lbrace 0,1,\dots ,k-1\rbrace$
$T_1 = \lbrace 1,2,\dots ,k-1\rbrace$

$n_i(t) = 1$ means that process was in state $i$ at time $t$, ($t \in T_0$),
$n_i(t) = 0$ otherwise,
$v_{ij}(t) = 1$ means that process changed from state $i$ to state $j$ between time $t-1$ and $t$, ($t \in T_1$), $v_{ij}(t) = 0$ otherwise.

I estimate probability of transition from state $i$ to state $j$ as

$\hat{p}_{ij} = \frac{\displaystyle\sum_{t \in T_1}v_{ij}(t)}{\displaystyle\sum\limits_{t \in T_1}n_i(t-1)}$

First question:
How many observations do I need to properly estimate transition matrix? If $p_{ij}$ is an ideal estimation, I don't want relative error between $\hat{p}_{ij}$ and $p_{ij}$ ( $\frac{|\hat{p}_{ij}-p_{ij}|}{p_{ij}}$ ) to be more than $\varepsilon = 0.01$ with probability $P \ge 0.99$.
I need this in form $k=k(r,\varepsilon, P)$.
I know that the number will be large but this does not matter. I just want to know the number.
For my purposes $r$ will be between 7 and 15, $\forall_i p_{ii} \neq 1$.
If there would be a state $i$ that was never entered or was entered at the last step (what could be written as $\sum\limits_{t \in T_1}n_i(t-1)=0$), I will just simply exclude that state from further calculations.

Second question:
I have first transition matrix $A$. That matrix describes "model" process and was estimated with sufficient number of observations. Then, second process was observed and new matrix $B$ was estimated (with sufficient number of observations).
How can I decide if second process matches "model" process? I want this method to have an "error factor" (I don't know how to name it) witch changeable value $\varphi$. I need 3 or 4 of those methods for comparison.

  • 1
    Jan, to bump your question back to the front page, so that it might come to more people's attention, you can make a trivial edit to the text (e.g. add some spaces, change one word, etc.). I have done this for you. If bumping it a few times doesn't help, another approach to attract attention and answers to your question would be to [place a bounty on it](http://math.stackexchange.com/faq#bounty), but unfortunately to do so requires 75 points.2011-11-29

2 Answers 2