1
$\begingroup$

I was wondering what literature there is on the following problem:

Given an $n\times n$ probability transition matrix $P$, where entry $P_{i,j}$ is the probability of going from vertex $i$ to vertex $j$, does there exist a weighted, undirected graph $G$ on vertices $\{1, ..., n\}$ such that the probability transition matrix of a random walk on $G$ is $P$? (Maybe add in the assumptions that $P$ is connected, and $P_{i, i}=0$ for all $i$).

This is not true for all $P$. Consider $P=\begin{pmatrix} 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \\ \frac{1}{5} & \frac{4}{5} & 0 \end{pmatrix}$.

If $w(\{i, j\})$ is the weight of the edge between vertices $i$ and $j$, then the first row says that $w(\{1, 2\}) = 2\cdot w(\{1, 3\})$, and the second row says $w(\{1, 2\}) = w(\{2, 3\})$. But together, these imply $w(\{2, 3\}) = 2\cdot w(\{1, 3\})$, which is not what the third row of $P$ says.

So I'm wondering, is there an if and only if condition on $P$ which says when this is possible?

  • 0
    Symmetry is sufficient, I think (at least, it removes the issue with your example). Is it necessary? – 2017-02-09
  • 0
    Yeah, I was thinking about symmetry. It's not strictly necessary, I think; for example, consider $P=\begin{pmatrix} 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 \\ 1 & 0 & 0 & 0 \end{pmatrix}$. – 2017-02-10
  • 0
    Hmm, and that rules out reversibility too, which was my previous idea. I think it might be reversibility if you additionally assume that $p_{ij}>0$ whenever $i \neq j$. – 2017-02-10
  • 0
    For example, if $P=\begin{bmatrix} 0 & 1/4 & 3/4 \\ 1/3 & 0 & 2/3 \\ 3/5 & 2/5 & 0 \end{bmatrix}$ then $P$ may be represented by $K_3$ with $w(\{ 1,2 \})=1,w(\{2,3 \})=2,w(\{ 3,1 \})=3$. This $P$ is not symmetric but it is reversible (I checked it numerically). – 2017-02-10
  • 0
    What do you mean by reversible? – 2017-02-10
  • 0
    A Markov chain is reversible if there exists a nonzero function $\pi$ such that $\pi(x) P(x,y) = \pi(y) P(y,x)$ for all $x,y$. Such a $\pi$ is necessarily an invariant measure and on a finite state space can be normalized into an invariant distribution. Thus you can find it by seeking a left eigenvector of $P$ with eigenvalue $1$. – 2017-02-10
  • 1
    In fact I think reversibility is exactly what you want. An intuitive way to view it is to think about CTMCs. If you have weights $w(\{ i,j \})$, the chain that you want to look at is the jump chain of a CTMC with generator $G$ where $g_{ij}=w(\{ i,j \})$. (I'm forbidding self-loops.) – 2017-02-10
  • 0
    Thus the generator is an arbitrary symmetric generator, but the transition probability matrix may not be symmetric because the overall rate of transitions from each state may not be the same. But that means that the transition probability matrix is "symmetrizable" by multiplying each row by the total rate of transitions from the corresponding state. I think the equations that follow from this symmetrizability requirement are just an unnormalized version of the reversibility equations. – 2017-02-10
  • 0
    Sorry, what is the jump chain of a CTMC? And what is its generator? – 2017-02-12
  • 0
    But okay, I think you are right. Reversibility is exactly what is needed. Thanks so much! – 2017-02-12
  • 0
    The jump chain of a CTMC is the DTMC describing where the transitions will be. The generator of a CTMC describes the rate of the transitions (with the diagonal elements chosen so that the row sum is zero, for technical reasons). The transition probability matrix $P$ and the generator $G$ are related through $p_{ij}=\frac{g_{ij}}{-g_{ii}}$ for $i \neq j$. – 2017-02-12

0 Answers 0