3
$\begingroup$

So I understand what it means for a stochastic process to have the Markov property, but I am not as sure about how to actually deduce this from information given.

For example,

Suppose a washing machine can serve people one at a time, moreover, if it is washing a persons clothes at time n, it has probability $p$ of finishing the cycle before time $n+1$.

Between the times $n$ and $n+1$, $P_{n}$ customers arrive, where $P_{n}$ is $Poisson(\lambda)$ independent of past arrivals.

Show that the total number of customers waiting service and currently being served at time n is a Markov chain.

So how could I show that

$$P[X_{n+1}=j|X_{0}=i_{0},X_{1}=i_{1},....X_{n}=i]$$

=

$$P[X_{n+1}=j|X_{n}=i]$$

Would the transition probabilities best be found first, or could they be deduced once we have formulated it?

One thing I am a bit confused about is,

Between times $n$ and $n+1$, can the washing machine only serve one customer?

Ie, it says it serves one customer at a time, is this implying at most 1 person could leave between these times?

Assuming that this is the case,

At time $n$ we have $X_{n}$ total customers, between times $n$ and $n+1$ , $P_{n}$ customers arrive, and either 0 customers leave or one leaves, with probability p.

Ie, $X_{n+1}=X_{n}+P_{n}-1$ with probability $p$ and $X_{n+1}=X_{n}+P_{n}$ with probability $(1-p)$

However, as suggested in the comments, I am still not exactly sure how that implies we can write

$X_{n+1}=X_{n}+P_{n}-B_{n}$ where $B_{n}$ is Bernoulli, because what does the $n$ have to do with it?

I tried breaking it into cases,

in any time period at most one person leaves, and $P_{n}$ arrive,

so either no people leave or one person leaves.

So

$X_{n+1}=X_{n}+P_{n}-1$ with probability p and

$X_{n+1}=X_{n}+P_{n}$ with probability $1-p$

Now, can I use that both the Bernoulli trials and Poisson random variable are respectively independent of past to make the conclusion?

I would also like to understand how it all relates to the transition probabilities for every (i,j) ie how that can be found from the information I am still pretty confused about this question and am looking for any solid explanations Thanks

  • 0
    Notice that $X_{n+1} = X_{n} + P_n - B_n$, where $B_n$ has Bernoulli distribution with probability $p$ and $B_n$ is independent from the past. – 2017-02-19
  • 0
    @echzhen So the total number of people at time n+1 would be the number of people at time n + the random $P_{n}$ , but what would the $-{Bn}$ represent? Because only one person could leave between time $n $and $n+1$, no? – 2017-02-19
  • 0
    if I understood the problem correctly: between $n+1$ and $n$ only one person can leave independently from the past. So, yes $B_n$ represents the person who is currently being served. – 2017-02-19
  • 0
    **Slight correction:** $X_{n+1}=(X_n+P_n-B_n)^+$, the positive part being needed when $X_n=P_n=0,B_n=1$. – 2017-02-19
  • 0
    @JohnDawkins Is that the case that there are zero people waiting, zero people arrive and the person being served is finished and leaves? – 2017-02-19
  • 0
    It's the case in which at time $n$ there is no one waiting or being served, and no one arrives between $n$ and $n+1$; if by chance $B_n$ were $1$ then the formula would yield $X_{n+1}=-1$; the 'positive part' fixes this defect. – 2017-02-19
  • 0
    @JohnDawkins But would that imply there is 1 person when it would really be zero? – 2017-02-19
  • 0
    @echzhen's formula would give $X_{n+1}=-1$ in that case, but actually $X_{n+1}=0$. – 2017-02-19
  • 0
    @JohnDawkins But how could a person finished being served if there were 0 people at time n? Ie, how is it even possible that $X_{n}=P_{n}=0$ and $B_{n}=1$ – 2017-02-19
  • 0
    @JohnDawkins thanks for the correction, the positive part is indeed needed – 2017-02-19
  • 0
    @JohnDawkins I still dont completely understand, partly about my above comment, and how it can give us the markov property directly – 2017-02-21
  • 0
    *'it says it serves one customer at a time, is this implying at most 1 person could leave between these times?'* -- It seems to me that this is one possible interpretation. Another possible interpretation is that the number of people that be served at any given time $n$ is a geometric random variable, which means that more than one person could be served at a given time, if more than one person is queueing. I don't think either interpretation matters for the Markov property, however. – 2017-02-23

0 Answers 0