- Consider a Markov Chain with
the following transition probability matrix:
1 2 3 4 +- -+ | 1/4 1/4 1/4 1/4 | 1 | 0 0 1 0 | 2 P = | 0 0 0 1 | 3 | 1 0 0 0 | 4 +- -+Questions:
- Draw the state transition diagram (the states are numbered as
1, 2, 3 and 4). (10 pts)
- Given that the system is current in
state 1, what is the
probability that
it will be state 3
in 4 transitions ?
(10 pts)
- What is the limiting (steady) state probablities π1, π2, π3 and π4. (10 pts)
- Draw the state transition diagram (the states are numbered as
1, 2, 3 and 4). (10 pts)
- A professor for a course left
3 books
- labeled 1, 2 and 3 -
in the reference section of the library.
Students are not allowed
to leave the library with these books.
However, Bob has discovered a way to beat the system:
- If he comes
just before the library closes,
then he can check out one of the books for the night and
return it the following morning.
- He is only allowed to check out one book at a time.
Bob's habit for checking out the books can be characterized as follows:
- Given that he checks out
book 1 today,
he is equally likely to check out
books 1 or 2
tomorrow.
- Given that he checks out
book 2 today,
tomorrow he will check out
book 1 with probability 0.5,
book 2 with probability 0.25
and
book 3 with probability 0.25.
- Given that he checks out book 3 today, tomorrow he will check out book 2 with probability 0.25 and book 3 with probability 0.75.
Questions:
- Draw the state transition diagram (the states are
to be numbered as
1, 2 and 3).
(5 pts)
- Give the one-step probability matrix
(5 pts)
- Given that Bob has checked out
book 1 today,
what is the probability that he will
check out book 3
in 4 days time ?
(10 pts)
- Find the limiting (steady) state probabilities π1, π2 and π3. (10 pts)
- If he comes
just before the library closes,
then he can check out one of the books for the night and
return it the following morning.
- Consider the following discrete-time Markov chain:
Questions:
- List all: (10 pts)
- transient states
- periodic states and their periods
- recurrent states
- chains and the states in each chain
- Given that the process is currently in
state 3,
what is the probability that it will
never enter
state 1
(10 pts)
- Give the one-step transition
probability matrix
(10 pts)
- Given that the process starts in
state 4.
We let the process run for a very long time
(say 1,000,000 steps) and then make an observation.
What is the probability that the process is in state 6 when we observe it ? (10 pts)
- List all: (10 pts)