ACTL2003 PASS Sample Mid Semester Paper

1. Let {Xn , n = 0, 1, 2, ...} be a time-homogeneous Markov chain with state space S = {1, 2, 3, 4, 5}
and one-step transition probability matrix
1

0 0 21 0
2
0 1 0 0 0

1
1
P=
0 0 0 2 2 .
1 0 0 0 0
0 0 1 0 0
(a) Specify the classes of the states and determine whether they are transient or recurrent.
(b) Suppose that the chain is equally likely to be in each state at time 0. For each transient
state, calculate the expected number of time periods that the Markov chain will be in
the state.
(c) Calculate the periods of all the states.
2. For a Markov chain {Xn, n = 0, 1, 2, ...} with states 0,1 and 2, the one-step transition matrix
is given as:
1 3

0
4
4

P = 0 13 23
1
0 12
2
(a) Classify the states of the Markov chain and explain why a limiting distribution of the
random state exists.
(b) Calculate the limiting probability of the state 0.
(c) Calculate the expected number of transitions to return to state 0 if you begin in state
0.
3. The matrix P of one-step transition probabilities is given for a Markov chain {Xn , n =
0, 1, 2, ...} with states 0,1 and 2 with:
3 1

0
4
4

P = 12 12 0
1
4

1
4

1
2

Furthermore,
43
P3 =

P4 =

64
42
64
35
64
171
256
170
256
155
256

21
0
64

22
0
64
21
8
64
64

85
0
256

86
0
256
85
16
256
256

(a) If at time 5 the process is in state 2, find the probability that at time 12 it will be in
state 2.
(b) If P r(X5 = 0) = P r(X5 = 1) = P r(X5 = 2) =
the state at time 8.

1
3

,find the unconditional distribution of

4. (a) Claims of an insurance company arrives according to a Poisson process at the rate 2 per
day.
i. What is the probability that there are two or more claims for a 5-day span?
1

ii. Suppose that there are 4 Mondays in July. What is the probability that the total
number of claims arriving on Mondays in July is 5?
(b) A living room has two lamps A and B. Each lamp uses a certain type of lightbulb.
The amount of time that a lightbulb of type A works before burning itself out is an
exponential random variable with mean 2000 hours. The amount of time that a lightbulb
of type B works before burning itself out is an exponential random variable with mean
1000 hours. Suppose we keep both lamps on all the time and whenever a lightbulb
burns, we replace it with a new one of the same type immediately.
i. What will be the expected number of lightbulbs of Type A that will be replaced in
a month (30 days)?
ii. What is the probability that in a month there that will be exactly 2 lightbulbs will
be replaced?
5. Suppose customers arrive at a post office according to a Poisson process at the rate 5 per
hour.
(a) If each customer has probability 0.6 of being male, what is the probability that there is
no female customers arriving between 1 : 00 p.m. and 2 : 00 p.m. on a day?
(b) Suppose that the amount of money each customer spends on purchasing stamps is
exponentially distributed with mean $2 and is independent of all the other customers.
What will be the expectation and variance of the total amount of revenue of the post
office from selling stamps in a day, assuming that the post office opens 8 hours a day?
Show all working.
6. A 2-state Markov jump process has the following transition rates
q01 = 3
q10 = 2
(a) Determine the transition rate matrix Q.
(b) Set up the Kolmogorov forward equations for this jump process. (Do not solve the
system of equations!)