Description

A discrete time Markov chain with state space S = {1, 2, 3, 4, 5, 6, 7} has the following transition matrix. P = ([0 2/3 0 1/3 0 0 0] [9/10 0 0 1/10 0 0 0] [0 0 0 0 2/5 0 3/5] [0 1 0 0 0 0 0] [0 0 2/3 0 0 1/3 0] [0 0 0 0 1/4 0 3/4] [0 0 0 0 0 1 0]). (a) Write down the communication classes of the chain. (b) Find the period of each communicating class. (c) Determine which classes are essential. (d) Classify each essential communicating class as transient or positive recurrent or null recurrent. (e) Describe the long run behaviour of the chain (including deriving long run probabilities where appropriate). (f) Given X0 = 6, find the long run proportion of time the chain spends in state j for each j in S. (g) Find the expected number of steps to return to state 3, given the chain starts from state 3.

Get Homework Help Now