Finding a Steady-State Probability Matrix

Date: 05/12/2000 at 08:15:24
From: Sean Hitch
Subject: Finding the steady state matrix

Dr. Math,

A security guard is employed to patrol a shopping complex. The guard 
is instructed to wait 10 minutes at each corner (numbered 1 to 4.) 
After 10 minutes, the guard must either stay where he is or move to 
one of the adjacent corners. Movements should be at random so that the 
chances of remaining or moving to each adjoining corner are the same.


     1_____2
     |    / |
     |   / /|
     |  / / |
     | / /  |
     |/ /___|
     3      4



     T = [1/3  1/3  1/3   0 ]
         |1/4  1/4  1/4  1/4|
         |1/4  1/4  1/4  1/4|
         [ 0   1/3  1/3  1/3]

Question:
Find the steady-state matrix for the likelihood of the guard's being 
at each of the corners.

Thank you once again for help in advance.

Sean Hitch

Date: 05/12/2000 at 12:51:17
From: Doctor Anthony
Subject: Re: Finding the steady state matrix

I ALWAYS work with the columns adding to 1 when using probability 
matrices.

So we require the column vector

     [a]
     [b]
     [c]
     [d]

to remain unchanged when the above matrix operates on it.

     [1/3   1/4   1/4    0]   [a]   [a]
     [1/3   1/4   1/4  1/3] * [b] = [b]
     [1/3   1/4   1/4  1/3]   [c]   [c]
     [0     1/4   1/4  1/3]   [d]   [d]

This will produce 4 equations:

     -(2/3)a + (1/4)b + (1/4)c +    0.d = 0
      (1/3)a - (3/4)b + (1/4)c + (1/3)d = 0       
      (1/3)a + (1/4)b - (3/4)c + (1/3)d = 0
         0.a + (1/4)b + (1/4)c - (2/3)d = 0

and solving for a, b, c, d in terms of one variable b we get:

    a = 3b/4
    b = b
    c = b
    d = 3b/4

Finally we can use

     a + b + c + d = 1

to get b = 2/7, and the stable vector is

     [3/14]
     [ 2/7]
     [ 2/7]
     [3/14] 

So the probability that guard is at corners 1 and 4 is 3/14 each and 
the probability that he is at corners 2 and 3 is 2/7 each.

The stationary matrix will have all 4 columns the same with the 
entries:

     [3/14]
     [ 2/7]
     [ 2/7]
     [3/14] 

- Doctor Anthony, The Math Forum
  http://mathforum.org/dr.math/