The Stationary Vector

Date: 11/13/2000 at 00:59:36
From: Jessica Searing
Subject: Markov Processes

What does a stationary vector tell you and how do you find it?

Date: 11/13/2000 at 11:20:00
From: Doctor Anthony
Subject: Re: Markov Processes

The stationary vector is a position from which no further change 
occurs. It will correspond to the eigenvector with eigenvalue equal to 
1. Below is an example where weather conditions from one day to the 
following day are considered.

The transition matrix is  

                        FROM 
                      --------
                 SUN    CLOUD   RAIN
           SUN [  0      1/4     1/4 ]
     TO  CLOUD [ 1/2     1/2     1/4 ]  
     --   RAIN [ 1/2     1/4     1/2 ] 

If we take higher and higher powers of the matrix, it will eventually 
settle down so that whatever the starting conditions are on day 1, the 
effect will not influence the nth day. In this situation the columns 
of the matrix are all the same, so that any starting vector gives the 
same resulting vector. This vector is in fact the eigenvector 
corresponding to an eigenvalue of 1. (Note that EVERY transition 
matrix where the columns each sum to 1 will ALWAYS have an eigenvalue 
of 1.)

To find the eigenvector [a,b,c] corresponding to the eigenvalue 1 we 
have to solve:

     (0-1)a +     (1/4)b + (1/4)c = 0
     (1/2)a + (1/2 - 1)b + (1/4)c = 0
     
     -a +  b/4 + c/4 = 0
     a/2 - b/2 + c/4 = 0

          a           -b           c
     ---------- = ---------- = ---------- 
     |1/4  1/4|   |-1   1/4|   |-1   1/4|
     |-1/2 1/4|   |1/2  1/4|   |1/2 -1/2|

      a      -b     c
     ---- = ---- = ----
     3/16   -3/8   3/8

     a/3 = b/6 = c/6

     a/1 = b/2 = c/2

     [a]   [1/5]
     [b] = [2/5]
     [c]   [2/5]

Therefore 1/5 of the days are sunny and each of the other two types 
occurs on 2/5 of the days.

We can check that [1/5, 2/5, 2/5] is a stable vector by multiplying 
this vector by the original matrix

       SUN [  0      1/4     1/4 ][1/5]   [1/5]
     CLOUD [ 1/2     1/2     1/4 ][2/5] = [2/5]
      RAIN [ 1/2     1/4     1/2 ][2/5]   [2/5]

and see that the following day is unchanged.

- Doctor Anthony, The Math Forum
  http://mathforum.org/dr.math/