Positive Definite Matrices

Date: 08/12/2008 at 23:27:25
From: Radhakrishnan
Subject: How to determine if a matrix is positive definite? 

i have a correlation matrix and i want to check if the matrix is a
positive definite matrix, if not i want to make the matrix to be
positive definite...... 

in some places I found that a given matrix is positive definite if the
Eigen values of the matrix is positive.....i really don't understand
what this eigen value is all about or where its applications are?

next why should a monte carlo algorithm always wants positive 
definite covariance matrix?

anything on how to approach the problem will be helpfull if you can
explain in steps using a small example

Date: 08/15/2008 at 08:21:51
From: Doctor George
Subject: Re: How to determine if a matrix is positive definite?

Hi Radhakrishnan,

Thanks for writing to Doctor Math.

There is a lot to your question, like a whole chapter from a linear
algebra book.  If you can find the book by Gilbert Strang it would help.

To determine whether a matrix is positive definite, a standard way is
to attempt a Cholesky decomposition.  A Google search on "Cholesky
decomposition" will give you plenty of information.  For your needs
you may not need the Eigenvalues.

The Cholesky decomposition is useful in monte carlo because it allows
you to transform a set of independent normally distributed random
variables into a correlated set.  It would take a longer discussion to
explain what this has to do with positive definiteness.  That is where
a good text will help.

I hope this is enough to get you started. Write again if you need more
help.

- Doctor George, The Math Forum
  http://mathforum.org/dr.math/ 

Date: 08/16/2008 at 04:20:05
From: Radhakrishnan
Subject: Thank you (How to determine if a matrix is postive definite? )

Hello George,

thank you very much for your answers.  i will start studying but at
least can you help me with the steps to make a matrix to be positive
definite?  again, if possible with a small example please.....,.

Date: 08/18/2008 at 08:28:46
From: Doctor George
Subject: Re: How to determine if a matrix is postive definite?

Hi Radhakrishnan,

Correlation and covariance are all about linear relationships between
random variables.  Let's set up a Monte Carlo simulation of two
variables that are normally distributed.  The strategy is to start
with two independent variables, and then apply a linear transformation
to create the desired dependency.

Let's call the vector of independent variables X, and the vector of
dependent variables Y.  Also let L be the matrix that applies the
linear transformation.  So we have

  Y = LX

If we compute the covariance matrix of Y it will be

  Cov(Y) = LL^T   (L times L transpose)

If the rows of L are linearly independent then Cov(Y) will be positive
definite.

So if we start with Cov(Y), the Cholesky decomposition will find L for
us.  If the decomposition fails then the matrix was not positive
definite, which means that there is no L that will convert X into the
desired Y.  (I'm not giving a complete proof here, just the strategy.
A good text will contain the proof.)

Now for the example.  Let

           |  1   0.6 |
  Cov(Y) = |          |
           | 0.6   1  |

The Cholesky decomposition leads to

      |  1     0  |
  L = |           |
      | 0.6   0.8 |

To perform the simulation, generate values for X1 and X2, and then
compute Y1 and Y2.

  | Y1 |     | X1 |
  |    | = L |    |
  | Y2 |     | X2 |

Does that make sense?

- Doctor George, The Math Forum
  http://mathforum.org/dr.math/ 

Date: 08/19/2008 at 02:53:07
From: Radhakrishnan
Subject: Thank you (How to determine if a matrix is positive definite? )

George,

Thank You very much..... it is really helpful. Once again i thank you
very much for taking time out of your schedule to help me. As per your
advice let me start with a good text.

Regards