The Math Forum

Ask Dr. Math - Questions and Answers from our Archives
Associated Topics || Dr. Math Home || Search Dr. Math

Eigenvectors of a Covariance Matrix

Date: 07/04/2004 at 21:34:22
From: Sesha
Subject: Eigenvectors of a Covariance Matrix

I understand the concepts of eigenvalues and eigenvectors.  My
question is why do the eigenvectors of a covariance matrix point in 
the direction of highest variance of the initial data set?

We create a covariance matrix using the data set.  Now, the covariance 
matrix becomes a transformation matrix, meaning if we multiply the 
eigenvector by the covariance matrix it would yield a multiple of the
eigenvector, and this multiplier is the eigenvalue.

Date: 07/06/2004 at 11:35:41
From: Doctor George
Subject: Re: Eigenvectors of a Covariance Matrix

Hi Sesha,

Thanks for writing to Doctor Math.

This is a great question.  The answer has to do with what is called a 
Rayleigh quotient.  You can get a theoretical explanation of this from
most any college linear algebra book.

Finding the directions of maximum and minimum variance turns out to be
the same as looking for the orthogonal least squares best fit line and 
plane of the data.  The sums of squares for that line and plane can be 
written in terms of the covariance matrix.  If you can work out the 
connections between them it should help you get an intuitive feel for 
your question.

Here are a couple articles on those subjects from our archives that 
may help you.

  Best-fitting Line to a Number of Points 

  Orthogonal Distance Regression Planes 

Write again if you need more help.

- Doctor George, The Math Forum 
Associated Topics:
College Linear Algebra

Search the Dr. Math Library:

Find items containing (put spaces between keywords):
Click only once for faster results:

[ Choose "whole words" when searching for a word like age.]

all keywords, in any order at least one, that exact phrase
parts of words whole words

Submit your own question to Dr. Math

[Privacy Policy] [Terms of Use]

Math Forum Home || Math Library || Quick Reference || Math Forum Search

Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.