Associated Topics || Dr. Math Home || Search Dr. Math

### Eigenvectors of a Covariance Matrix

```Date: 07/04/2004 at 21:34:22
From: Sesha
Subject: Eigenvectors of a Covariance Matrix

I understand the concepts of eigenvalues and eigenvectors.  My
question is why do the eigenvectors of a covariance matrix point in
the direction of highest variance of the initial data set?

We create a covariance matrix using the data set.  Now, the covariance
matrix becomes a transformation matrix, meaning if we multiply the
eigenvector by the covariance matrix it would yield a multiple of the
eigenvector, and this multiplier is the eigenvalue.

```

```
Date: 07/06/2004 at 11:35:41
From: Doctor George
Subject: Re: Eigenvectors of a Covariance Matrix

Hi Sesha,

Thanks for writing to Doctor Math.

This is a great question.  The answer has to do with what is called a
Rayleigh quotient.  You can get a theoretical explanation of this from
most any college linear algebra book.

Finding the directions of maximum and minimum variance turns out to be
the same as looking for the orthogonal least squares best fit line and
plane of the data.  The sums of squares for that line and plane can be
written in terms of the covariance matrix.  If you can work out the
connections between them it should help you get an intuitive feel for

Here are a couple articles on those subjects from our archives that

Best-fitting Line to a Number of Points
http://mathforum.org/library/drmath/view/51809.html

Orthogonal Distance Regression Planes
http://mathforum.org/library/drmath/view/63765.html

Write again if you need more help.

- Doctor George, The Math Forum
http://mathforum.org/dr.math/
```
Associated Topics:
College Linear Algebra

Search the Dr. Math Library:

 Find items containing (put spaces between keywords):   Click only once for faster results: [ Choose "whole words" when searching for a word like age.] all keywords, in any order at least one, that exact phrase parts of words whole words

Submit your own question to Dr. Math
Math Forum Home || Math Library || Quick Reference || Math Forum Search