Eigenvectors of a Covariance Matrix
Date: 07/04/2004 at 21:34:22 From: Sesha Subject: Eigenvectors of a Covariance Matrix I understand the concepts of eigenvalues and eigenvectors. My question is why do the eigenvectors of a covariance matrix point in the direction of highest variance of the initial data set? We create a covariance matrix using the data set. Now, the covariance matrix becomes a transformation matrix, meaning if we multiply the eigenvector by the covariance matrix it would yield a multiple of the eigenvector, and this multiplier is the eigenvalue.
Date: 07/06/2004 at 11:35:41 From: Doctor George Subject: Re: Eigenvectors of a Covariance Matrix Hi Sesha, Thanks for writing to Doctor Math. This is a great question. The answer has to do with what is called a Rayleigh quotient. You can get a theoretical explanation of this from most any college linear algebra book. Finding the directions of maximum and minimum variance turns out to be the same as looking for the orthogonal least squares best fit line and plane of the data. The sums of squares for that line and plane can be written in terms of the covariance matrix. If you can work out the connections between them it should help you get an intuitive feel for your question. Here are a couple articles on those subjects from our archives that may help you. Best-fitting Line to a Number of Points http://mathforum.org/library/drmath/view/51809.html Orthogonal Distance Regression Planes http://mathforum.org/library/drmath/view/63765.html Write again if you need more help. - Doctor George, The Math Forum http://mathforum.org/dr.math/
Search the Dr. Math Library:
Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.