Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: SVD for PCA: The right most rotation matrix
Replies: 21   Last Post: Nov 6, 2012 2:10 AM

 Messages: [ Previous | Next ]
 Paul Posts: 517 Registered: 2/23/10
Re: SVD for PCA: The right most rotation matrix
Posted: Nov 1, 2012 3:53 PM

I might be missing some linear algebra theory here, but I looked up
gettrans() and I'm not sure what is meant by a column rotation in that
context.

On Nov 1, 1:21 pm, Gottfried Helms <he...@uni-kassel.de> wrote:
> Hi Paul,
>
> if I understood you correctly, you setup the SVD on your X-data such
> that (let Z denote Sigma and W and Vt the unique rotations which
> cause that Z becomes diagonal)
>
> Z = W * X * Vt
>
> The X-data contain n samples along the columns taken with m sensors
> defining the rows (I assume, they are centered, and for the example
> below that they are also standardized)
>
> Then I understood your question that you ask, what relevance has Vt
> and especially in terms of a multidimensional euclidean model.
>
> If I understand you correctly so far, then the following might be
>
> The key is, that the n samples define m vectors in an n-dimensional
> euclidean space; simply each column of X can be seen as a spatial
> dimension. In that n-dimensional space there are m vectors, where
> the number m is smaller than n. Any rotation in that space
> repositions the vectors, but *not* the relation, or better: the
> angles, between them

I'm not sure why *any* rotation in n-space would not preserve angles.
I thought that a rotation is by definition a unitary transformation
(from a recent brush-up on linear algebra at Wikipedia e.g.
http://en.wikipedia.org/wiki/Orthogonal_matrix).

> ...So we can rotate the vector model X
> (columnwise) first such, that
> sensor 1 defines the x-axes,
> sensor 2 and 1 define the x-y-plane
> sensor 3 to 1 define the x-y-z-space
> and so on.

I don't quite follow what you mean by "rotat[ing] the vector [model] X
columnwise". If you interpret each column of X as a point (or vector)
in n-space, we get what you describe (sensor 1 is the x-axis, sensor 2
is the y-axis, etc.). However, a rotation is not needed for this.

> In effect, that rotation provides a matrix X1 which is triangular
> with as many nonzero-columns as the rank of the matrix is (and we
> assume for simplicityness, that it equals m)

I think I'm missing something fundamental...the data matrix is not
triangular, though the (n)x(n) covariance matrix (Xt)(X) is symmetric.

> Then the matrix X1 can be rotated to the position of their principal
> components (we're talking already of the nonzero columns only),
> let's call this X2

I see that the data must be rotated so that the principal axes align
with the axes of m-space (not n-space), and then the diagonal matrix
Sigma performs the anisotropic axial stretching.

> That two rotations together form your matrix Vt. After that, X2 can
> be rotated by rotation of its rows to diagonal form - this is your
> rotation-matrix W, which rotates for the principal components with
> respect of the rows in X2 (and which is the same as the rotation
> with respect of the rows in X).

But W is not applied after Vt, Sigma is (the anisotropic axial
scaling). After that, however, I see that W does rotate the axially
scaled body of data points so that maximally stretched axis becomes
aligned with the principal component of the measured data, the 2nd
most stretched axis aligns with the 2nd principal component, etc.

So the rotation by W is very intuitive to me, while the rotation by Vt
is not. And as I described, it's all the more mysterious when you
consider that X isn't actually a transformation that is applied to
data -- it *is* the data. For this reason, I find it difficult to see
the decomposition of X as a series of transformations (rotate,
stretch, rotate) despite the intuitive appeal of the (W)(Sigma) (there
is no intuition on my part concerning Vt).

This inability to picture (X)(Sigma)(Vt) as a transformation shows up
particularly in my lack of intuition concerning Vt...it is the first
of the 3 decomposed transformations that gets applied to any data
point/vector that is subjected to the 3-step stransformation. The
question is "What is this data that gets subjected to this
transformation? And in n-space, no less". It seems that the 3-step
transformation and the data are the same!

Furthermore, when I am seeking correlation between the m sensors, it
confounds me to think about why one would picture the data points in n-
space. As an analogy, if I am doing simple linear regression on a
cloud of 1000 points in the x-y plane, I don't try to picture the data
points in 1000-dimension space.

> I've done this step-by-step with my matrix-calculator MatMate and
> show the matrices where we have only (m=)3 sensors and (n=)6
> samples.
>
> We generate a random dataset for 3 sensors, and 6 samples, centered
> and standardized normal distributed data in matrix X
> [22] set randomstart=41
> [23] X =zvaluezl(abwzl( randomn(3,6)))
> X :
> 0.2096 0.2276 -2.0787 -0.1263 0.9337 0.8340
> 0.0668 0.0848 0.8837 -1.6367 -0.7827 1.3842
> 0.4677 -0.4803 0.8500 0.2395 0.9091 -1.9860
>
> Each row defines the coordinates of one vector in the n=6
> dimensional space. Now wet get the rotation-matrix t1, which
> rotates X to triangular form, preserving the angles (=cosines,
> correlations) between the vectors:
> [24] t1 = gettrans(X,"Drei")
> t1 :
> 0.0856 0.0449 0.3898 0.6802 -0.4701 -0.3937
> 0.0929 0.0538 -0.1865 -0.1958 0.3348 -0.8963
> -0.8486 0.1986 0.1513 0.2615 0.3856 -0.0206
> -0.0516 -0.6916 -0.5339 0.4630 0.1392 0.0151
> 0.3812 -0.2498 0.5843 0.1452 0.6459 0.1125
> 0.3405 0.6441 -0.4049 0.4418 0.2858 0.1685

Sorry, I tried to google gettrans, but wasn't able to find much beyond
the fact that it is a column rotation. It's not clear to me what is
meant by that. Consequently, I wasn't able to follow the rest of the
example.

However, I appreciate the time that you took to compose the attempted
explanation.

Date Subject Author
10/28/12 Paul
10/29/12 Ray Koopman
10/29/12 Paul
10/29/12 Ray Koopman
10/29/12 Paul
10/29/12 Art Kendall
10/29/12 Art Kendall
10/29/12 Paul
10/29/12 Art Kendall
10/29/12 Paul
10/29/12 Art Kendall
10/29/12 Paul
10/30/12 Art Kendall
11/1/12 Paul
10/29/12 Richard Ulrich
10/29/12 Paul
11/1/12 Gottfried Helms
11/1/12 Paul
11/2/12 Gottfried Helms
11/4/12 Paul
11/4/12 Gottfried Helms
11/6/12 Paul