Eigenvalues and Hermitian Matrices

Date: 01/03/2006 at 03:32:30
From: Sunder
Subject: Linear Algebra: Proof

Let A be a symmetric matrix (such a matrix appears when considering
multivariate distributions).  Show that the eigenvalues of A are real.

Surfing the net, I found out that it's about Hermitian matrices.  Is 
there a simpler approach?  I don't understand it well enough to 
prove it.  Your guidance is much appreciated.

Date: 01/03/2006 at 04:51:25
From: Doctor Luis
Subject: Re: Linear Algebra: Proof

Hi Sunder,

Let's take a real symmetric matrix A.  The eigenvalue equation is:

  Ax = ax

where the eigenvalue a is a root of the characteristic polynomial

  p(a) = det(A - aI)

and x is just the corresponding eigenvector of a.  The important part
is that x is not 0 (the zero vector).

Well, anyway.  Let's calculate the following inner product 
(here, x_i* is the complex conjugate of x_i):

  <x,Ax> = sum_i  x_i* (Ax)_i
         = sum_i  x_i* (sum_j A_ij x_j)
         = sum_i sum_j  x_i* A_ij x_j

That's the inner product expanded out, which we'll use later. 
But for now, note that since x is an eigenvector, we know that 
Ax = ax.  We can use this fact to conclude:

  <x,Ax> = <x,ax>
         = sum_i x_i* (ax)_i
         = sum_i x_i* a x_i
         = a  sum_i x_i* x_i
         = a (sum_i |x_i|^2)

Note that sum_i |x_i|^2 is always positive since x is nonzero.  We'll
use this fact later, too.  Next, we should find the following inner
product (again, y* means complex conjugate of y):

  <Ax,x> = sum_i (Ax)_i* x_i
         = sum_i (sum_j A_ij x_j)* x_i
         = sum_i (sum_j A_ij* x_j*) x_i
         = sum_i sum_j  x_i A_ij* x_j*

But now, we can use the fact that A^t = A and that A is real.  In
particular, that A_ij* = A_ij, and A_ji = A_ij.

  <Ax,x> = sum_i sum_j x_i A_ij x_j*
         = sum_i sum_j x_i A_ji x_j*
         = sum_j sum_i x_j* A_ji x_i
         = sum_I sum_J x_I* A_IJ x_J   (renaming j->I, i->J)
         = sum_i sum_j x_i* A_ij x_j   (dummy variables J->j, I->i)
         = <x,Ax>

So, because A is real and symmetric, we have A = A^t and 
<Ax,x> = <x,Ax>.

Now, take the eigenvalue equation again:

  Ax = ax

Now, take the transpose and then complex conjugate:

  (Ax)^t = (ax)^t

  x^t A^t = a x^t

  x^t A = a x^t         (since A^t = A)

  (x^t A)* = (a x^t)*

  (x*)^t A* = a* (x*)^t

  (x*)^t A = a* (x*)^t    (since A* = A)

Now, just multiply both sides by x, (on the right),

  (x*)^t A x = a* (x*)^t x

  sum_i (x*)_i (Ax)_i = a* sum_i (x*)_i x_i

  sum_i x_i* (sum_j A_ij x_j) = a* sum_i x_i* x_i

  sum_i sum_j x_i* A_ij x_j = a* (sum_i |x_i|^2)

or

  <Ax,x> = a* (sum_i |x_i|^2)

But, we already found that <x,Ax> = a (sum_i |x_i|^2),
and that <Ax,x> = <x,Ax>. Therefore,

  0 = <Ax,x> - <x,Ax>

    = a* (sum_i |x_i|^2)  -  a (sum_i |x_i|^2)

  0 = (a* - a) (sum_i |x_i|^2)

Since sum_i |x_i| > 0, we can divide this last equation by it,
which gives us

  0 = a* - a

or

  a = a*

Since a is any eigenvalue of A, we have proven that the complex
conjugate of a is a itself.  This can only happen if a is real,
which concludes the proof.


Note that we spent most of the time doing inner product math in the
long-winded explanation given above.  All we really wanted to say was
that <x,Ax> = <A'x,x>, where A' is the adjoint matrix to A (adjoint
for matrices means transpose and complex conjugation).

A matrix which is its own adjoint, i.e. A = A', is called self-adjoint
or Hermitian.  That's all it means.  Clearly, a real Hermitian matrix
is just a symmetric matrix.  Do you see why?


Now, the short proof.

Consider the inner product

   <u,v> = sum_i u_i* v_i

and let A be a Hermitian matrix.  Let x be an eigenvector of A
with eigenvalue a.  Then,

  <x,Ax> = <x,ax> = a <x,x>

and

  <Ax,x> = <ax,x> = a* <x,x>       (can you explain this last step?)

Lastly, note that

  <x,Ax> = <A'x,x>     (adjoint matrix)
         = <Ax,x>      (since A is self-adjoint)

Thus,

  0 = <Ax,x> - <x,Ax>
    = a* <x,x> - a <x,x>
  0 = (a* - a) <x,x>
  0 = a* - a           (we can divide by <x,x> since it's nonzero)
  a = a*

Therefore, any eigenvalue a of a Hermitian matrix A is real.

See how much simpler the Hermitian proof is?  Well, let us know if you
have any more questions.

- Doctor Luis, The Math Forum
  http://mathforum.org/dr.math/ 

Date: 01/04/2006 at 01:00:42
From: Sunder
Subject: Thank you (Linear Algebra: Proof)

Dear Dr. Luis,

Thanks for your prompt reply.  You asked good questions to prod my
understanding.