Inner Product and L-2 Distance

Date: 11/14/2001 at 00:21:22
From: Queenie Lee
Subject: Distance between two functions

Dear Dr. Math, 

I have a question on the theory behind the "distance between two 
functions." Why is "distance between two functions" calculated by 
multiplying f(x) and g(x) and then integrating with respect to x 
within the defined domain? What made mathematicians define it this 
way, and how did they arrive at this definition? 

Thanks a million and best wishes.

Yours sincerely,
Queenie Lee

Date: 11/14/2001 at 09:16:52
From: Doctor Fenton
Subject: Re: Distance between two functions

Hi Queenie,

Thanks for writing to Dr. Math. The distance between two functions
isn't usually calculated as the integral of the product, but rather 
the integral of some power of the absolute value of the difference
between the two functions: for f and g defined on an interval [a,b]
(just to be specific; the same approach is used for any measure space)
for example, the L-p distance between them is

                  b
               [ /                  ]^(1/p)
   ||f-g||_p = [ | |f(x)-g(x)|^p dx ]
               [ /                  ]                  .
                 a

p=2 is probably the most common distance measure.

The quantity you described,
 
      b
     /
     | f(x)g(x) dx
     /
     a

is called the "inner product" of f and g, and it is much like the dot
product between vectors.  If v and w are two 3-dimensional vectors,
v = <v1,v2,v3> and w = <w1,w2,w3>, then

   v.w = v1*w1 + v2*w2 + v3*w3

is the dot (or inner) product of vectors, and it is related to the 
angle between the vectors. In particular, two vectors are at right
angles if their dot product is 0.

You can think of a 3-dimensional vector as a function defined on the
index set {1,2,3}, so the vector <v1,v2,v3> is a function

   v:{1,2,3} --> R , with v(1) = v1,v(2) = v2, and v(3) = v3.

Then a function on an interval is much like a vector with an infinite
(and uncountable) index set, and the integral

          b
         /
 <f,g> = | f(x)g(x) dx
         /
         a

has properties very similar to the dot product of vectors, and in 
particular, we say that two functions are orthogonal if this integral
is 0.  It is more of a measure of the "angle" between the two 
functions than the distance, although the p = 2 measure is related:

   [ ||f-g||_2 ]^2 = <f-g,f-g> ,

just like the square of the length of a vector, ||v||^2, is the 
dor product of the vector with itself.

   ||v||^2 = v1^2 + v2^2 + v3^2

           = v.v   .

It is this analogy which led mathematicians to define the inner 
product and L-2 distance between functions the way they did.

Does this answer your questions?  If not, please write back and I can
try to explain further.

- Doctor Fenton, The Math Forum
  http://mathforum.org/dr.math/