Moment Generating Functions

Date: 06/20/98 at 12:03:10
From: Kenneth W. Hodgkinson
Subject: Moment Generating Functions

I would appreciate any help you can give me on explaining Moment 
Generating functions, specifically these questions:  

1. Prove that M(x+y)(t) = Mx(t) * My(t). What assumptions are 
   needed? Here, Mx means the moment generating function of x, My 
   means the moment generating function of y and M(x+y) means the 
   moment generating function of x+y.

2. Find the moment generating function of the exponential distribution 
   with mean a. What is the restriction on t and why is it needed?

3. Prove Var(x + y) = Var(x) + Var(y). What assumptions are needed, 
   and why?

Sincerely,
K.W. Hodgkinson

Date: 06/20/98 at 16:03:22
From: Doctor Anthony
Subject: Re: Moment Generating Functions

Problem 1:

By definition:

   Mx(t) = E(e^(tx)) and My(t) = E(e^(ty))

Then:

   Mx(t) * My(t) = E[e^(tx) e^(ty)] provided x and y are independent.

                 = E[e^(t(x+y))]

                 = M(x+y)(t)

The necessary condition is that x and y are independent.

Problem 2:

The probability distribution function (pdf) of an exponential 
distribution with parameter a is f(x) = ae^(-ax). So:

   Mx(t) = INT(0 to inf)[e^(tx) * ae^(-ax) * dx]

         = a * INT[e^(x(t-a)) dx]

This integral converges only if t < a. So the mgf exists only for 
those values of t. Then we have:

   Mx(t) = a/(t-a) e^(x(t-a)) for 0 to infinity

         = a/(a-t)  t < a

Problem 3:

Provided x and y are independent, M(x+y)(t) = Mx(t) My(t).

We have:

   e^(tx) = 1 + tx + (tx)^2/2! + (tx)^3/3! + ...

Taking expectations

   Mx(t) = 1 + tE(x) + t^2 E(x^2)/2! + t^3 E(x^3)/3! + ...

   My(t) = 1 + tE(y) + t^2 E(y^2)/2! + t^3 E(y^2)/3! + ...

To find M(x+y)(t) = Mx(t) My(t), we multiply the two series:

   M(x+y)(t) = 1 + t[E(x) + E(y)] + (t^2/2!)[E(x^2) + 
                                       2.E(x) E(y) + E(y^2)] + ... 

Then:

   Var(x) =  E(x^2) - [E(x)]^2

          = coeff of t^2/2! - [coeff of t]^2   in the series for Mx(t)

Similarly:

   Var(y) = E(y^2) - [E(y)]^2
          
          = coeff of t^2/2! - [coeff of t]^2   in the series for My(t)

Similarly,

   Var(x+y) = coeff of t^2/2! - [coeff of t]^2 in the series M(x+y)(t)

     = [E(x^2 + 2E(x) E(y) + E(y^2)] - [E(x) + E(y)]^2

     = E(x^2) + E(y^2) + 2E(x) E(y) - [E(x)]^2 - 2E(x) E(y) - [E(y)]^2

     = E(x^2) - [E(x)]^2 + E(y^2) - [E(y)]^2

     = Var(x) + Var(y)

Remember that this only applies if x and y are independent.     

-Doctor Anthony,  The Math Forum
Check out our web site! http://mathforum.org/dr.math/   

Date: 06/21/98 at 15:18:11
From: Ken Hodgkinson
Subject: Moment Generating Functions

I am having problems understanding the Moment Generating Function.  

1) If a random variable x has the discrete uniform distribution 
   f(x;k) = 1/k, x = 1,2,3,...k and 0 elsewhere, can it be shown that 
   the MGF of x is Mx(t) = (e^t(1-e^kt)/k(1-e^t) ?

2) If x has a Poisson distribution p(x;m) = e^-m m^x/x! for x = 0, 1,   
   2 ... can it be shown the MGF is = e^(m(e^t-1)) ?

3) Can it be shown that the mean and variance of the chi-squared 
   distribution with v degrees of freedom are, respectively, v and 2v?

Thank you for your time and help.

Sincerely,
K.W. Hodgkinoson

Date: 06/21/98 at 16:20:16
From: Doctor Anthony
Subject: Re: Moment Generating Functions
 
Problem 1:

We define Mx(t) = E(e^(xt)). Then for the uniform distribution given,

   Mx(t) = SUM[e^(xt)(1/k)]

         = (1/k)[e^t + e^(2t) + e^(3t) + ..... + e^(kt)]  
                                             (a geometric progression)

       = (1/k)e^t[1-e^(kt)]/(1-e^t)  as required.

We have used the formula for the sum of a geometric progression:

   a(1-r^n)/(1-r)

Problem 2:

   Mx(t) = SUM[e^(tx) m^x e^(-m)/x!]   x = 0 to infinity

         = e^(-m) SUM[(me^t)^x (1/x!)]
 
         = e^(-m) e^(me^t)

         = e^(m(e^t - 1))  


Problem 3:

The mgf of the X^2 (chi-squared) distribution with v degrees of 
freedom is:

   Mx(t) = (1 - 2t)^(-v/2)

So:

  Mx(t) = 1 + (-v/2)(-2t) + (-v/2)(-v/2 -1)(-2t)^2/2! + .....

        = 1 + t(v) + (t^2/2!)[(v^2/4 + v/2)4] + ....

        = 1 + t(v) + t^2/2!)[v^2 + 2v] + .....

Then  E(x) = coefficient of t  =  v

and   E(x^2) = coefficient of t^2/2  =  v^2 + 2v

So, the mean = v and     

    variance =  E(x^2) - mean^2

             =  v^2 + 2v - v^2

             =  2v

And, as required, we get mean = v  and variance = 2v.    

-Doctor Anthony,  The Math Forum
Check out our web site! http://mathforum.org/dr.math/   

Date: 07/01/98 at 18:24:46
From: Ken Hodgkinson
Subject: Probability

Dr. Math,

I was wondering if you could help me with another question:

   Explain why the "units" of a density f(x) are "probability per 
   unit length." This is based on what assumption?

I appreciate any help you might be able to give me.

Sincerely,
Ken Hodgkinson

Date: 07/02/98 at 05:18:18
From: Doctor Anthony
Subject: Re: Probability

The area standing on any interval of the density function represents 
the probability that x lies in that interval. For example the 
probability that X lies in the interval x to x+dx is f(x) dx. This 
means that f(x) is the probability per unit length at that value of x. 
[i.e. if dx = 1 unit, then probability is f(x), so f(x) is the 
probability per unit length].

- Doctor Anthony, The Math Forum
Check out our web site! http://mathforum.org/dr.math/