Deriving the Normal From the Binomial

Date: 07/11/98 at 22:45:41
From: Jason 
Subject: Derivation of normal distribution

Please tell me how the normal distribution is derived from the 
binomial distribution. Can it be proven that the normal distribution 
approximates the binomial distribution as the number of trials goes to 
infinity? Or can you please tell me a Web site or a book I should 
consult? 

Thank you very much for your time.

Date: 07/12/98 at 07:25:51
From: Doctor Anthony
Subject: Re: Derivation of normal distribution

Derivation of the Normal distribution from the Binomial distribution
---------------------------------------------------------------------

Let a variate take values 0, k, 2k, 3k, ..., nk  with probabilities 
given by successive terms of (q + p)^n.

Then the mean m = npk and the variance s^2 = npqk^2.

Suppose:

   y = probability of occurrence of rk = C(n,r) p^r q^(n-r)

Also let:

   y' = probability of occurrence (r+1)k  = C(n,r+1)p^(r+1) q^(n-r-1)

Then:

   y' - y = C(n,r+1)p^(r+1) q^(n-r-1) - C(n,r)p^r q^(n-r)

             n!p^r q^(n-r-1)
          =  ---------------[(n-r)p - (r+1)q]
              (r+1)! (n-r)!       

And:

   y' - y      1                         1
   ------  = ------[np - r(p+q) - q] = ------[np - r - q] (Equation 1)
     y       (r+1)q                    (r+1)q

Let:

   x = rk - npk, so that x is now the variate measured from the mean.

Then:

   r = x/k + np      and       r+1 = x/k + np + 1

Thus:

   k(r+1) = x + k + npk

   k^2 (r+1)q = (x + k + npk)qk

Multiply top and bottom of the righthand side of Equation 1 by k^2. 
Then:

   y' - y   [(np-r)k - qk]k       
   ------ = ---------------           [note that (np-r)k = -x]
      y     [x + k + npk]qk 

              (-x - kq)k
          = ----------------           (Equation 2)
            npqk^2 + (x+k)qk 


We now let k = dx, so that y' - y = dy and let n ->infinity in such a 
way that nk^2 is finite. Equation 2 can then be written as:

    dy     (-x - q dx)dx
   ---- = ----------------
    y     s^2 + (x+dx)q dx

As dx -> 0 this becomes:
 
    dy     -x dx
   ----  = ------
    y       s^2

Integrating, we get:

   ln(y) = - x^2/(2s^2) + constant

   y = e^(-x^2/(2s^2) + constant)

   y = A e^(-x^2/(2s^2))

If we integrate this from -infinity to +infinity we get the area under 
the curve, and we choose the constant A to make this area equal to 1. 

To see how this integral is obtained, see:

   Algebraic Integration of Standard Normal Distribution Function
   http://mathforum.org/library/drmath/view/53628.html   
 
We find that:

           1
   A = -----------
      s.sqrt(2.pi)
                                              
So the formula for the Normal pdf curve is:

        1
   ----------- e^(-x^2/(2s^2))
   s sqrt(2pi)

- Doctor Anthony, The Math Forum
  http://mathforum.org/dr.math/   

Date: 06/21/2003 at 04:27:27
From: Jeff 
Subject: Deriving the Normal From the Binomial

I'm having trouble following one of the steps in the derivation:

 |     y' - y     (-x - kq)k
 |     ------ = ----------------          
 |        y     npqk^2 + (x+k)qk 
 |
 |
 |  We now let k = dx, so that y' - y = dy and let n ->infinity 
 |  in such a way that nk^2 is finite.  The equationcan then be 
 |  written as:
 |
 |      dy     (-x - q dx)dx
 |     ---- = ----------------
 |      y     s^2 + (x+dx)q dx
 |
 |   As dx -> 0 this becomes:
 |
 |      dy     -x dx
 |     ----  = ------
 |      y       s^2

It seems to me that if we simplify the above expression just before 
the dx -> 0 we get

       -x dx - q dx^2
   -----------------------
    s^2 + x q dx + q dx^2

The terms with dx^2 will go to 0 very quickly and will contribute 
very little, leaving 

     -x dx
  --------------
   s^2 + x q dx

What am I missing?

-Jeff

Date: 06/30/2003 at 12:30:25
From: Doctor Anthony
Subject: Re: Deriving the Normal From the Binomial

Hi Jeff, 

As dx -> 0, xq dx is zero _compared_ to s^2, which is why 

     -x dx
  --------------
   s^2 + x q dx

reduces to 

    -x dx
  ---------
     s^2 

In the numerator, we are _multiplying_ by dx and this cannot be ignored;
but in the denominator we are _adding_ a term with dx and this _can_ be ignored.

If we derive the formula for dy/dx in a simple case such as y = x^2 from
first principles we can reason as follows:

If (x+dx, y+dy) is a point on y = x^2 then

      y+dy = (x+dx)^2

      y+dy = x^2 + 2x dx + (dx)^2         Cancel y and x^2

        dy =  2x dx + (dx)^2              Now divide through by dx

     dy/dx = 2x + dx

Now let dx -> 0, so dy -> 0 also; and the left hand side -> 0/0 
while the right hand side becomes 2x.  (We can now ignore dx since it is 
_added_, and not multiplied.)

Now 0/0 is NOT equal to 0 or 1 but tends to different things in different
situations.  (In this case it tends to 2x.)

For example  

  17 * 0 = 0  
  
so dividing through by 0 (assuming this is allowed) you get

  17 -> 0/0

Clearly we could also get 

  213 -> 0/0 
  
or 

  -78 -> 0/0 
  
or 

  anything you care to think of -> 0/0  
  
In fact, the various values for dy/dx are all values we obtain for 0/0.

When you see an expression like dy/dx = 2x  you can think of an 
infinitely small increment in y divided by an infinitely small 
increment in x (in effect 0/0) has the value 2x.

- Doctor Anthony, The Math Forum
  http://mathforum.org/dr.math/