The Law of Large Numbers

Date: 02/08/2001 at 17:30:41
From: Garrick Rothstein
Subject: Laws of large numbers

Can you explain the law of large numbers to me?

Date: 02/09/2001 at 11:31:54
From: Doctor Jordi
Subject: Re: Laws of large numbers

Hi Garrick - thank you for writing to Dr. Math.

You want to know about the law of large numbers? Sure, I can explain 
it to you. It really is not something you don't already know from 
common experience, but perhaps its mathematical formulation is not 
quite as simple.

Simply stated, the law of large numbers states that if you repeat a 
random experiment, such as tossing a coin or rolling a die, many, 
many, many times, your outcomes should on average be equal to the 
theoretical average.

For example, say you have a fair coin, such that the odds of getting 
heads are the same as the odds of getting tails. Say you toss this 
coin 100 times. How many heads and how many tails would you expect to 
get?

Well, you can't predict exactly how many tails and heads you'll get.  
But you can probably give me an estimate: about 50 heads and about 50 
tails. Maybe a little more on one side or the other, say maybe 48 
heads and 52 tails, but somewhere in that range.

What would you think if you got 90 heads and only 10 tails? Even 
worse, what if you got all heads and no tails at all? Is that 
impossible? Well, certainly not - it is conceivable that by a great 
coincidence you could toss the coin 100 times and get all heads, but 
then something very fishy would be afoot. You would have a very 
strong reason to suspect that your coin was not fair at all.

Basically, that's the law of large numbers. By the law of large 
numbers, if you toss the coin many times, say 100, you should expect 
to get about the same number of heads and tails. Furthermore, if you 
tossed it many more times than 100, say 10,000, you would expect to 
get even closer to a 1:1 ratio of heads and tails (closer and closer 
to 50% heads and 50% tails). The more times you repeat the 
experiment, the closer you should get to the true theoretical average 
of heads and tails.

On the Web, see Philip B. Stark's Glossary: Law of Large Numbers:

  http://www.stat.berkeley.edu/~stark/SticiGui/Text/gloss.htm#l   

and his Java applet, which lets you change the number of trials and 
the probability of success in each trial, and toggle between viewing 
either the difference between the number of successes and the expected
number of successes, or the difference between the percentage of 
successes and the probability of success in each trial: 

  http://www.stat.berkeley.edu/~stark/Java/lln.htm   

If this is still confusing, or if you need a more solid mathematical 
formulation of the law of large numbers, please write back.

- Doctor Jordi, The Math Forum
  http://mathforum.org/dr.math/