Statistical Bias

Date: 09/27/2001 at 23:28:00
From: Kendra
Subject: What is this question asking me?

Suppose that insted of choosing random samples of 25 students from 
a population of 100 students, you selected the first 25 students 
for the first sample, the next 25 students for the second sample, and 
so on. How might this sampling procedure bias the statistical results?

Thank you for your time.
Kendra

Date: 09/29/2001 at 23:36:04
From: Doctor Wolfson
Subject: Re: What is this question asking me?

Hi Kendra,

Let me ask you a similar question that might help clarify it. Let's 
say that I ask you to find out how much food a bear eats during the 
course of a year. I ask you to take three friends with you to study 
the bear for a year, and report back to me one year from today, so I 
can compare your results.

There are (at least!) two ways you could go about studying the bear 
with your friends. In the first way, each morning each of you flips a 
coin twice in a row. If you get two Heads, then you'll watch the bear 
for the day. Otherwise, you'll take the day off. Sometimes none of you 
will get two heads and watch the bear, and sometimes more than one of 
you will be watching at once. But in the end, you'll each have a 
pretty good idea of the bear's eating habits.

In the second way, you decide that you will be the one to watch the 
bear for the first three months, and write your report so you can 
relax for the rest of the year. Then each of your friends will take a 
three-month turn watching the bear and writing a report. In a year 
when you report back to me, will the four reports look similar? Well, 
your report (the first one, during the autumn) will say that the bear 
eats an incredible amount of food, and that if he kept eating like 
that he should be much larger than he really is. Your next friend who 
watched the bear during the winter will tell me that the bear spent 
the whole time sleeping and didn't eat at all! If bears never eat, how 
do they get so big?

Of course the reason that you and your friend got such different 
answers is that there are patterns in the bear's eating habits; during 
the fall, he eats a lot to store up energy to hibernate for the 
winter. But it would be a mistake to take this trend and assume that 
it applies all year long. And if one of the reports (yours) is higher 
than the true average amount the bear eats, then at least one of them 
has to be lower, to keep things in balance. That's why your winter 
friend never got to see the bear eat. And, most likely, the last two 
friends saw relatively normal eating habits.

Can you see how this relates to your question? If you don't pick a 
random sample, then any patterns that you didn't account for in your 
sample can skew the data. And even worse, if the samples don't 
overlap, then each skew will be balanced by a counter-skew; in other 
words, if one sample contains a disproportionately large amount of 
something, then another one has to offset this and be too small. (An 
"average" can't have all the data above it, so if there's an "above 
average," there has to be a "below average.")

I hope this helps. Feel free to write back if you'd like more 
clarification.

- Doctor Wolfson, The Math Forum
  http://mathforum.org/dr.math/