In article <email@example.com>, firstname.lastname@example.org (Randy Poe) wrote: > Caught just a piece of a news story on the radio a couple of days ago, > where they were talking about the difficulties pollsters face trying > to get meaningful projections in this close US presidential race. One > pollster was saying that he knew of cases where two polls differed by > 15 points in the same state, and he seemed to be implying that was > connected to the closeness of the race. > > This started me wondering what sort of model could explain that. > > We're trying to estimate p = the fraction of voters who will vote for > Gore. We do this by measuring p-hat, a fraction of people polled who > say they'll vote for Gore. Mostly I guess p-hat is assumed to be > normal with mean p and variance depending on sample size. > > Obviously the sample has to be truly random and unbiased in some sense > for this to work, and presumably the polling companies have techniques > they use to try to eliminate bias. This guy seemed to be implying that > the nature of the race could either be introducing biases, or > increasing the variance of p-hat in some other way. > > What could be going on here? > > - Randy >
Scott Rasmussen (of the Portrait of America poll) was interviewed on TV the other night. He was asked to explain why two polls in the NY Senate race were so far apart. His answer was that the NY Times assumed that 31% of the voters would come from NY City, while Zogby assumed instead 26% or 27%. This is certainly enough to explain at least a part of the discrepancy, though not perhaps the whole of it. For the record, the Times assumption is based on the results in the 1996 Presidential election, while the Zogby assumption is based on the results in the 1998 Senate election.
It used to be that pollsters used a large enough sample size (about 2000 in a Presidential election) to keep the expected error down. However, this makes it expensive and slow to do a poll, and there is still a lot of jitter in the numbers. Nowadays, they sample a much smaller group and then twiddle the numbers to conform to their expectations, resulting in a self-fulfilling prophecy. This makes their clients in the media happier, at the expense of being "mathematically incorrect". The press is by and large innumerate anyway, so no one is likely to complain. The only poll today which behaves as I would expect a mathematically correct poll to behave is Gallup, and that's the one the press distrusts most.
The existence of systemic bias is easy to prove. In the absence of bias, one would expect that for any two contemporaneous independent polls, the spread in one should be greater than the spread in the other about 50% of the time. An actual comparison over the last 10 days shows that the Battleground poll has Bush leading by a wider margin than the Zogby poll for 9 of the 10 days, with the 10th day being a tie. This is highly unlikely if there is no bias. Comparisons between other polls lead to a similar conclusion.