EXIT POLL FINALE….Edison/Mitofsky, the company responsible for exit polling in the 2004 election, released their final report today about what went wrong. For starters, it appears that although there was one technical glitch related to sample weighting, weighting was not the overall problem:
The weighted national survey numbers showed Kerry with 51% and Bush with 48%. The final national popular vote margin ended up being 2.5% for Bush. Thus, the national exit poll had an error of 5.5 points on the difference in the Democratic direction which is similar to the 5.0 average from the state surveys.
In other words, even with the initial weightings applied, the results were still wrong by 5.5 percentage points.
So what was the problem?
The inaccuracies in the exit poll estimates were not due to the sample selection of the polling locations at which the exit polls were conducted. We have not discovered any systematic problem in how the exit poll data were collected and processed. Exit polls do not support the allegations of fraud due to rigging of voting equipment. Our analysis of the difference between the vote count and the exit poll at each polling location in our sample has found no systematic differences for precincts using touch screen and optical scan voting equipment.
Having eliminated those possibilities, they end up back where they started: they believe the problem is that Kerry voters were more likely to fill out exit poll questionnaires than Bush voters. There’s no way to really prove this, but they offer several suggestive statistics:
The error rate was smaller in precincts where the interviewers attempted to survey every voter. Error rates went up in larger precincts where interviewers surveyed every third or every tenth voter.
Error rates were higher in swing precincts (those that had approximately equal numbers of Democrats and Republicans).
Older interviewers had smaller error rates than younger interviewers.
Error rates went up significantly if interviewers were required to stand 100 feet away from the polling place.
The report’s conclusion is that when they had greater discretion (i.e., in swing precincts and in larger precincts where they couldn’t interview everyone), younger interviewers unconsciously tended to select younger voters, who were more likely to have voted for Kerry. Likewise, younger voters were more likely than older voters to agree to fill out a survey from a young interviewer.
There’s still a fair amount of guesswork involved in this, though, and it seems to have been mostly a process of elimination. No other source of error appears to have been present, so the error must have come ? somehow ? from an oversampling of Kerry voters in individual precincts.
And the solution? Basically, better training of interviewers and “legal remedies” to reduce the distance that some polling officials enforce on the interviewers. Not very exciting stuff.