Now that a couple of national polls have come out showing Romney moving a bit closer to Obama, we’re beginning to hear that ol’ chestnut: “The presidential race is in a statistical tie.” As it happens, the best brief demolition of that claim, and the best brief description of what “margin of error” means in polls, was done right here at PA by Kevin Drum in 2008. Here’s a snippet:
I originally wrote about this back in 2004, but here it is again. The idea of a “statistical tie” is based on the theory that (a) statistical results are credible only if they are at least 95% certain to be accurate, and (b) any lead less than the MOE is less than 95% certain.
There are two problems with this: first, 95% is not some kind of magic cutoff point, and second, the idea that the MOE represents 95% certainty is wrong anyway. A poll’s MOE does represent a 95% confidence interval for each individual’s percentage, but it doesn’t represent a 95% confidence for the difference between the two, and that’s what we’re really interested in.
In fact, what we’re really interested in is the probability that the difference is greater than zero — in other words, that one candidate is genuinely ahead of the other. But this probability isn’t a cutoff, it’s a continuum: the bigger the lead, the more likely that someone is ahead and that the result isn’t just a polling fluke. So instead of lazily reporting any result within the MOE as a “tie,” which is statistically wrong anyway, it would be more informative to just go ahead and tell us how probable it is that a candidate is really ahead.
By this standard, horse-race results within the MoE aren’t “ties,” but simply represent a reduced probability of the lead reported in the top-line results. While it’s possible Romney is actually at parity with Obama, it’s very unlikely, particularly since the Republican has been stuck at 47% or below throughout the entire cycle. It’s also worth noting that the MoE tends to be larger in state polls, since samples are typically smaller, but the same cautions must be expressed in talking about “statistical ties.”
It’s amusing in any event to see some pro-Romney gabbers seamlessly shifting from claims that all the polls are “skewed” to crowing over any relatively positive numbers that happen to come out, but I guess they are “arguing in the alternative.” Any evidence that the Great National Repudiation of Barack Obama is not just on the horizon is psychologically intolerable.
