Galluping To Conclusions

It’s safe to say that yesterday’s Gallup Tracking Poll showing Mitt Romney opening up a 7-point lead among likely voters produced a serious freakout amongst the chattering classes. Weren’t we supposed to be seeing some positive momentum for Obama from the Biden/Ryan debate and then the second presidential debate? Hadn’t the Romney Surge from the first debate subsided? Some conservatives began to cultivate their inner Dick Morris and prepared celebrations for The Romney Landslide. All this overreaction, of course, was fed by a relative dearth of big national polls in the last few days, aggravated by widely varying surveys of battleground states (though most showed significant progress for Romney).

It’s possible to get a little more perspective this morning. Nate Silver published a long post last night that noted Gallup had published outlier polls in past elections:

Usually, when a poll is an outlier relative to the consensus, its results turn out badly.

You do not need to look any further than Gallup’s track record over the past two election cycles to find a demonstration of this.

In 2008, the Gallup poll put Mr. Obama 11 points ahead of John McCain on the eve of that November’s election.

That was tied for Mr. Obama’s largest projected margin of victory among any of the 15 or so national polls that were released just in advance of the election. The average of polls put Mr. Obama up by about seven points.

The average did a good job; Mr. Obama won the popular vote by seven points. The Gallup poll had a four-point miss, however.

In 2010, Gallup put Republicans ahead by 15 points on the national Congressional ballot, higher than other polling firms, which put Republicans an average of eight or nine points ahead instead.

In fact, Republicans won the popular vote for the United States House by about seven percentage points — fairly close to the average of polls, but representing another big miss for Gallup.

Apart from Gallup’s final poll not having been especially accurate in recent years, it has often been a wild ride to get there. Their polls, for whatever reason, have often found implausibly large swings in the race.

Nate Cohn chipped in with a look under the hood of the Gallup methodology, and particularly its likely voter screen, which produced a projected electorate that looked more like 2010’s than 2008’s:

[T]he idea that the electorate would be less white than the 2010 midterm elections seems harder to imagine, but Gallup’s likely voter universe is actually even whiter than their likely voter surveys prior to the 2010 midterm elections, which was 79 percent white.

And finally, this morning Ezra Klein published excerpts from a discussion with Gallup’s Frank Newport, after noticing that virtually the entire Romney lead was produced among respondents from the South, making him wonder if Gallup might be projecting a split between the popular and electoral college votes:

Last night, I spoke with Frank Newport, editor-in-chief of Gallup, to ask him if I was missing something. He said I wasn’t. “That’s certainly what it looks like,” he says.

But Newport was cautious in interpreting his numbers. Gallup’s poll cheered Romney supporters because it showed Romney gaining ground even after the second debate. But Newport didn’t see it like that. Remember, he warned, it’s a seven-day poll. “I think we’re still seeing leftover positive support for Romney and I don’t think we’re seeing impact yet from the second debate,” he says.

What you think is going on in the race depends on whether you think the electorate will ultimately look more like Gallup’s “likely voter” model, where the race is a blowout, or all registered voters, where it’s a dead heat. So I asked Newport to explain the likely voter model to me.

“The likely voters model takes into account changes in the response to questions about how closely they’re following and how enthusiastic they are,” he said. “It’s not just capturing underlying movement — it’s representing changes in enthusiasm.”
That sounds, I replied, like a model that would tend to overstate the effects of major events that favored one candidate or the other, as their supporters would grow temporarily more enthusiastic and attentive, while the other side would grow temporarily disillusioned. Newport agreed. “I wouldn’t use the word ‘overstate,’ ” he said. “But it would be very sensitive to changes in enthusiasm.

Okay. So Gallup’s own chief seems to be hinting we are likely to see additional big swings in their Tracking Poll quite soon, a tendency that Nate Silver notes is not unknown to characterize Gallup results in the recent past. So there will be even more attention paid to the Gallup site than is usual when it posts its next update at 1:00 EDT this afternoon.

Meanwhile, NBC/WSJ/Marist published two new battleground polls partially capturing post-second-debate sentiment (they were in the field Monday through Wednesday) and showing Obama’s leads holding up pretty well in Iowa (8 points among likely voters) and Wisconsin (6 points among LVs). Both polls suggested a landscape similar to that that prevailed before any of the debates. The Wisconsin results were particularly interesting as a contrast to yesterday’s Marquette Law School survey, taken entirely before the second debate, which showed Obama’s lead down to one point. And the NBC/WSJ/Marist findings also indicated that Obama was doing very well among early voters in both IA and WI.

I’d normally say “we’ll know soon who’s right,” but then again, we have another presidential debate on Tuesday, so Lord only knows when there will be a juncture of non-event-driven public opinion that can be reliably measured. For the moment, everybody has reason to freak out with fear or joy.

Ed Kilgore

Ed Kilgore, a Monthly contributing editor, is a columnist for the Daily Intelligencer, New York magazine’s politics blog, and the managing editor for the Democratic Strategist.