Beware the Crosstabs! Part the Second

Last week I took someone to task for placing too much weight on Gallup’s reports of Latino voters supposedly disproportionate turn away from Barack Obama. One week more, and…it’s disappearing even more. In the Gallup weekly reports, Obama trickled down to 40% approval overall after three weeks at 41%, but his approval among Hispanic citizens is up for the third week in a row and now stands at 54%, 14 points higher than his overall rating. That’s consistent with where he was back in September (that is, it’s down, but not down more than the overall dip).

It’s certainly possible that there’s something going on here, but it’s also possible, and perhaps a bit more likely, that the earlier dip never actually happened. It was just a bit of random variation.

I wanted to post on this not to beat up on the same thing, but because there’s one additional point I didn’t mention which is actually pretty important. Gallup publishes, in their weekly reports, a colossal 41 different splits. Look: flip a single coin 1000 times, and if heads comes up 600 times, something is probably going on. But what you’re doing here is walking into a coin flipping factory, with some flipped 1000 times, and some 100 times, and some only 10 times, and yeah, if all you are doing are looking for oddball results then you’re going to find what you’re looking for.

And so, like clockwork, we have another reporter tweeting out that “Obama’s approval has dropped the sharpest among Eastern wealthy post-grads.” Give it a rest, folks: just don’t trust these blips in the crosstabs. There’s no way of knowing whether they’re real or not.

[Cross-posted at A plain blog about politics]

Washington Monthly - Donate Today and your gift will be doubled!

Support Nonprofit Journalism

If you enjoyed this article, consider making a donation to help us produce more like it. The Washington Monthly was founded in 1969 to tell the stories of how government really works—and how to make it work better. Fifty years later, the need for incisive analysis and new, progressive policy ideas is clearer than ever. As a nonprofit, we rely on support from readers like you.

Yes, I’ll make a donation

Jonathan Bernstein

Jonathan Bernstein is a political scientist who writes about American politics, especially the presidency, Congress, parties, and elections.