Just had to pass along and endorse this criticism by The Atlantic‘s Emma Green of really dumb poll questions about really big issues:
In one of the most ambiguous poll questions ever, Gallup has asked Americans to once again act as a tenuous bellwether for the impending death of religion. In a May survey, more than 1,000 people were asked to pick between two vague sense impressions of faith: “Do you believe that religion can answer all of today’s problems, or that religion is largely old fashioned and out of date?”
Shockingly, only 13 percent of people took the out of answering “no opinion” or “other,” which is one way of saying “I will not fall for the misleading set-up of your question, pollster man.” This year, 57 percent of respondents said religion is the answer for basically everything, and 30 percent said it’s “old-fashioned.”
As far as false choices go, this one is pretty extreme. First and foremost, most of the respondents probably would have answered “some of both,” had that been possible. Being anachronistic and being useful aren’t mutually exclusive concepts; different parts of different faiths could arguably qualify as either, both, or neither.
But more importantly, these options are conceptually meaningless. When the poll says “all of today’s problems,” is it referring to political gridlock? America’s inefficient healthcare system? “Mo’ money”? The concept of “out-of-date” or “old-fashioned” is even more troubling: It’s built on a very particular idea of history, which is that human civilization has been progressing linearly toward a modern age. The underlying assumption is that secularization theorists are at least somewhat correct: Religion is somehow “old” or pre-modern or even anti-modern, whereas secular life is “new” or fully modern or post-modern. Even though the poll is ostensibly asking respondents to say whether they think this is right, the premise of secularization theory is baked into the way they asked the question. That’s why it’s framed in terms of “new” and “old”—or, in other words, relevant and irrelevant.
You certainly don’t have to be religious to be offended by this mode of questioning, which mischaracterizes both belief and unbelief. And it provides another cautionary tale about poll findings that are cited out of their sometimes questionable context and projected as empirical evidence of very big happenings.