FEAR OF HELL….This is a mega-wonky post on a subject of no importance, but what the hell. Feel free to ignore it if you want.
Dan Drezner links today to a report from the St. Louis Fed and concludes that it’s “pretty weak.” I read the same report last night, and it turns out that “pretty weak” is being altogether too polite.
Basically, two economists decided to test the hypothesis that fear of hell is good for economic growth. Now, it turns out that they actually have raw data on this: a column of numbers showing the percent of people in each country who believe in hell, and another column of numbers showing GDP per capita for each country. (Why GDP per capita when it’s economic growth they’re concerned about? Beats me. But let’s move on anyway.)
So what’s the correlation? It turns out to be -.21. However, the number you care about is the square of the correlation, which is .04, or 4%. This means that even if you assume the raw data is both correct and robust ? a fairly heroic assumption ? belief in hell explains only 4% of the variance in GDP per capita between different countries. In other words, it hardly explains a thing.
But the authors weren’t satisfied with that, so they did two things. First, instead of using raw data they ranked each country and computed a rank correlation. (Is that legit? Maybe. Hard to say. And did they even do a rank correlation, which is a specialized statistic? Doesn’t look like it. But let’s keep moving.)
Second, they chose to break the correlation into two correlations:
Belief in hell vs. amount of corruption
Amount of corruption vs. GDP per capita
Having done all that, they computed the first correlation and got a value of -.34. Square that and you get .11, or 11%, which is a lot better than 4%. Hurrah!
However, you can also compute the rank correlation directly: belief in hell vs. GDP per capita. If you do, the correlation is .15. Square it and you get 2%. That’s even worse than the 4% that Dan got using the raw data, and is so low as to be completely worthless.
This strikes me as worthy of John Lott:
Run a correlation on the raw data. Hmmm. 4%. Not so good.
Try a rank correlation instead. Hmmm. 2%. Even worse.
I know! Do a rank correlation and break the correlation into two stages. Bingo! As long as you don’t look too hard.
How does stuff like this get past peer review?
UPDATE: After I published this post I clicked all the links to make sure they were correct (as I always do). And guess what? The Fed article now has an editor’s note:
Below is a new version of this article, which discusses recent research in economics regarding a possible relationship between economic growth and religion. It is the second revision that has been posted. In both the original version and the first revision, the article ended with a discussion of simple correlations between countries? religiosity, levels of corruption and per capita incomes. The purpose of these discussions was to use a very simple framework to illustrate the results found in the literature. Because the discussion did not go any deeper than simple correlations, it was never intended to be a substitute for serious statistical analysis.
Thanks to the keen eyes of a number of readers, however, we have discovered that the charts used in both of these versions of the article contained errors. Consequently, the version below does not include discussions of the correlations between religiosity, corruption and per capita income. It is important to note that this has no bearing on the results in the literature that are discussed in the article. It is not uncommon, for example, for simple correlations between two variables to provide different answers from regressions that control for a longer list of variables.
I would like to apologize to any of our readers who have been inconvenienced by this series of corrections. In addition, I would like to thank all of those who picked up on the errors and let us know about them.
In other words: this was just simplistic crap and it wasn’t even computed correctly at that, so we’re deleting the whole thing except for the literature review that was formerly just an introduction to the data. Glad to see that the St. Louis Fed holds itself to such high standards.