Ken Rogoff and Carmen Reinhart have become known for warning that debt-to-GDP ratios over 90 percent are linked to poor economic growth. But a new working paper from economists at the University of Massachusetts says Reinhart and Rogoff’s findings are created by bad methodology. They seem to have a point.
Thomas Herndon, Michael Ash and Robert Pollin say that the Reinhart-Rogoff finding of sharply lower average growth in high-debt countries rests on three errors: a bad weighting method, inexplicable exclusion of data from certain countries and years, and an Excel coding error. After fixing those problems, they find that “average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.”
Yesterday’s defenses from Reinhart and Rogoff are most notable for what they do not contain: a defense of the methodology that Herndon and his colleagues criticize. Instead, they make two main arguments. One is that a 2012 paper, which Reinhart and Rogoff wrote with Vincent Reinhart, supports their finding that high debt is associated with low growth, as do several papers from other researchers. Their other response is that the really important matter is median performance, not the mean performance statistic that the UMass researchers are critiquing.
“We have generally emphasized the 1% differential median result in all our discussions and subsequent writing,” Rogoff told me in an e-mail. That is, in the graphic of means and medians below, from their 2010 American Economic Review paper (similar free version available here), the really important statistic is the median growth figure. That number is about 1 percentage point lower for the high-debt countries, not the average figure that falls off a cliff.
Source: “Growth in a Time of Debt.” Carmen M. Reinhart and Kenneth S. Rogoff. American Economic Review Volume 100. May 2010
In the same e-mail, Rogoff points out that 1 percent less GDP growth per year is a big deal. “Note that because the historical public debt overhang episodes last an average of over 20 years, the cumulative effects of small growth differences are potentially quite large. It is utterly misleading to speak of a 1% growth differential that lasts 10-25 years as small.”
This isn’t a very satisfying defense, for two reasons.
One is the simple sloppiness of the apparent error that Reinhart and Rogoff made in calculating the average GDP growth for high-debt countries. Barry Ritholtz put it well: “Whenever you hear the phrase ‘the fundamental case is intact,’ you best run screaming from the room.” It’s reminiscent of the “fake but accurate” defense of the forged memos about President George W. Bush’s service in the Texas Air National Guard.
Dean Baker has a good breakdown of what went wrong. What happened is that Reinhart and Rogoff included data from New Zealand in 1951, a year when debt/GDP exceeded 90 percent and GDP growth was -7.6 percent. But they excluded New Zealand data for the period 1946 to 1949, when debt also exceeded 90 percent and growth was often very strong. Include those years, and New Zealand’s average economic growth in high debt years is not -7.6 percent but +2.6 percent.
Because there were only seven countries in the data set that Reinhart and Rogoff used to calculate average GDP growth under high debt conditions, and because they weighted each country’s average growth equally, getting New Zealand wrong by more than 10 percentage points was a very big deal, shaving 1.5 percentage points off their estimate of average growth.
This error seems to negate the main reason that the Reinhart-Rogoff paper got so much attention in the first place, since observers tended to fixate on the growth cliff for high-debt countries, a cliff apparently created solely by this New Zealand-related oversight.
Early this morning, Reinhart and Rogoff put out a more detailed response saying they excluded the pre-1950 New Zealand data because it was newly available at the time they published the paper in 2010 and they had “not yet vetted the comparability and quality” of those data. They take exception to the claim that the omission of the pre-1950 data was “selective.”
Even if Reinhart and Rogoff had a plausible reason for leaving out these data, the huge effect of the exclusion demonstrates the fragility of their method: New Zealand is one of just a handful of rich countries that experienced a debt-to-GDP ratio over 90 percent since World War II, and leaving those years in or out materially changes the conclusion about the average growth rate of such countries.
And what of Rogoff’s point that even Herndon et al. agree that higher debt is associated with slower growth? The most damaging response to this is a figure near the back of the UMass paper, plotting every observation of GDP growth and debt-to-GDP ratio from the advanced countries, from the 2010 Reinhart-Rogoff paper. This figure seems to show what Rogoff says it does: Even the UMass researchers’ take on the data show a negative relationship between government debt and GDP growth.
Source: “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff.” Thomas Herndon, Michael Ash and Robert Pollin. Public Economy Research Institute at University of Massachusetts: Amherst Working Paper Series. April 2013
Unfortunately for Reinhart and Rogoff, the most notable feature of this chart is not the trend. It is how weakly the data fit the trend. Is this a chart that suggests to you that countries seeking to improve real GDP growth should focus on constraining their ratios of public debt to GDP?
Such policies pose significant economic risks and human costs, and yet what this chart tells you is that low debt and poor growth, and high debt and strong growth, are both reasonably common outcomes.
Unlike the cliff chart that made the Reinhart-Rogoff paper so arresting, this way of looking at the data suggests that the historical performance of countries with varying debt-to-GDP ratios has little to tell us about what today’s fiscal policies should be.
