When disputes break out among academics, most people don’t care. For good reason: Academic controversies are usually hard to follow, and concern topics that wouldn’t matter to most of us even if we understood them. (I was in an academic dispute once, and my side won. Trust me, you don’t want to hear about it.)
But this week a controversy broke out in economics, and it actually deserves your attention. A paper that has had a major influence on public policy around the world turns out to be wrong. And not just wrong in a subtle way that only geniuses can see, or even wrong in an everybody’s-human way that you look at and say, “Oh yeah, I’ve done that.” This one was wrong in three different ways that make you (or at least me) say, “That can’t be an accident.”
The bogus paper came out in 2010: “Growth in a Time of Debt” by Carmen Reinhardt and Ken Rogoff (both from Harvard). The paper that refutes it appeared last Monday: “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff“ by Thomas Herndon, Michael Ash, and Robert Pollin (all from the University of Massachusetts).
Before I get into the back-and-forth of it, let’s return to why you should care. It has to do with whether the government should be trying to create jobs or cut spending.
Stimulus vs. austerity. Many countries came out of the Great Recession with a much larger national debt, but persistent unemployment and slow growth. And that led to a debate: The usual thing a government does when it has high unemployment and slow growth is spend money. (People need jobs and the private sector is skittish about expanding, so the government hires people to do things that need doing: building highways, fixing sewers, insulating homes, and so on. Or maybe the government boosts the economy by subsidizing certain kinds of consumption, like the popular cash-for-clunkers program that got a bunch of old gas-guzzling cars off the road.)
But maybe this time the thing to do was to cut spending, because of all that debt. Maybe spending more, and so increasing the national debt, would just make things worse.
The same debate was happening in all countries, and none of them went completely one way or the other. But the poster child for austerity has been the United Kingdom, where it hasn’t worked. Here’s how British economic growth has compared to the projections made by the UK’s Office for Budget Responsibility. Austerity has brought the UK essentially no economic growth for three years.
The US has had its own stimulus/austerity debate, which has kept the Obama administration from spending as much as it wanted (or as much as Paul Krugman wanted, which was even more). But compared to the other major economies, the US has been on the stimulus side of the debate, which is probably why (disappointing as our economy has been these last few years) we’re doing better than most other countries. (This graph is scaled so that all countries are equal when austerity-loving David Cameron became the UK’s prime minister.)
Basically, the US and Germany are the only countries in that group that have seen any net growth since 2008.
The gist of what we’ve seen since 2008 is: Keynes was right. In the long run you probably want to keep your national debt under some kind of control, but not when you have high unemployment and slow growth.
How Reinhart/Rogoff leads to Ryan. Now, obviously, the budget debate we keep having in Washington doesn’t acknowledge this reality at all. Conservatives like Paul Ryan and Rand Paul, who want drastic cuts in government spending (to them, the sequester is just a down payment), somehow get away with claiming to have a “pro-growth” agenda.
How is that possible? Well, partly it’s just dogma. The Gospel According to Ayn Rand states that government is always and eternally bad for the economy — she called for “a complete separation of state and economics” — and no accumulation of facts can outweigh holy writ.
But also, a handful of economists provide academic cover for the “pro-growth” austerity nonsense. And the biggest fig leaf in the bunch is the Reinhart/Rogoff paper. In his 2013 budget proposal, Ryan wrote:
Even if high debt did not cause a crisis, the nation would be in for a long and grinding period of economic decline. A well-known study completed by economists Ken Rogoff and Carmen Reinhart confirms this common sense conclusion. The study found conclusive empirical evidence that gross debt (meaning all debt that a government owes, including debt held in government trust funds) exceeding 90 percent of the economy has a significant negative effect on economic growth.
More precisely, R/R found a “threshold” that gets crossed when a nation’s public debt exceeds 90% of the annual GDP. (The United States currently has a debt-to-GDP ratio around 100%. It was comfortably below the 90% “threshold” until almost exactly the moment the R/R paper appeared.) In other words: All your economic intuition and experience might tell you not to cut spending in a slow-growth environment, but something magic happens when debt crosses 90%. Beyond that point, debt suddenly becomes toxic.
Jared Bernstein comments on the significance:
Those whose goal is severely shrinking the size of government in general and social insurance in particular need hair-on-fire results like this from established experts to keep the fire going, even in the face of statistics that lean strongly the other way
What they did and why it’s wrong. Reinhart and Rogoff looked at 20 industrialized countries year-by-year and divided the country-years into four bins: years when the national debt was 0-30% of GDP, 30-60%, 60-90%, and over 90%. They found significantly lower average economic growth in the over-90% bin. The average annual growth rates for the four bins in the 1946-2009 (post-WW2) period were 4.1%, 2.8%, 2.8% and negative .1%.
Now, if you look at those countries and years one-by-one, the case isn’t always impressive. For example, 1946 in the US. We had a lot of debt because we’d just fought World War II, and we had a recession because all the discharged soldiers and laid-off tank-factory workers hadn’t found new jobs yet. So high debt and negative growth were happening at the same time, but not because government debt was killing the economy.
Those are the kinds of one-off situations that you hope cancel out in the averages. And they kinda-sorta do, if you assemble your data honestly and do the math right. Unfortunately, R/R did neither. When Herndon/Ash/Pollin go back and do the analysis right, growth in the over-90% bin jumps from negative 0.1% to positive 2.2%.
So what mistakes did R/R make? Well, one was really stupid: They plugged the wrong row number into a formula on their spreadsheet, so their average skipped a bunch of rows, representing 6 of the 20 countries. (They’ve confessed to that mistake.)
Second, their data-set didn’t really include all the country-years it should have. So, for example, New Zealand only has one year in their average, when it ought to have five. Unfortunately, that makes a huge difference in the country average, because that one year NZ had -7.9% growth, when the five-year average was +2.6%.
And third, they made the bizarre choice to average by country rather than by country-year. So that one anomalous year in New Zealand ended up constituting 1/14th of the entire average rather than the 1/110th it should have.
Why it’s so bad. The significance of the R/R paper comes entirely from those mistakes.
Yes, an honest and accurate accounting still shows a negative correlation between growth and debt-to-GDP ratios, but everybody would have expected that anyway, because there’s well-known causality in the other direction: recessions cause debt/GDP ratios to rise*. (GDP goes down because that’s the definition of a recession. Debt goes up for two reasons: Revenue drops because there’s less income to tax, and spending rises to pay for more unemployment insurance and food stamps.)
The only significant part of R/R was the threshold, and that was wrong: The something-magic-happens-at-90% was just a spreadsheet typo plus statistical sleight-of-hand.
So the data R/R assembled provides absolutely no reason to have some special fear about the current level of debt in the US. We haven’t just passed through some economic equivalent of the sound barrier. To the extent that debt was bad before, it’s still bad, and to the extent that it didn’t matter before, it still doesn’t matter.
Fraud. I anticipate taking heat for using the word fraud in the title. The Herndon/Ash/Pollin paper doesn’t use it, and to fully justify fraud you’d have to see into the hearts of Reinhart and Rogoff. Responsible academics are slow to use words like fraud, because academics are cautious in general. You’re not supposed to publish something you can’t fully prove, even if your rivals do.
But I’m not an academic any more, so I’m using a preponderance-of-evidence standard, not a beyond-reasonable-doubt standard. Let’s look at the three mistakes.
The spreadsheet error shows an unbelievable level of negligence, but if that were the only mistake I’d be inclined to give R/R some benefit of the doubt. The original mistake was almost certainly honest, but not finding the mistake is the real culpability. They didn’t look the gift horse in the mouth; the mistake gave them the result they wanted, so they didn’t check too hard.
They claim to have filled in the missing data in later research, but they’ve done nothing to point out what a difference it makes. And they defend their weighting scheme — an argument I could buy if they had defended that scheme in the original paper while pointing the major difference it made in the result. But they didn’t. They were hoping the readers wouldn’t notice.
In their response to H/A/P, Reinhart and Rogoff, defend their non-spreadsheet errors “in the strongest possible terms”.
But surely the authors do not mean to insinuate that we manipulated the data to exaggerate our results.
I can’t speak for H/A/P, but I won’t insinuate anything, I’ll say it outright: Yeah, R&R, you manipulated the data to exaggerate your results.
R/R’s response. One proof of the fraud is that they’re still doing it. Their response claims:
We do not, however, believe this regrettable slip [the spreadsheet error] affects in any significant way the central message of the paper or that in our subsequent work.
And that’s just flatly false.
Do Herndon et al. get dramatically different results on the relatively short post war sample they focus on? Not really. They, too, find lower growth associated with periods when debt is over 90 per cent.
And that’s sophistry. The “relatively short post war sample” are the economies that happen to resemble the United States today. And “lower growth” is not the result the paper is noted for; no one would care if that were the whole message, because that is completely explained by the well-known recession-causes-debt relationship. The 90% threshold is the paper’s claim to fame, and that result has blown up completely.
And finally, while they don’t explicitly claim that they’ve found a debt-causes-slow-growth relationship, they keep using their result as if they had. They do so even in their response:
There is also the question of whether these growth effects can be economically large. Here it is very misleading to think of 1 per cent growth differences without recognizing that the typical high debt episode lasts well over a decade (23 years on average in the full sample.)
It is utterly misleading to speak of a 1 per cent growth differential that lasts 10-25 years as small. If a country grows at 1 per cent below trend for 23 years, output will be roughly 25 per cent below trend at the end of the period, with massive cumulative effects.
That point is utterly meaningless if the causality works in the other direction, if the slow growth is causing the debt rather than the other way around. And another re-analysis of the R/R data shows that’s what’s happening. That analysis also was simple to do. As Matt Yglesias comments:
it’s striking that R&R didn’t even check this. I don’t begrudge any academic’s right to rush into publication with an interesting empirical finding based on the assembly of a novel and useful dataset. I don’t even begrudge them the right to keep their dataset private for a little while so they can internalize more of the benefits. But Reinhart and especially Rogoff have spent years now engaged in a high-profile political advocacy campaign grounded in a causal interpretation of their empirical work that both of them knew perfectly well was not in fact supported by their analysis.
Buying apples, selling oranges. And that’s the important point. The biggest reason R/R’s paper has been so badly misused in our political debate is that they have been out there misrepresenting their results. Senator Coburn described their testimony to 40 senators a few months before the debt-ceiling debacle in 2012. After listening to their initial testimony,
Senator Kent Conrad, D-N.D., the chairman of the Senate Budget Committee, then offered his own stern warning to the assembled senators. Turning around in his chair in the middle of the room, he explained to his colleagues that when our high debt burden causes our economy to slow by 1 point of GDP, as Reinhart and Rogoff estimate, that doesn’t slow our [economic growth] by 1 percent, but by 25 to 33 percent, because we are growing at only 3 or 4 percent per year.
Did either professor interrupt to say, “Wait, Senator, we’re not saying the debt causes a slowdown. Our data just shows a correlation that could be explained by slowdowns causing high debt.”? No.
Reinhart echoed Conrad’s point and explained that countries rarely pass the 90 percent debt-to-GDP tipping point precisely because it is dangerous to let that much debt accumulate.
Fraud. Fraud, fraud, fraud.
* A point I often make when numbers appear in the Sift: Correlation is not causation. Correlation just means that two things tend to go together; causation means that one causes the other. A very common fallacy is to display a graph showing that A and B go up (or down) together, and then say that A causes B.
My favorite way to demonstrate the fallacy: Birthdays are good for you; people who have a lot of birthdays tend to live long lives.