By Simon Johnson
One of the most basic questions in economics is: Which countries are rich and which are relatively poor? Or, if you prefer a highly relevant question for today’s global situation, who recovers faster and sustains higher growth?
The simplest answer, of course, would be just to compare incomes – i.e., which country’s residents earn the most money, on average, at a point in time and how does that change over time?
But prices differ dramatically across countries, so $1,000 in the United States will generally buy fewer goods and services than would the same $1,000 in Guinea-Bissau (although this immediately raises issues regarding consumer’s preferences, the availability of goods, and the quality of goods in very different places.)
The standard approach developed by economists and statisticians, working with great care and attention to detail on a project over the past 40 years known as the “Penn World Tables”, is to calculate a set of “international prices” for goods – and then to use these to calculate measures of output and income in “purchasing power parity terms.” For countries with lower market prices for goods and services, this will increase their measured income relative to countries with higher market prices (with Gross Domestic Product, GDP, per capita being the standard precise definition, but components and variations are also calculated along the way).
Some of the limitations inherent in the Penn Tables are well known. But it turns out there are other, quite serious issues, that should have a big effect on how we handle these data – and how doubtful we are when anyone claims that a particular country has grown fast or slow relative to other countries.
The Penn Tables are based on collecting detailed price information – what it actually costs to buy all kinds of things in different places. But the basic problem is that the people running the Tables do not have access to such data for all years and all countries – so they have to make a number of moderately heroic assumptions.
In “Is Newer Better?”, we show that a particular technical issue – the extrapolation of estimated price levels backwards and forwards in time – has a big impact on estimated GDP. This in turn changes, dramatically in some cases, the calculated growth rates for particular countries; and these changes can be huge for smaller countries with less good data, particularly when the year in question is quite far from the moment when prices were actually “benchmarked” though direct observation.
Just to illustrate our point, in Table 1 we show that the ranking of growth rates – e.g., top 10 and worst 10 countries, in terms of growth performance – within Africa, from 1975 to 1999, is completely different if you use Penn World Tables version 6.1 or if you use version 6.2. Just speaking for ourselves, we were quite shocked by these differences – and consequently spent a long time digging through the details (see the appendices of the paper for much more than you wanted to know about how this kind of sausage is made). We’ve also tried to figure out exactly how much these issues matter both for how people have studied growth in the past (to do this, we replicated and checked the robustness of 13 influential and indicative papers), and for how to think about (and measure) economic success and failure moving forward.
Our bottom line is: while the Penn Tables are reasonably reliable for comparing changes in income level over long periods of time (e.g., 30-40 years), they are much less appealing – and results based on them will generally not be robust – as a source for annual data. You should regard claims based on such annual data with a great deal of skepticism.
We also suggest there is a different and – for some purposes – better way to use the information in the Tables (see Section 6 of our paper). In essence, we suggest combining estimated GDP levels directly from the Tables, rather than using the standard (and problematic) extrapolation method.
Looking at annual growth rates from national statistics is fine – or at least raises different issues – for thinking about short-term growth dynamics (i.e., who is in crisis, who is recovering, who may be overheating). But for considering longer-run comparisons, say over 5-10 years or longer, you unfortunately cannot avoid worrying about comparable prices and some sort of purchasing power adjustment.
Whether or not you like our specific proposal, the main takeaway point is the same: do not rely on just one growth series. Check that your claims (or anyone else’s) hold across different versions of the Penn World Tables, and – if you are focused on annual growth rates – look also at estimates from the World Bank’s World Development Indicators.
If you are interested in these issues more broadly, see the papers presented at the “Measuring and Analyzing Economic Development” conference at the University of Chicago today.