Progress measuring progress

Forums are quite commonplace these days, but really useful ones are rare. We tend to think of Davos or Porto Alegre, or perhaps even the OECD’s own annual public forum. We imagine headlinemaking topics, on climate change, security, society, telecommunications or business innovation. But a recent OECD world forum held in Palermo, Italy, was different than most, for it had as its billing the rather unspectacular subject of statistics.

Curiously, the Forum was a hit. For nearly four days in November 2004, despite sometimes complicated technical presentations, the great amphitheatre holding the first OECD World Forum on Statistics, Knowledge and Policy was packed. Like in all good forums, business people and civil society activists, government officials, policymakers and experts converged on the wonderful Teatro Massimo to interact, network and busily exchange arguments and ideas. The media watched closely and, according to our data (yes, more counting), thousands of people from all over the world followed the debates live from their desktops.

Heartening stuff for us statisticians, but why such broad interest? Quite simply, this forum was not about the environment, or social exclusion, or new technology. Rather, it was about all of these things, about the data that underlies our knowledge and hence our action. The point is that today the role of statistics goes well beyond the production of figures, and touches upon people’s everyday lives.

When governments set a new budget, when households decide which school their children should attend, when businesses invest, when NGOs launch campaigns: all these decisions are often based on some sort of statistical information, which we turn into knowledge and use to inform our choices.

We live in a data-rich world in which ordinary people have become familiar with notions like GDP, or inflation rates. There seems to be no shortage of surveys, on the cost of living, on corruption, economic freedom, or on the state of our hospitals. There are surveys on the environment, or even on how depressed or happy we feel. But people quickly see the limits of such surveys. Some of them are serious, others are there just to entertain us, but which ones? And they paint so many different pictures about us that we end up being confused.

On the other hand, people want to see statistics that capture the quality of their lives, and not just any one aspect. They want to measure progress based on a fuller picture of where they stand.

People inevitably turn to statisticians to sort out what is measurable and what is not. But given the stakes, everyone wants to be involved. Hence the interest in Palermo. And the overarching question people from all backgrounds were asking was: how can we measure our lives better?

True, our life expectancy is higher, we seem healthier and smarter, and many countries are richer. But are we really making progress? Even in developed countries, we sense that the progress we have made is uneven, and perhaps not sustainable or secure. How can we be sure? Without sound data, advice sounds rhetorical, and policy prescriptions ideological.

There are several ways to measure the overall situation of a country, but three attract particular attention. The first extends traditional economic accounts based on GDP, for instance, to capture the environment or social concerns. A second way is to develop composite indicators of wellbeing that combine detailed information into a single measure. And the third seeks to identify a certain number of key indicators covering economic, social and environmental domains, without deriving any particular single measure. Each of these approaches has strengths and weaknesses.

The first approach is quite an investment in terms of the amount of data to be collected and the resources needed to make them coherent. It encounters theoretical and practical difficulties in expressing some aggregates in monetary terms, such as environmental resources. And while it may be very powerful analytically, for instance, for simulating the economic, social and environmental effects of different policy measures, it is hard to see how such a complicated approach could easily win over governments, investors or the public.

Composite indicators seem more attractive from this perspective, as they can combine a wide range of indicators – even “apples and oranges” – into a single measure. Human development indices and quality of life indicators are produced this way. They lend themselves easily to rankings among countries, and are loved by journalists and readers alike. On the other hand, they can be very misleading, depending on the selection of indicators and weights used to aggregate the results. Their authors exercise their own tastes and judgements, yet do not always warn readers how individual components might distort the final results. So, if air quality gets a high weight, this may benefit the position of countries with low urban density.

An intermediate approach is the choice of sets of key indicators, based on a more holistic view of what matters most for development and progress in each country. It has the advantage of covering a wide range of topics – say, 20-30 or so – without the need of estimating individual weights. For instance, countries might focus on a list that covers economic growth, healthcare, air pollution, youth employment and new investment. The responsibility of making a comprehensive performance assessment lies with the users, according to their vision, expectations and culture. Such key indicators may not provide a single measure of progress and do not have the methodological soundness of some accounting frameworks. But they are gaining popularity because they provide users – the stakeholders – with a manageable amount of information whose selection reflects their choices, while providing a yardstick against which performance can be openly scrutinised. As David Walker, the US comptroller general, would put it, such indicators help people keep score.

The OECD is working, in one way or another, on all these approaches. We produce statistical measures of economic, social and environmental conditions in both countries and sub-regions. We undertake methodological research on accounting frameworks and composite indicators, and we hold international workshops, including on key indicators, with the Palermo forum being a prime example. Naturally, some critics wonder whether all of this work is nothing more than statistical overindulgence, with number crunchers just looking for yet new ways to skin the same cat. There is certainly a danger of statistical overload, and of according too much importance to certain figures. As some speakers at the forum showed, governments and other decision-makers can over-react to the power of numbers, too easily allowing rankings and targets to set policy priorities.

But what counts most is to know how to treat numbers, and to develop the knowledge we need to act on them. As European Central Bank chief, Jean-Claude Trichet, points out, good decisions also depend on good judgement. At the same time, there is so much we do not know, and such are the uncertainties of life, from climate change to market behaviour, that no one can afford to be complacent.

The forum in Palermo showed that statistics represent a fundamental tool in developing knowledge, so making better evidencebased decisions possible. Statisticians do not take the decisions, but they do an important job. As Thomas Jefferson said, “I know of no safe depository of the ultimate powers of the society but the people themselves; and if we think them not enlightened enough to exercise their control with wholesome discretion, the remedy is not to take it from them, but to inform their discretion by education".

©OECD Observer No 246/247, December 2004-January 2005 

Economic data

GDP growth: +0.6% Q1 2019 year-on-year
Consumer price inflation: 2.3% May 2019 annual
Trade: +0.4% exp, -1.2% imp, Q1 2019
Unemployment: 5.2% July 2019
Last update: 8 July 2019

OECD Observer Newsletter

Stay up-to-date with the latest news from the OECD by signing up for our e-newsletter :

Twitter feed

Subscribe now

<b>Subscribe now!</b>

To order your own paper editions,email

Online edition
Previous editions

Don't miss

  • MCM logo
  • The following communiqué and Chair’s statement were issued at the close of the OECD Council Meeting at Ministerial level, this year presided by the Slovak Republic.
  • Food production will suffer some of the most immediate and brutal effects of climate change, with some regions of the world suffering far more than others. Only through unhindered global trade can we ensure that high-quality, nutritious food reaches those who need it most, Angel Gurría, Secretary-General of the OECD, and José Graziano da Silva, Director-General of the United Nations Food and Agriculture Organization, write in their latest Project Syndicate article. Read the article here.
  • Globalisation will continue and get stronger, and how to harness it is the great challenge, says OECD Secretary-General Gurría on Bloomberg TV. Watch the interview here.
  • OECD Secretary-General Angel Gurría with UN Secretary-General António Guterres at the 73rd Session of the UN General Assembly, in New York City.
  • The new OECD Observer Crossword, with Myles Mellor. Try it online!
  • Listen to the "Robots are coming for our jobs" episode of The Guardian's "Chips with Everything podcast", in which The Guardian’s economics editor, Larry Elliott, and Jeremy Wyatt, a professor of robotics and artificial intelligence at the University of Birmingham, and Jordan Erica Webber, freelance journalist, discuss the findings of the new OECD report "Automation, skills use and training". Listen here.
  • Do we really know the difference between right and wrong? Alison Taylor of BSR and Susan Hawley of Corruption Watch tell us why it matters to play by the rules. Watch the recording of our Facebook live interview here.
  • Has public decision-making been hijacked by a privileged few? Watch the recording of our Facebook live interview with Stav Shaffir, MK (Zionist Union) Chair of the Knesset Committee on Transparency here.
  • Can a nudge help us make more ethical decisions? Watch the recording of our Facebook live interview with Saugatto Datta, managing director at ideas42 here.
  • The fight against tax evasion is gaining further momentum as Barbados, Côte d’Ivoire, Jamaica, Malaysia, Panama and Tunisia signed the BEPS Multilateral Convention on 24 January, bringing the total number of signatories to 78. The Convention strengthens existing tax treaties and reduces opportunities for tax avoidance by multinational enterprises.
  • Globalisation’s many benefits have been unequally shared, and public policy has struggled to keep up with a rapidly-shifting world. The OECD is working alongside governments and international organisations to help improve and harness the gains while tackling the root causes of inequality, and ensuring a level playing field globally. Please watch.
  • Checking out the job situation with the OECD scoreboard of labour market performances: do you want to know how your country compares with neighbours and competitors on income levels or employment?
  • Trade is an important point of focus in today’s international economy. This video presents facts and statistics from OECD’s most recent publications on this topic.
  • The OECD Gender Initiative examines existing barriers to gender equality in education, employment, and entrepreneurship. The gender portal monitors the progress made by governments to promote gender equality in both OECD and non-OECD countries and provides good practices based on analytical tools and reliable data.
  • Interested in a career in Paris at the OECD? The OECD is a major international organisation, with a mission to build better policies for better lives. With our hub based in one of the world's global cities and offices across continents, find out more at .
  • Visit the OECD Gender Data Portal. Selected indicators shedding light on gender inequalities in education, employment and entrepreneurship.

Most Popular Articles

OECD Insights Blog

NOTE: All signed articles in the OECD Observer express the opinions of the authors
and do not necessarily represent the official views of OECD member countries.

All rights reserved. OECD 2019