Progress measuring progress

Forums are quite commonplace these days, but really useful ones are rare. We tend to think of Davos or Porto Alegre, or perhaps even the OECD’s own annual public forum. We imagine headlinemaking topics, on climate change, security, society, telecommunications or business innovation. But a recent OECD world forum held in Palermo, Italy, was different than most, for it had as its billing the rather unspectacular subject of statistics.

Curiously, the Forum was a hit. For nearly four days in November 2004, despite sometimes complicated technical presentations, the great amphitheatre holding the first OECD World Forum on Statistics, Knowledge and Policy was packed. Like in all good forums, business people and civil society activists, government officials, policymakers and experts converged on the wonderful Teatro Massimo to interact, network and busily exchange arguments and ideas. The media watched closely and, according to our data (yes, more counting), thousands of people from all over the world followed the debates live from their desktops.

Heartening stuff for us statisticians, but why such broad interest? Quite simply, this forum was not about the environment, or social exclusion, or new technology. Rather, it was about all of these things, about the data that underlies our knowledge and hence our action. The point is that today the role of statistics goes well beyond the production of figures, and touches upon people’s everyday lives.

When governments set a new budget, when households decide which school their children should attend, when businesses invest, when NGOs launch campaigns: all these decisions are often based on some sort of statistical information, which we turn into knowledge and use to inform our choices.

We live in a data-rich world in which ordinary people have become familiar with notions like GDP, or inflation rates. There seems to be no shortage of surveys, on the cost of living, on corruption, economic freedom, or on the state of our hospitals. There are surveys on the environment, or even on how depressed or happy we feel. But people quickly see the limits of such surveys. Some of them are serious, others are there just to entertain us, but which ones? And they paint so many different pictures about us that we end up being confused.

On the other hand, people want to see statistics that capture the quality of their lives, and not just any one aspect. They want to measure progress based on a fuller picture of where they stand.

People inevitably turn to statisticians to sort out what is measurable and what is not. But given the stakes, everyone wants to be involved. Hence the interest in Palermo. And the overarching question people from all backgrounds were asking was: how can we measure our lives better?

True, our life expectancy is higher, we seem healthier and smarter, and many countries are richer. But are we really making progress? Even in developed countries, we sense that the progress we have made is uneven, and perhaps not sustainable or secure. How can we be sure? Without sound data, advice sounds rhetorical, and policy prescriptions ideological.

There are several ways to measure the overall situation of a country, but three attract particular attention. The first extends traditional economic accounts based on GDP, for instance, to capture the environment or social concerns. A second way is to develop composite indicators of wellbeing that combine detailed information into a single measure. And the third seeks to identify a certain number of key indicators covering economic, social and environmental domains, without deriving any particular single measure. Each of these approaches has strengths and weaknesses.

The first approach is quite an investment in terms of the amount of data to be collected and the resources needed to make them coherent. It encounters theoretical and practical difficulties in expressing some aggregates in monetary terms, such as environmental resources. And while it may be very powerful analytically, for instance, for simulating the economic, social and environmental effects of different policy measures, it is hard to see how such a complicated approach could easily win over governments, investors or the public.

Composite indicators seem more attractive from this perspective, as they can combine a wide range of indicators – even “apples and oranges” – into a single measure. Human development indices and quality of life indicators are produced this way. They lend themselves easily to rankings among countries, and are loved by journalists and readers alike. On the other hand, they can be very misleading, depending on the selection of indicators and weights used to aggregate the results. Their authors exercise their own tastes and judgements, yet do not always warn readers how individual components might distort the final results. So, if air quality gets a high weight, this may benefit the position of countries with low urban density.

An intermediate approach is the choice of sets of key indicators, based on a more holistic view of what matters most for development and progress in each country. It has the advantage of covering a wide range of topics – say, 20-30 or so – without the need of estimating individual weights. For instance, countries might focus on a list that covers economic growth, healthcare, air pollution, youth employment and new investment. The responsibility of making a comprehensive performance assessment lies with the users, according to their vision, expectations and culture. Such key indicators may not provide a single measure of progress and do not have the methodological soundness of some accounting frameworks. But they are gaining popularity because they provide users – the stakeholders – with a manageable amount of information whose selection reflects their choices, while providing a yardstick against which performance can be openly scrutinised. As David Walker, the US comptroller general, would put it, such indicators help people keep score.

The OECD is working, in one way or another, on all these approaches. We produce statistical measures of economic, social and environmental conditions in both countries and sub-regions. We undertake methodological research on accounting frameworks and composite indicators, and we hold international workshops, including on key indicators, with the Palermo forum being a prime example. Naturally, some critics wonder whether all of this work is nothing more than statistical overindulgence, with number crunchers just looking for yet new ways to skin the same cat. There is certainly a danger of statistical overload, and of according too much importance to certain figures. As some speakers at the forum showed, governments and other decision-makers can over-react to the power of numbers, too easily allowing rankings and targets to set policy priorities.

But what counts most is to know how to treat numbers, and to develop the knowledge we need to act on them. As European Central Bank chief, Jean-Claude Trichet, points out, good decisions also depend on good judgement. At the same time, there is so much we do not know, and such are the uncertainties of life, from climate change to market behaviour, that no one can afford to be complacent.

The forum in Palermo showed that statistics represent a fundamental tool in developing knowledge, so making better evidencebased decisions possible. Statisticians do not take the decisions, but they do an important job. As Thomas Jefferson said, “I know of no safe depository of the ultimate powers of the society but the people themselves; and if we think them not enlightened enough to exercise their control with wholesome discretion, the remedy is not to take it from them, but to inform their discretion by education".

©OECD Observer No 246/247, December 2004-January 2005 




Economic data

E-Newsletter

Stay up-to-date with the latest news from the OECD by signing up for our e-newsletter :

Twitter feed

Suscribe now

<b>Subscribe now!</b>

To receive your exclusive print editions delivered to you directly


Online edition
Previous editions

Don't miss

  • “Nizip” refugee camp visit
    July 2016: OECD Secretary-General Angel Gurría visits the “Nizip” refugee camp, situated between Gaziantep and the Turkish-Syrian border, accompanied by Turkey’s Deputy Prime Minister Mehmet Şimşek. The camp accommodates a small number of the 2.75 million Syrians currently registered in Turkey, mostly outside the camps. In his tour of the camp, Mr Gurría visits a school, speaks with refugees and gives a short interview.
  • OECD Observer i-Sheet Series: OECD Observer i-Sheets are smart contents pages on major issues and events. Use them to find current or recent articles, video, books and working papers. To browse on paper and read on line, or simply download.
  • Queen Maxima of the Netherlands gives a speech next to Mexico's President Enrique Pena Nieto (not pictured) during the International Forum of Financial Inclusion at the National Palace in Mexico City, Mexico June 21, 2016.
  • How sustainable is the ocean as a source of economic development? The Ocean Economy in 2030 examines the risks and uncertainties surrounding the future development of ocean industries, the innovations required in science and technology to support their progress, their potential contribution to green growth and some of the implications for ocean management.
  • OECD Environment Director Simon Upton presented a talk at Imperial College London on 21 April 2016. With the world awash in surplus oil and prices languishing around US$40 per barrel, how can governments step up efforts to transform the world’s energy systems in line with the Paris Agreement?
  • Happy 10th birthday to Twitter. This 2008 OECD Observer interview with Henry Copeland said you’d do well.
  • The OECD Gender Initiative examines existing barriers to gender equality in education, employment, and entrepreneurship. The gender portal monitors the progress made by governments to promote gender equality in both OECD and non-OECD countries and provides good practices based on analytical tools and reliable data.
  • Once migrants reach Europe, countries face integration challenge: OECD's Thomas Liebig speaks to NPR's Audie Cornish.

  • Message from the International Space Station to COP21

  • COP21 Will Get Agreement With Teeth: OECD Secretary-General Angel Gurría on Bloomberg

  • The carbon clock is ticking: OECD’s Gurría on CNBC

  • If we want to reach zero net emissions by the end of the century, we must align our policies for a low-carbon economy, put a price on carbon everywhere, spend less subsidising fossil fuels and invest more in clean energy. OECD at #COP21 – OECD statement for #COP21
  • They are green and local --It’s a new generation of entrepreneurs in Kenya with big dreams of sustainable energy and the drive to see their innovative technologies throughout Africa. blogs.worldbank.org
  • Pole to Paris Project
  • In order to face global warming, Asia needs at least $40 billion per year, derived from both the public and private sector. Read how to bridge the climate financing gap on the Asian Bank of Development's website.
  • How can cities fight climate change?
    Discover projects in Denmark, Canada, Australia, Japan and Mexico.
  • Climate: What's changed, what hasn't, what we can do about it.
    Lecture by OECD Secretary-General Angel Gurría, hosted by the London School of Economics and Aviva Investors in association with ClimateWise, London, UK, 3 July 2015.

  • Climate change: “We should not disagree when scientists tell us we have a window of opportunity–10-15 years–to turn this thing around” argues Senator Bernie Sanders.

  • In the long-run, the EU benefits from migration, says OECD Head of International Migration Division Jean-Christophe Dumont.
  • Is technological progress slowing down? Is it speeding up? At the OECD, we believe the research from our Future of ‪Productivity‬ project helps to resolve this paradox.
  • Is inequality bad for growth? That redistribution boosts economies is not established by the evidence says FT economics editor Chris Giles. Read more on www.ft.com.
  • Catherine Mann, OECD Chief Economist, explains on Bloomberg why "too much bank lending can slow economic growth".
  • Interested in a career in Paris at the OECD? The OECD is a major international organisation, with a mission to build better policies for better lives. With our hub based in one of the world's global cities and offices across continents, find out more at www.oecd.org/careers .

Most Popular Articles

Poll

What issue are you most concerned about in 2016?

Unemployment
Euro crisis
International conflict
Global warming
Other

OECD Insights Blog

NOTE: All signed articles in the OECD Observer express the opinions of the authors
and do not necessarily represent the official views of OECD member countries.

All rights reserved. OECD 2016