Quality data

The new OECD statistical information system

While the importance of quality data cannot be overestimated, quite often it is the handling and interpretation of the data by both users and suppliers that cause the real problems. The OECD is taking action to correct this by launching a new statistical information system.

Ironically in today’s information world, getting hold of proper data is not as easy as it should be. There may be several sources to choose from, in international organisations, government and companies. Too often, though, they generate inconsistent information, even for straightforward numbers like GDP growth. The Internet is awash with suppliers, and though it is the tried and trusted sources that researchers and policymakers turn to (and indeed pay for), no database can be taken for granted.

Even at the OECD, the quality and reliability of our data depend on handling and processing information that flows in and out of the organisation every day from around the world. Statisticians are not mere number librarians, but have a central role to play in ensuring that the entire process, from conception to dissemination, is robust and reliable, both for themselves and their clients.

Maintaining quality data in the face of heavy demands is an enormous challenge. There is defining and gathering the statistics; ensuring relevance, veracity and comparability; providing the right metadata such as definitions, sources and caveats; and then disseminating and updating, all in a fast-moving technological environment.

Information technology has naturally been a tremendous boon to this work. The OECD is taking advantage of the opportunities it presents, and a number of software tools for management and exchange of statistical information are being developed. For example, the OECD and UN Statistics Division have a shared responsibility for collection of annual foreign trade data, and when a new system goes into service in 2005, data will periodically be replicated from one site to the other, creating a consistent annual foreign trade database for both organisations.

The OECD’s latest venture is even more impressive: a state-of-the-art statistical information system is now being rolled out, with the aim of improving the efficiency of data collection and eliminating errors and inconsistencies in statistical outputs. Moreover, the new system will shorten publishing cycles, and should make it easier to locate and retrieve our statistical products online.

The architecture of the new statistical information system consists of three layers: a production layer for collection, validation, processing and management of statistical data and related metadata; a storage layer where validated statistics and related metadata are placed; and a dissemination layer for generating statistical publications and interactive electronic products. The three layers are supported by a workflow system that automates statistical and publication processes wherever possible, and tracks the steps involved.

The design makes innovative use of new information technology standards, such as Web Services (which enable applications running on diverse computer platforms to communicate with each other) and XML (eXtensible Markup Language, which facilitates the interchange of data among various applications). The use of these standards permits a "loose coupling" of components of the statistical information system. So if, for example, it was decided to redevelop the central data warehouse (OECD.Stat) in the middle of the system because of a change in the underlying database software, it would be possible to do this without having to change either the tools for preparing validated data for the data warehouse, or the tools used for publishing validated data from the warehouse. Furthermore, the use of Web Services and XML means the statistical information system is fully prepared for global data sharing initiatives, such as Statistical Data and Metadata eXchange (SDMX).

OECD.Stat is the central repository or “intelligence warehouse” where validated statistical data and metadata are stored. The OECD.Stat warehouse has been designed to preserve the decentralised nature of OECD directorates’ statistical activities, while making their data and metadata part of a coherent corporate system. It will in due course become the sole coherent source of statistical data and related metadata for the organisation’s statistical publications, whether in print or electronic form.

StatWorks is a tool for managing the organisation’s production databases, with functions including data collection, transformation, processing (e.g. aggregation), validation, and ultimately export of validated data to the OECD.Stat warehouse for dissemination. By providing a standard hosting environment for managing production databases, StatWorks is progressively replacing a multiplicity of software platforms presently in service. The validation process is one of the most important features of StatWorks, while its common toolset helps reduce training and support requirements.

MetaStore is the OECD’s corporate metadata facility, and is designed to improve the efficiency of metadata preparation, storage, access, management and incorporation with the OECD’s statistical products. MetaStore is designed to overcome problems of fragmented metadata located in numerous production databases. The implementation of MetaStore provides a common environment for managing this metadata, helping ensure coherence of definitions, eliminating duplication and resolving inconsistencies. Like OECD.Stat, MetaStore can accommodate any kind of metadata related to any level of detail of the corresponding statistics, such as recent updates, methodology, seasonal adjustments, or breaks in series.

Of course, the OECD is also a major publisher of statistical information, which is where PubStat comes in. Technology can be used to gather the statistical data and metadata for a publication, and dispatch it to different uses, whether print or online. And by limiting the risk of human intervention along the way, the margin of error is narrowed. PubStat reduces time-to-publish, not least because it reduces the time that statisticians spend dealing with dissemination and formatting issues. It provides the database manager with an interface for managing content and for generating an output file, combining statistical data and metadata with contemporary, eye-catching publication layout instructions. It is a veritable publishing robot.

This all sounds ideal, if you can find the data. But as everyone knows, locating statistics of interest can be a real challenge for users not already familiar with the structures of the parent production databases. That is why a state-of-the-art presentation tool (“browser”) and new interface are being developed for access to the OECD.Stat warehouse. The new browser will not only be easier to use, but also incorporate advanced search features and allow multiple combinations of data, such as combining environmental indicators with statistics on economic growth.

Putting it all together has been a challenge, and the new statistical information system has been developed following the best professional practices and according to well established IT standards. Each component is thoroughly tested before being put in service. The OECD intends to share its software developments, such as StatWorks and MetaStore and their design principles, with national and international organisations worldwide, particularly where management of decentralised statistical activities is involved. Indeed, sharing arrangements are already under way with FAO, ILO, UNCTAD and UNESCO. Such co-operation will not only minimise duplication of effort, but also, in the spirit of “open source” software, generate feedback so the system can continue to be improved for the benefit of all.

Lee Samuelson, OECD Information Technology and Network Services,and Lars Thygesen, OECD Statistics Directorate

©OECD Observer No 246/247, December 2004-January 2005




Economic data

GDP growth: +0.5% Q3 2018 year-on-year
Consumer price inflation: 2.9% Sept 2018 annual
Trade: +2.7% exp, +3.0% imp, Q4 2017
Unemployment: 5.2% Sept 2018
Last update: 22 Nov 2018

E-Newsletter

Stay up-to-date with the latest news from the OECD by signing up for our e-newsletter :

Twitter feed

Suscribe now

<b>Subscribe now!</b>

To receive your exclusive paper editions delivered to you directly


Online edition
Previous editions

Don't miss

  • Food production will suffer some of the most immediate and brutal effects of climate change, with some regions of the world suffering far more than others. Only through unhindered global trade can we ensure that high-quality, nutritious food reaches those who need it most, Angel Gurría, Secretary-General of the OECD, and José Graziano da Silva, Director-General of the United Nations Food and Agriculture Organization, write in their latest Project Syndicate article. Read the article here.
  • Globalisation will continue and get stronger, and how to harness it is the great challenge, says OECD Secretary-General Gurría on Bloomberg TV. Watch the interview here.
  • OECD Secretary-General Angel Gurría with UN Secretary-General António Guterres at the 73rd Session of the UN General Assembly, in New York City.
  • The new OECD Observer Crossword, with Myles Mellor. Try it online!
  • Watch the webcast of the final press conference of the OECD annual ministerial meeting 2018.
  • Listen to the "Robots are coming for our jobs" episode of The Guardian's "Chips with Everything podcast", in which The Guardian’s economics editor, Larry Elliott, and Jeremy Wyatt, a professor of robotics and artificial intelligence at the University of Birmingham, and Jordan Erica Webber, freelance journalist, discuss the findings of the new OECD report "Automation, skills use and training". Listen here.
  • Do we really know the difference between right and wrong? Alison Taylor of BSR and Susan Hawley of Corruption Watch tell us why it matters to play by the rules. Watch the recording of our Facebook live interview here.
  • Has public decision-making been hijacked by a privileged few? Watch the recording of our Facebook live interview with Stav Shaffir, MK (Zionist Union) Chair of the Knesset Committee on Transparency here.
  • Can a nudge help us make more ethical decisions? Watch the recording of our Facebook live interview with Saugatto Datta, managing director at ideas42 here.
  • The fight against tax evasion is gaining further momentum as Barbados, Côte d’Ivoire, Jamaica, Malaysia, Panama and Tunisia signed the BEPS Multilateral Convention on 24 January, bringing the total number of signatories to 78. The Convention strengthens existing tax treaties and reduces opportunities for tax avoidance by multinational enterprises.
  • Rousseau
  • Do you trust your government? The OECD’s How's life 2017 report finds that only 38% of people in OECD countries trust their government. How can we improve our old "Social contract?" Read more.
  • Globalisation’s many benefits have been unequally shared, and public policy has struggled to keep up with a rapidly-shifting world. The OECD is working alongside governments and international organisations to help improve and harness the gains while tackling the root causes of inequality, and ensuring a level playing field globally. Please watch.
  • Checking out the job situation with the OECD scoreboard of labour market performances: do you want to know how your country compares with neighbours and competitors on income levels or employment?
  • Trade is an important point of focus in today’s international economy. This video presents facts and statistics from OECD’s most recent publications on this topic.
  • The OECD Gender Initiative examines existing barriers to gender equality in education, employment, and entrepreneurship. The gender portal monitors the progress made by governments to promote gender equality in both OECD and non-OECD countries and provides good practices based on analytical tools and reliable data.
  • Interested in a career in Paris at the OECD? The OECD is a major international organisation, with a mission to build better policies for better lives. With our hub based in one of the world's global cities and offices across continents, find out more at www.oecd.org/careers .
  • Visit the OECD Gender Data Portal. Selected indicators shedding light on gender inequalities in education, employment and entrepreneurship.

Most Popular Articles

OECD Insights Blog

NOTE: All signed articles in the OECD Observer express the opinions of the authors
and do not necessarily represent the official views of OECD member countries.

All rights reserved. OECD 2018