Measuring up: performance indicators for better healthcare

Page 18 

David Rooney

Industrial countries spend a large proportion of GDP on healthcare. But how can they be sure they are getting value for money? One way is to use performance indicators, but these can have their drawbacks, as well as their advantages.

There is no doubt vaccination works, yet 20,000 people die every year in developed countries from vaccine-preventable diseases, according to WHO estimates. Numerous studies over the years have found no evidence that antibiotics help treat the common cold, yet 40-60% of doctors still prescribe them. These are just two examples of the gap between knowledge and performance in our healthcare systems.

Why does this gap exist? A key problem is the sheer weight of new knowledge. In the mid-1960s, about 100 articles from randomised clinical trials were published. By the 1990s, approximately 10 000 such articles were being published every year and nearly half (49%) of all the extant medical literature has been published in the past five years. The result is an estimated 15-20 year time lag between identifying more efficient treatment and incorporating it into routine practice. The gap applies across the board, from complicated clinical conditions to routine medical problems, such as the common cold.

Other key problems include the complexity of healthcare systems, the increasing demands fuelled by patient expectation, ageing and new technologies, as well as the reality of constraints and poor distribution of resources. Improving the performance of healthcare systems is becoming a priority in many countries as a result of economic pressure to optimise health spending, as well as increasing evidence of the scale of deficiencies in quality of care.

The scientific evidence is accompanied by widespread concern about eroding performance. In a 1998 survey of five countries, more than half the doctors questioned in Canada (59%), the United States (57%) and New Zealand (53%) reported that their ability to provide quality care had worsened over the past five years. Some 46% of UK physicians and 38% of Australians told the same story. Only a quarter or less of the physicians surveyed reported that their ability to provide high quality care had improved over the period. A similar survey of nurses in Canada, Germany, Scotland, England and the US in 1998-99 found that between 17% and 44% of them believed quality had deteriorated in the past year.

These findings cannot be dismissed as simply the complaints of demoralised clinicians, since patients respond similarly. In the 1998 survey, the public indicated overwhelmingly that the health systems in their country required fundamental change.

Consensus is emerging on the principal areas to cover in performance measurement: namely, effectiveness, efficiency, responsiveness and equity. But once you have defined your terms, how can performance measurement actually improve quality? There are numerous possible methods, but scant evidence available to provide a basis for selecting one. Factors such as underlying values, financing and organisational arrangements come into play.

The choice of method will also depend on whose behaviour you are trying to change: providers, professional bodies, citizens or managers. Identifying a ¡§best method¡¨ may not be realistic, but being aware of the possible approaches, their strengths and limitations, and the experience of countries that have tried them, can help in making a choice.

Performance indicators are employed for four basic functions: facilitating accountability; monitoring healthcare systems and services as a regulatory responsibility; modifying the behaviour of professionals and organisations at both a macro (population) and micro (patient) level; and forming policy initiatives.

Public demand for accountability is rife and nations are responding in their own ways. OECD countries generally apply three models of accountability in healthcare ¡V professional, economic and political. They use various combinations of these, all of them relying on performance indicators to some degree.

Professional accountability, dominant in most health systems historically, views the physician as the key to controlling quality and uses certification, accreditation, licensing and litigation as instruments for enforcement. But the professional model of accountability is increasingly regarded as insufficient unless accompanied by one of the other two.

The economic model, of which the US is the clearest example, is based on the idea that the competitive market can be used to enforce accountability. Health plans can influence physicians¡¦ choice of treatment by declining to fund some practices or encouraging others. And accountability through public reporting is believed to have resulted in improved performance in certain areas. For example, the rates of beta-blocker prescribing following heart attacks rose from 62% in 1996 to 85% in 1999 after standardised reporting was introduced. The political model meanwhile views the citizen as receiving a public good, so the government¡¦s role is to act as an agent of change on behalf of the public.

Objective measures of performance are increasingly used at several levels. They can dramatically influence policy, for instance. UK Prime Minister Tony Blair¡¦s decision to invest significant new resources in the National Health Service (NHS) was influenced, in part, by data showing his country to be spending a lower proportion of GDP on health than most northern European countries. And the US was influenced to train more general practitioners when data showed that it trains a higher proportion of specialists than most European countries.

Importantly, performance indicators can help to make policy priorities explicit, for example by defining national priorities and then identifying specific performance targets within those priorities. Australia has had a system of identified national health priorities since 1996; namely asthma, depression, diabetes, cardiovascular and injury, with reports describing the best available data in most priority areas.

The UK¡¦s NHS, after finding that outstanding claims for alleged clinical negligence in its hospitals have reached $5.6 billion, has set targets for reducing serious harm, such as a 40% reduction in prescribing errors by 2005.

Assisting healthcare professionals in practicing evidence-based medicine is a key objective for improving quality. Performance indicators, embedded in clinical guidelines and peer reviews, are among the most common approaches aimed at bridging the knowledge gap, but have limited effectiveness when used alone to change physician behaviour.

Still, scepticism remains, as was demonstrated in a recent survey of more than 100 doctors in England. Some 85% said they would ignore the guidance of the newly established National Institute for Clinical Evidence, responsible for developing the evidence basis for guidelines and protocols, if they thought it was wrong. There are promising signs of potential in incorporating practice guidelines into computer support systems. Indeed, analysis shows that the use of computer-generated prompts can improve preventive services and how drugs are prescribed.

Experience in the US suggests that institutions too can use performance data to improve care processes, to identify poor performers, and to respond to patients¡¦ preferences or complaints. One often-cited example is the decision by the New York State Health Department to publish mortality rates after coronary artery bypass graft (CABG). In the first five years of the programme (1987-92), the mortality rate in New York declined twice as fast as the national average. But in the face of this dramatic success, so far there has been little effort to achieve similar results with this method either for CABG in other states or for other types of treatment in New York. What¡¦s more, nobody is sure why this approach has not been more widely adopted ¡V is this a problem of professional resistance, technical capability or resources?

A US movement sometimes referred to as informed consumerism has been relied upon as a means of using competitive market forces and individual choice to drive up performance. But an article in JAMA in 1999 found that patients/consumers have made very little use of performance data when making healthcare decisions, continuing largely to rely on word-of-mouth. One reason may be that most of the published performance data was very specialised and not useful for most patients, such as CABG mortality rates.

However, performance data designed specifically for consumers, such as information about success rates for common procedures and treatments, may be the most efficient way of delivering data to the public.

Patient empowerment is not just politically correct, it can cut costs and improve quality. There is now a body of literature showing that better-informed patients have better outcomes, choose less risky procedures and avoid equivocal treatments. This should increase our confidence that patients can not only make constructive use of performance data designed for them, but can also be reliable informants for performance assessment.

Performance data designed specifically for consumers, such as information about success rates for common procedures and treatments, may be the most efficient way of delivering data to the public.

Another issue is cost. Since employers are the dominant buyers of healthcare in the US, they theoretically have both the motive and clout to buy health services or insurance coverage based on performance. But in reality, price trumps all other performance data. Two studies, which between them looked at more than 1 500 employers across the US, concluded that their use of performance data was limited, again at least partly because the data was not packaged to be useful to them.

Performance measurement and reporting have clearly made dramatic advances in the past decade but more needs to be done so that fair and accurate assessments can be predictably and credibly provided. Performance indicators, whether length of waiting lists or choice of treatment for a particular ailment, may light the way forward, but face significant challenges.

The state of the art of performance measurement is embryonic, with insufficient understanding of exactly how data can help achieve change, under what circumstances and with what consequences.

New resources will be required to build capacity, most notably in informatics and information technology. Most importantly, performance data will have to be made even more useful to target audiences with clear, measurable and achievable goals for improvement.


• Blendon R.J., Schoen C., Donelan K., Osborn R., DesRoches C.M., Scoles K., Davis K., Binns K., Zapert K., "Physicians' View on Quality of Care: A Five-Country Comparison", Health Affairs, May-June, 2001.

 Emanuel E.J., Emanuel L.L., "What is Accountability in Health Care?", Annals of Internal Medicine, January, 1996.

 Marshall M., Shekelle P., Leatherman S., Brook R., "What Do We Expect to Gain from the Public Release of Performance Data? A Review of the Evidence", Journal American Medical Association 2000, Vol. 283, No.14.

©OECD Observer No 229, November 2001 

Economic data


Stay up-to-date with the latest news from the OECD by signing up for our e-newsletter :

Twitter feed

Suscribe now

<b>Subscribe now!</b>

To receive your exclusive print editions delivered to you directly

Online edition
Previous editions

Don't miss

  • Africa's cities at the forefront of progress: Africa is urbanising at a historically rapid pace coupled with an unprecedented demographic boom. By 2050, about 56% of Africans are expected to live in cities. This poses major policy challenges, but make no mistake: Africa’s cities and towns are engines of progress that, if harnessed correctly, can fuel the entire continent’s sustainable development.
  • “Nizip” refugee camp visit
    July 2016: OECD Secretary-General Angel Gurría visits the “Nizip” refugee camp, situated between Gaziantep and the Turkish-Syrian border, accompanied by Turkey’s Deputy Prime Minister Mehmet Şimşek. The camp accommodates a small number of the 2.75 million Syrians currently registered in Turkey, mostly outside the camps. In his tour of the camp, Mr Gurría visits a school, speaks with refugees and gives a short interview.
  • OECD Observer i-Sheet Series: OECD Observer i-Sheets are smart contents pages on major issues and events. Use them to find current or recent articles, video, books and working papers. To browse on paper and read on line, or simply download.
  • Queen Maxima of the Netherlands gives a speech next to Mexico's President Enrique Pena Nieto (not pictured) during the International Forum of Financial Inclusion at the National Palace in Mexico City, Mexico June 21, 2016.
  • How sustainable is the ocean as a source of economic development? The Ocean Economy in 2030 examines the risks and uncertainties surrounding the future development of ocean industries, the innovations required in science and technology to support their progress, their potential contribution to green growth and some of the implications for ocean management.
  • OECD Environment Director Simon Upton presented a talk at Imperial College London on 21 April 2016. With the world awash in surplus oil and prices languishing around US$40 per barrel, how can governments step up efforts to transform the world’s energy systems in line with the Paris Agreement?
  • Happy 10th birthday to Twitter. This 2008 OECD Observer interview with Henry Copeland said you’d do well.
  • The OECD Gender Initiative examines existing barriers to gender equality in education, employment, and entrepreneurship. The gender portal monitors the progress made by governments to promote gender equality in both OECD and non-OECD countries and provides good practices based on analytical tools and reliable data.
  • Once migrants reach Europe, countries face integration challenge: OECD's Thomas Liebig speaks to NPR's Audie Cornish.

  • Message from the International Space Station to COP21

  • The carbon clock is ticking: OECD’s Gurría on CNBC

  • If we want to reach zero net emissions by the end of the century, we must align our policies for a low-carbon economy, put a price on carbon everywhere, spend less subsidising fossil fuels and invest more in clean energy. OECD at #COP21 – OECD statement for #COP21
  • They are green and local --It’s a new generation of entrepreneurs in Kenya with big dreams of sustainable energy and the drive to see their innovative technologies throughout Africa.
  • Pole to Paris Project
  • In order to face global warming, Asia needs at least $40 billion per year, derived from both the public and private sector. Read how to bridge the climate financing gap on the Asian Bank of Development's website.
  • How can cities fight climate change?
    Discover projects in Denmark, Canada, Australia, Japan and Mexico.
  • Climate: What's changed, what hasn't, what we can do about it.
    Lecture by OECD Secretary-General Angel Gurría, hosted by the London School of Economics and Aviva Investors in association with ClimateWise, London, UK, 3 July 2015.
  • Is technological progress slowing down? Is it speeding up? At the OECD, we believe the research from our Future of ‪Productivity‬ project helps to resolve this paradox.
  • Is inequality bad for growth? That redistribution boosts economies is not established by the evidence says FT economics editor Chris Giles. Read more on
  • Interested in a career in Paris at the OECD? The OECD is a major international organisation, with a mission to build better policies for better lives. With our hub based in one of the world's global cities and offices across continents, find out more at .

Most Popular Articles


What issue are you most concerned about in 2016?

Euro crisis
International conflict
Global warming

OECD Insights Blog

NOTE: All signed articles in the OECD Observer express the opinions of the authors
and do not necessarily represent the official views of OECD member countries.

All rights reserved. OECD 2016