Table 3 Highlights of selected health systems performance assessment frameworks

Australia National Health Performance Framework NHPF

• Work on PAF since the 90s

• The Lalonde model, appreciating both the healthcare & non-healthcare determinants of health

• Healthcare intended to be universally accessible

• Purpose: provide structure for reporting at national level & for developing PI sets for lower levels

• Rationalized and converged previous efforts at PA including indicator definitions, data processes, and local needs

• Present information in performance reports and HCAs

• Adapted from CHIRII

• Shared responsibility by federal & state governments for funding, regulation, & provision of services

• Dimensions (2^nd edition of NHPF): Effectiveness, responsiveness, accessibility, safety, continuity, efficiency, & sustainability

• Led by national & state ministers & using technical experts

• Dimensions: health status & outcomes, determinants of health, HS performance

• National & international comparison

• Equity as key concern

• Linkage with generic national bodies responsible for funding & PA

• Accreditation & professionalism

• NHCAs outline goals & HS roles & responsibilities for government bodies

• Indicators emphasize: national standards, worthiness, relevancy, validity, reliability, priority (minority) groups, user understanding

• NHPF developed in 2001 & reviewed in 2009

• Accountability & consumer & participation

• Has been in use for more than 10 years – with review in 2009;

• Involving a number of organizations: ACSQHC, COAG Reform Council, NHPAC, NHPC,, NICS, National HCAs

• Quality of care initiatives

• Epidemiological analysis linking inputs, processes, outputs, & outcomes

• Learning process with adjustment of dimensions, indicators, & reporting given current priorities, data availability, & possibility of interpretation

• Financial incentives for building capacity for quality & safety

Canadian Health Indicator Framework CHIF

• Initiated in 1998, endorsed by First Minister’s Meeting in 2000

• Lalonde model – appreciating healthcare and non-healthcare determinants of health

• Federal, provincial, & territorial levels roles & responsibilities

• To provide governments, providers, & public with reliable, comparable data across entities & assist in its use & interpretation

• Integrated network of HIS initiatives & structures, across country & levels including CIHI, SC, HC, CCHSA, CMA, AIM

• Biennial National Report

• Has been in use, evolving over more than a decade

• Public (mainly) & private funding

• Domains: acceptability, accessibility, appropriateness, competence, continuity, effectiveness, efficiency, safety

• Built on previous work by CIHI and CCHSA

• Defined up to 70 indicators

• Various providers

• Dimensions: health status, non-medical determinants, HS performance, community, & HS characteristics

• Provincial & regional governments link to plans & targets

• Informed the development of frameworks for the OECD, Australia, & Netherlands

• periodical pan-Canadian surveys for consumer opinion

• Wide consultation at national, regional and local levels;

• Minority populations with equity concerns

• Extensive use of evidence

• Marked financial & logistical investment over the last decade through CHIRII

• Benchmarking, CQI, Certification/Accreditation with professional bodies

• Change in indicators given data availability & interest

• Accountability, through making Information available to public;

• Learning, innovation, sharing best practices

• National Consensus Conferences on Indicators

Ghana Holistic Assessment of Health System

• Developed by the MoH and discussed with sector stakeholders, first time at the April 2009 Health Summit

• Health in center of national development agenda

• The assessment relates to the Health Sector PoW & the GPRS, guided by National Health Policy & MDGs

• Provide balanced and transparent assessment of sector performance indicating factors that may have influenced performance and suggest corrective measures

• Carried out by MoH & stakeholders & external reviewers

• Presented in briefs and reports discussed at national and regional forums

• Has been used for 4 years, to be adjusted with development of new PoW

• Data mostly from HMIS, surveys, and KIIs

• Goals – child survival & RH, decreasing burden of disease, & health services availability & use

• Dashboard approach, with 3-step process: assessment of indicators & milestones, assessment against goals & targets, & assessment of whole sector

• Receives information from districts, regions, agencies, & MoH

• Uses 22 out of 34 PoW indicators

• Thematic areas: healthy lifestyle & environment, provision of health, RH and nutrition services, HS capacity development & governance & financing

• Marked challenges in data availability and quality – sanctions proposed for those who do not submit data as required

• Prizes proposed for good performers

• High donor contribution to sector including through the Multi-donor Budget Support, MDBS

• Decentralization, with geographical equity concerns

Netherlands Dutch National Health System Performance Framework

• Consultative process between MoH & RIVM, & researchers over period 2002–2005

• Lalonde model for health determinants & Balanced Score Card (BSC) model of HSPA

• Transition from budget-driven healthcare system to regulated market

• Focus on technical healthcare quality, keeping other dimensions in sight

• Close working relationship between MoH & RIVM & researchers for ownership, & evidence base

• To provide evidence to make appropriate policy decisions

• Adapted from experiences in Canada (Lalonde model); and UK, US and Dutch healthcare organizations (BSC model)

• Not really designed to link information with management strategy

• Used evidence in form of frameworks from elsewhere, consideration of roles of MoH & other stakeholders, & existing information infrastructure

• Interface of Lalonde model & BSC is the consumer, relating population health & health management

• Emphasis on transparency & results oriented management

• Linked existing databases; created new cost-effective sources of data as required

• BSC - consumer, financial, internal business processes & innovative perspectives

• Adapted in Ontario and & for OECD’s HCQI Project

• BSC model adapted to a non-corporate, market-oriented entity

• Indicators selected in line with core questions posed on each perspective

• Compares healthcare performance with healthcare needs

South Africa District Health Barometer SA DHB

• Developed by the Health Systems Trust (HST), a non-governmental organization in consultation with DoH

• Equitable access to good healthcare as a major goal of the health system

• Decentralized, with bulk of primary health care services funded by government

• To monitor progress & support improvement of equitable provision of PHC

• Housed by HST a private entity with research & HSPA skills, working in close consultation with DoH

• Annual reports with tables, graphs and maps comparing all districts and within metro and rural districts;

• Has been in place with annual publications since 2005

• Adjustments made with improving data availability and quality and perceived needs for information

• Post-apartheid inequality in access to healthcare

• Equity analysis,

• Research and consultation with experts

• Use of evidence

• Information to policy makers and managers at national, provincial & district levels &public domain including academic/research institutions

• Indicators: socioeconomic, input, process, output, outcome & impact, related to MDGs

• Uses secondary data from various government institutions

• Geographical equity a major issue

• Poor health information systems and quality of data cited

• For comparison of all provinces & districts and within the categories of rural and metropolitan districts;

• Equity as a major focus;

• Trends studied

World Health Organization Health System Performance Assessment Framework

• Developed by WHO technocrats with wide stakeholder involvement only after the World Health Assembly of 2000 and marked criticism

• WHO introduced a number of concepts about a HS including health actions, boundaries, goals, functions and building blocks

• Intended as a l tool for use by all member states and therefore supposed to be generic and usable for assessment of and in widely varying contexts across the globe;

• For the purpose of helping member states to measure own performance, understand factors behind this and improve response;

• Global and national support for HSPA including establishment of EHSPI

• Presents information of member states in the World Health Report in league tables and plots;

• Has been in place since 2000 with substantial consultations following its launch; some adjustments have been made including dropping the composite goal performance index and elaboration of specific methodologies;

• Utilise DALYs and DALEs as measures of overall population health;

• Development of tools and approaches for data collection and analysis

• Extensive use of evidence

• Computation of indicator of composite goal performance in 2000.

• Main (extrinsic) Goals indicated as: improving population health, responsiveness, & fair financial contribution

• Assessment of 5 components of the HS using a number of indicators: population health level and distribution; responsiveness level and distribution; distribution of financial burden;

• Use of WHO regional groupings, research institutions and international organizations for consultation;

• Relates DALES to health systems’ potential given country/health system resources.

• Has been adapted and used for subnational assessments and also adapted for use by Health Systems 20/20 in several countries.

• Benchmarking and competition

• Public reporting & accountability

• Highlighting stewardship as important for system design, performance assessment, priority setting, inter-sectoral advocacy, rule setting, and consumer advocacy