We applied the methodology developed and used in previous study which indicating the important role of transparency in the gap of reporting timeliness in infectious disease [21]. While Systemic Rapid Assessment (SYSRA) is a framework includes External contexts as social-environmental factors and health-specific elements which echoing the element of national responsibilities required by IHR 2005, it was consulted to be the conceptual and analytical guidelines for the evaluation of health systems and infectious disease control programs [22, 23]. Therefore, we collected transparency data and measurements based on this framework for further analysis.
IHRMT (self-evaluation) and JEE (external evaluation)
IHRMT is a questionnaire to monitor progress in implementing the IHR of countries [5]. The questionnaire consists of 13 sections including 8 core capacities, points of entry and 4 ‘other hazards’ as identified and delineated by the WHO to match the obligations outlined in Annex 1 of the IHR. Eight core capacities mainly for infectious disease control include legislation, coordination, surveillance, response, preparedness, risk communication, human resources and laboratory. The 4 hazards include zoonosis, food safety, along with the chemical and radionuclear ones. Individual questions were grouped by components and indicators in the questionnaires including 256 total attributes.
The JEE is a data gathering instrument designed to evaluate a country’s capacities for health security, including all IHR core capacities across relevant sectors at a national level [24]. The tool has 19 technical areas that includes the core capacities identified by IHRMT. The JEE also includes capacities specially identified for health security, such as Antimicrobial Resistance, Biosafety and Biosecurity, Immunization, Emergency Response Operations, Linking Public Health and Security Authorities and Medical Countermeasures and Personnel Deployment. The JEE process involves a self-evaluation by the country, followed by an external assessment team visit, that then produces a full JEE report that includes scores for the Indicators, as well as identified priority actions.
Data collection
We obtained countries’ self-reported implementation percentages as scores from the WHO website on 4rd November 2018 [25]. There were 127 countries’ self-reported IHRMT (now advanced to be IHR Self-Assessment Annual Reporting Tool, SPAR) scores in 2016 and 163 countries’ self-reported IHRMT scores in 2017 available and used in the study. Seventy-four countries’ published JEE reports were further collected on 27th November, 2018 and used for analysis [26].
The average score of 8 core capacities was further calculated to represent overall national capacity regarding infectious disease control.
Measurements
Civil liberties scores from the Freedom House were collected as indicators of transparency for each country. The Freedom House is an independent nongovernmental organisation that dedicating to the expansion of democracy and freedom around the world [27]. This group annually evaluates the political rights and civil liberties of each country. In our study, we used only civil liberties as an index of transparency. Civil liberties which reviewed by a 15 questions checklist included 4 key areas: freedom of expression and belief (4 questions), associational and organisational rights (3 questions), rule of law (4 questions) and personal autonomy and individual rights (4 questions). The total number of points on the civil liberties checklists will further be transformed into a rating scale ranged from 1 to 7. Score 1 represents the highest degree of freedom and 7 represents the lowest degree. The details of the method are described in the methodology section of the Freedom House website [27]. We collected the civil liberties scores of 2016 and further divided the analysed countries into free, partly free and not free countries according to these scores. Countries with civil liberties scores of 1 and 2 were designated as free countries, countries with scores of 3 to 5 were considered partly free countries, and countries with scores as 6 and 7 were not free countries.
Based on the framework of SYSRA toolkit, we further searched the Human Development Index (HDI) from the United Nations Development Program (UNDP) and information regarding the density of physician and nurses from WHO o represent the general health capacity of the country [28, 29].
According to the definition, Human development encompassed three dimensions: life expectancy at birth which indicating population health and longevity; adult literacy rate which indicating the knowledge and education level and the gross domestic product per capita indicating the purchasing power parity. With indicators mainly collected from official statistics, the human development index was calculated as a simple average of the dimension indices ranging between 0 and 1, with 1 representing the highest degree of human development and 0 the lowest. The details of methods are described in the Technical Notes section of the report [30]. We used the human development index of 2016 to represent the human development status of each country in that year. In addition, the categories used by the UN, i.e., very high, high, medium and low development countries were also used in the study.
Information of each country’s density of physicians and nurses was collected from WHO websites [31]. Then the sum of these two scores was calculated and used as the index of the health workforce in the study. We then categorized countries into high, middle or low health workforce countries based on the sum of the density of physicians and nurses in each country. Countries with upper tertile scores of health workforce density were defined as countries with high health workforce. Countries with the middle and lower tertile scores of health workforce density were defined as middle and low health workforce countries, respectively.
Analysis
The response for IHR from countries comprises the percentage of implementation ranging from 0 to 100. The JEE is scored on a scale from 1 to 5 to represent the level of a countries’ capacity to meet an indicator of health security, with 5 being the highest level of capacity. In order to make the IHR and JEE scores comparable, we re-scored the IHR results by dividing the scores by 20 to condense the scores into the scale of 5. We then calculated the difference of average score of each item between IHR self-evaluation and JEE to represent the gap between the different evaluation approaches. And the gap is further divided into 2 groups (less gap v.s. big gap) with average as cutting point. Similarly, the score gap between IHR 2016 and IHR 2017 is divided into 3 groups (negative gap, no gap, positive gap) for further analysis. The negative gap groups refers to countries with lower IHR 2017 scores than IHR 2016.
Pair-t test is used to compare the score of each item between IHR 2016 and IHR 2017, IHR 2016 and JEE, and IHR 2017 and JEE as it represent countries’ original self-judgment of their capacity without external interference. Chi-square test was then applied to compare countries’ HDI, civil liberty, health workforce between the gap group of IHR 2016 and IHR 2017, IHR 2016 and JEE, IHR 2017 and JEE. And we further compare countries’ HDI, civil liberty and health workforce between the gap group of countries’ first IHR self-evaluated score and JEE scores. Logistic regression was then applied to evaluate the association between countries’ HDI, CL and the gap between countries’ first IHR and JEE. Though HDI, CL and HWD were all significantly different between the gap groups of countries’ first IHR and JEE, we included only HDI and Cl in the regression analysis due to the fact that HDI and HWD was significantly correlated by Person correlation coefficient test.
All analysis was performed using the software SPSS, Version 18.0.