Categories of public health policies
We identify and code policies falling into several categories: border closures (international and domestic), school closures, social gathering and social distancing limitations, home-bound policies (curfew, stay-at-home, lockdown), medical isolation policies (self-isolation and mandatory quarantine), closure/restriction of businesses and services (closure of nonessential businesses, restaurants, entertainment venues, government offices, public transportation, work from home requirements), the introduction of the state of emergency, and requiring mandatory personal protection equipment and physical distancing.
The border closures categories were of paramount importance in responding during the ‘alert period’ (the very first weeks) of the pandemic and remained important in preventing new infections due to inter-territorial externalities in public health response. The relative weight of border closures in the PPI is the main difference between Methods 1 and 2.
Limits/closures on schools, large venues, restaurants, non-essential businesses, public transportation closures, government offices, and places of employment are policies aimed at reducing infections from community spread. Each of these is also graduated based on the strictness of the restriction implemented. We code them separately because different governments chose to close some but not others and/or implemented closures of each at different times.
Restrictions on mobility for the healthy population (personal mobility, social gatherings) are similarly aimed to reduce infections from community spread. Again, these categories are graduated based on the stringency of the measures put in place. These measures are distinct from the previous set because they represent an infringement on individual liberties. They do, however, function similarly from an epidemiological perspective to limit community spread.
Publicly mandated hygiene practices-mandatory PPE and quarantine for suspected exposures – are intended to minimize/prevent community spread. This works by severely limiting the ability of the virus to spread through droplets expelled by individuals who are infected but might not know in the case of PPE, and by physically isolating the carrier from the rest of the population in the case of quarantine. Note that these do not include changing requirements imposed on medical facilities during the pandemic since we only measure public-health policies and not medical practices.
Lastly, state of emergency is a category that captures the commitment of the government to handling the crisis through policy. It enables the adoption of some policies that may violate civil liberties and enables enforcement of many of the hygiene practices and business restrictions.
We differentiate between the policies made by national and subnational governments. For each category of public health policies, subnational unit, and day, we produce three values: the stringency of policies made in that category by the subnational government, the stringency of policies made in that category by the national government, and the total stringency of policies in that category, computed as the highest between the national and subnational values. Often policies announced by different level governments were duplicates of one another at a given time. For example, there may be a nationally-mandated self-isolation period and a self-isolation/quarantine requirement for a region based on the same or similar criteria. In this case, the national policy would be counted towards the value of the National PPI and the subnational policy towards the Regional PPI. The policy would take the highest value out of the two for the value that goes into calculating the Total PPI (see Appendix A3).
Note that there is variation on stringency within the policy categories, with some policy adoptions being more stringent than others (i.e. self-isolation versus lockdowns, partial school closings versus full school closings). To this end, we weighed more stringent policies in each category in the index more heavily. These values are normalized to range between 0 and 1. Data are collected from national and sub-national news sources, government resources, and press releases. The auxiliary file “changes_regions_m1.csv” contains a column with the source links for each recorded change in the policy variables. The changes reported in this file are what gives rise to the running values of the policy variables and their aggregates in the main files.
Because all categories that we code are public mandates for mitigation, sources include official documents, news reports, as well as the announcements of relevant policies made by the officials. To match the content of policies with specific policy categories and maintain comparability across countries, the coders were trained to use the meaning rather than the language of the policy. Appendix 2 describes the meaning of policies as coded. Global regions curators regularly met to discuss the challenges of interpretation that their teams encountered in specific countries.
We calculate the Public Health Protective Policy Index (PPI) based on coded public health policy responses. The dataset contains three types of PPI scores: regional PPI for each subnational unit on each day; national PPI for each subnational unit on each day, based on nationally issued policies; and Total PPI for each subnational unit on each day. The Total PPI reflects the strictest between the national and subnational policies adopted within each category for that unit for that day. The indices are scaled to range between 0 and 1.
We seek to weigh public health policies according to their expected efficacy to construct the index of these policies. Under normal circumstances, in recurrent crises, previous outbreaks involving similar pathogens have enabled evaluation of the effectiveness of such policies based on observed transmission within and between communities. These responses are pre-considered to the extent possible though they generally focus on influenza12,13,14. Due to the unique virulence of the coronavirus responsible for COVID-19, the public-health community’s initial perception of what constituted a stringent response was based very much on the evolving expectations of the disease’s unique features based on historical experience with related outbreaks (such as MERS and SARS) (see CDC, 2020 pp. 554 for a discussion of uncertainty around transmission during the early months of the pandemic).
We use two methods for computing indices to account for the evolving understanding of the most effective measures to inhibit the spread of the coronavirus. The first method of computing the PPI (indexing method 1), as shown in Table 1 reflects what was known in early- to mid-spring 2020. Note that these weights were deemed appropriate during the alert phase (the very first days/weeks of rising global infections) and the early weeks of the pandemic phase. The second method (indexing method 2, shown in Table 2) reflects the changes in guidance later in the year. All such judgements and any such index construction could be at best based on evidence at the time of index construction, and the initial expectations of the stringency were in fact derived from experiences with prior, non COVID-19, pandemics and epidemics. The approach here is that at the beginning and even today, the weight was based on the prevailing idea that the more isolating the measure the stronger it is. So the more strictly the measure isolated someone, the heavier the weight.
Because the efficacy of measures in pandemics/epidemics/infectious disease management etc., depends on the virus, and even that varies with its variants, we make no claim that the weights in our indices are unique and accurately assigned. We believe that they are reasonably assigned. Future will tell, and for that purpose, our codebook provides all components as well as their individual constituent policies in order for researchers to have easy access to reassigning the weights. The codebook contains formulae for calculating full PPIs and category stringencies for types of policies and for all calculated variables. All indices thus can be adjusted to recalculate as needed, from the input–coded variables–to match the nature of research questions that the users might pursue or the future evidence of the relative efficacy of these measures.
At the country level, the Average Regional PPI, Average National PPI, and Average Total PPI are computed by weighing the different units’ PPI values by the units’ population shares (see Appendix A3).
Data collection and processing workflow
As described in Fig. 1, the data collection process consists of seven phases for each time interval under the collection. Three time periods were established as intervals for data collection, within each of those the full process in Fig. 1 applies: the first period was between January 1 and April 24, 2020; the second period lasted between April, 24 and July, 31, 2020; the third interval covered the period from August 1, 2020 until the end of the year. The coder or coders for individual countries and their sub-national units remained consistent within each time period. Coders worked as teams based on global regional (GR) grouping. Senior personnel serving as GR curators generally remained consistent for the entirety of the project.