GAO-21-104622, Accessible Version, TECHNOLOGY ASSESSMENT, Exposure Notification: Benefits and Challenges of Smartphone Applications to Augment Contact Tracing

United States Government Accountability Office

Report to Congressional Addressees

TECHNOLOGY ASSESSMENT

Exposure Notification

Benefits and Challenges of Smartphone

Applications to Augment Contact Tracing

September 2021

GAO-21-104622

Accessible Version

The cover image displays a stylized depiction of the use of an exposure notification app by various individuals and an

example of an exposure notification message.

Cover source: GAO. | GAO-21-104622

United States Government Accountability Office

GAO Highlights

Highlights of GAO-21-104622, a report to

congressional addressees

September 2021

TECHNOLOGY ASSESSMENT

Exposure Notification

Benefits and Challenges of Smartphone

Applications to Augment Contact Tracing

What GAO found

Exposure notification applications (apps)—which determine the proximity of users

and notify people who have been in close contact with another user who was likely

infectious—are expected to enhance the speed and reach of contact tracing and

help slow the spread of infectious diseases such as COVID-19. As of June 2021,

almost half (26/56) of U.S. states, territories, and the District of Columbia had

deployed an app for COVID-19, all using a system developed jointly by Google and

Apple (see figure). In the absence of a national app, states independently launched

apps, resulting in a staggered rollout over 10 months beginning in August 2020.

Map of deployment of exposure notification apps by U.S. states and territories, as of

June 2021

Reported app development costs for selected states varied, ranging from no cost

(provided by a nonprofit organization) to $700,000. Marketing costs for selected

states ranged from $380,000 to $3.2 million. Reported app download levels in the

selected states ranged from 200,000 to more than 2 million, as of June 2021.

GAO identified several challenges limiting app use and the ability of states and

others to determine whether the apps were effective:

Accuracy of

measurements

Technical limitations to measuring distance and exposure can result in

inaccurate exposure notifications.

Privacy and

security concerns

The public may lack confidence that its privacy is being protected, in

part, due to a lack of independent privacy and security assessments and

a lack of federal legal protections.

Adoption

States have faced challenges attracting public interest in downloading

and using an exposure notification app.

Verification code

delays

States faced challenges in promptly providing people who tested

positive for COVID-19 with a verification code necessary to notify other

close contacts of potential exposure using the app.

Evidence of

effectiveness

Limited data are available to evaluate the effectiveness of the apps.

Source: GAO. | GAO-21-104622

View GAO-21-104622. For more information,

contact Karen L. Howard at (202) 512-6888

or, [email protected] or Vijay A. D’Souza, at

(202) 512-6240, [email protected].

Why GAO did this study

With the emergence and rapid global

spread of COVID-19, smartphone

apps have been developed to

supplement manual contact tracing,

which is a public health measure used

to slow the spread of infectious

disease.

GAO was asked to conduct a

technology assessment of exposure

notification apps. This report

discusses (1) the benefits of exposure

notification apps; (2) the current level

of deployment in the U.S.; (3)

challenges affecting their use; and (4)

policy options that may help address

these challenges for future use.

To address these objectives, GAO

reviewed agency documentation, met

with officials from several federal

agencies, and conducted a review of

technical and policy literature. GAO

also interviewed representatives

from companies involved in the

development of exposure notification

apps, public health organizations,

federally funded research and

development centers, and academic

researchers. In addition, GAO

analyzed information from a selection

of states. GAO is identifying policy

options in this report.

GAO received technical comments on

a draft of this report from five federal

agencies and five organizations

included in the review, which it

incorporated as appropriate.

GAO developed the following four policy options that could help address challenges related to exposure notification apps. The policy

options identify possible actions by policymakers, which may include Congress, other elected officials, federal agencies, state and

local governments, and industry. See below for details of the policy options and relevant opportunities and considerations.

Policy Options to Help Address Challenges of Exposure Notification Apps for Future Use

Opportunities

Considerations

Research and Development

(report page 41)

Policymakers could promote

research and development to

address technological

limitations.

· Research on technological limitations could help

increase accuracy, encouraging users to download

and use the apps.

· Research on technologies and architectures other

than those used by U.S. states could lead to

improvements.

· Partnerships with technology companies could spur

innovation and help with integrating improvements.

· The research needed may be costly.

· Improvements may not be cost-effective,

since existing apps may already be

sufficiently accurate.

· Research may result in apps that are not

functional for the next pandemic, since the

current apps were developed for COVID-19.

Privacy and Security

Standards and Practices

(report page 42)

Policymakers could promote

uniform privacy and security

standards and practices for

exposure notification apps.

· Uniform standards and best practices could help

address real and perceived risks to the public’s data,

potentially increasing adoption.

· Standards developed by a broad coalition of

stakeholders could increase the likelihood of

stakeholder agreement and buy-in.

· Policymakers would need to balance the

need for privacy and security with the costs

of implementing standards and practices.

· Implementation of privacy requirements

may need to be flexible, since jurisdictions

could use different approaches.

· Standards and practices could be

challenging to oversee and enforce.

Best Practices (report page

43)

Policymakers could promote

best practices for

approaches to increasing

adoption and to measure

the effectiveness of

exposure notification apps.

· Best practices could help authorities better promote

app adoption.

· Best practices could help state public health

authorities by providing information on procedures

and potential approaches for distributing

verification codes in a timely manner.

· Best practices could help public health authorities

establish a more rigorous way to measure the

extent of app use and any resulting improvements

in notifying exposed people.

· Best practices could require consensus from

many public- and private-sector

stakeholders, which can be time- and

resource-intensive.

· Current best practices may have limited

relevance to a future pandemic.

· In some cases, stakeholders may lack

sufficient information or the experience to

develop best practices.

National Strategy (report

page 44)

Policymakers could

collaborate to enhance

the pandemic national

strategy and promote a

coordinated approach

to the development

and deployment of

exposure notification

apps.

· Enhanced national coordination that builds on

the underlying infrastructure and lessons

learned from COVID-19 could prompt faster

deployment of apps in the future.

· A future national marketing campaign with

cohesive and coherent messaging could result

in wider adoption.

· Policymakers could recommend a national app that

public health authorities could decide to use based

on their individual needs. A national app could add

more functions by integrating exposure notification

capabilities with test scheduling and vaccine

delivery coordination.

· A coordinated national approach would

likely have associated costs and require

sustained funding during the pandemic.

· Coordination of groups with divergent

perspectives and interests may pose

challenges to defining outcomes, measuring

performance, and establishing a leadership

approach.

· It is unclear whether potential users

would be more or less likely to trust a

national exposure notification app than

one developed by a state government.

Source: GAO. | GAO-21-104622

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published

product may be reproduced and distributed in its entirety without further permission from GAO. However, because this

work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you

wish to reproduce this material separately.

Exposure Notification GAO-21-104622 i

Table of Contents

GAO Highlights .................................................................................................................... 3

Why GAO did this study ........................................................................................................... 3

What GAO found ..................................................................................................................... 3

Introduction ........................................................................................................................ 1

1 Background ...................................................................................................................... 4

1.1 COVID-19 ........................................................................................................................... 4

1.2 Manual contact tracing for COVID-19 ............................................................................... 4

1.3 Roles of states, federal agencies, and other stakeholders ................................................ 6

2 Benefits and Design of Exposure Notification Apps ........................................................ 9

2.1 Exposure notification apps are expected to provide enhanced speed and reach ............ 9

2.2 How exposure notification apps work ............................................................................. 10

2.3 Apps can use either a centralized, decentralized, or hybrid system to manage data ..... 12

2.4 States are widely using the Google and Apple Exposure Notifications system .............. 14

3 Deployment and Adoption of Exposure Notification Apps ........................................... 18

3.1 About half of the states have deployed an exposure notification app ........................... 18

3.2 About half of the states use a customized app ............................................................... 21

3.3 States reported that app development time and cost varied based on several factors . 25

3.4 Officials reported varying download levels and use........................................................ 26

4 Challenges Associated with Exposure Notification Apps ............................................... 27

4.1 Accuracy of measurements ............................................................................................. 27

4.2 Privacy and security concerns ......................................................................................... 28

4.3 Adoption and use of apps ................................................................................................ 31

4.4 Verification code delays .................................................................................................. 32

4.5 Limited evidence of effectiveness ................................................................................... 33

5 Policy Options That Could Help Address Challenges for Future Use ............................. 41

5.1 Policy option: Research and development ...................................................................... 41

5.2 Policy option: Privacy and security standards and best practices ................................... 42

5.3 Policy option: Best practices to measure effectiveness .................................................. 43

5.4 Policy option: Enhance the national strategy .................................................................. 44

Exposure Notification GAO-21-104622 ii

6 Agency and Expert Comments ....................................................................................... 46

Appendix I: Objectives, Scope, and Methodology ............................................................ 48

Appendix II: Exposure Notification App Adoption Rates for Selected U.S. States ........... 51

Appendix III: GAO Contacts and Staff Acknowledgments ................................................ 55

Exposure Notification GAO-21-104622 iii

Abbreviations

APHL

Association of Public Health Laboratories

BLE

Bluetooth Low Energy

CDC

Centers for Disease Control and Prevention

COVID-19

Coronavirus Disease 2019

DHS

Department of Homeland Security

FCC

Federal Communications Commission

FTC

Federal Trade Commission

GPS

Global Positioning System

HHS

Health and Human Services

NIH

National Institutes of Health

NIST

National Institute of Standards and Technology

quick response

WHO

World Health Organization

Exposure Notification GAO-21-104622 1

441 G St. N.W.

Washington, DC 20548

Introduction

September 9, 2021

Congressional Addressees

For more than a century, public health authorities have used contact tracing to track and limit

the spread of infectious diseases. Manual contact tracing involves interviewing infected people

to identify others they have been in contact with, notifying those contacts that may have been

exposed, and advising the infected individual and contacts to take appropriate measures.

Manual contact tracing can be effective, but it has limitations. Specifically, it is a resource-

intensive process, and it is most effective during the early stage of an outbreak, when case

numbers tend to be lower, or during phases with fewer cases.

In addition, its effectiveness

relies, in part, on prompt and complete identification of individuals and notification of contacts,

which could be difficult with a rapidly spreading disease. Other limitations of manual contact

tracing include the reliance on human recall of contacts and movements (which can be prone to

error), and the inability to identify strangers. Further, people may not be forthcoming in sharing

information about their contacts.

With the emergence and rapid spread of the highly infectious Coronavirus Disease 2019 (COVID-

19), digital contact tracing technologies have been developed to supplement manual contact

tracing and help address its limitations. One such technology is a type of application (app)

developed for use on a smartphone.

Referred to as an exposure notification app,

it is intended

to be used to notify a smartphone user who has been in close contact with another user who

later tested positive for COVID-19.

This type of app is intended to reduce transmission by

notifying potentially exposed people faster than manual contact tracing, including contacts the

infected person may not have known.

Center for Infectious Disease Research and Policy, COVID-19: the CIDRAP Viewpoint, (Minneapolis, Minn.: University of Minnesota,

June 2, 2020).

An estimated 85 percent of adults in the U.S. own a smartphone, according to a survey conducted by the Pew Research Center.

However, this decreases to an estimated 61 percent for people 65 and older. In addition, the rate varies based on other factors,

including income level and whether a person lives in an urban or rural area. This estimate was based on a survey of U.S. adults,

conducted between Jan. 25 and Feb. 8, 2021. See Pew Research Center, Mobile Fact Sheet, (Washington, D.C.: Apr. 7, 2021),

accessed July 1, 2021, https://www.pewresearch.org/internet/fact-sheet/mobile.

An exposure notification app can include both software that a user downloads to use on a smartphone, or a function built into the

phone’s operating system that can be activated by users.

While contact tracers have previously used smartphone apps, including for data entry, we are unaware of any exposure notification

apps in public use prior to the COVID-19 pandemic.

Exposure Notification GAO-21-104622 2

In July 2020, we issued a Science & Tech Spotlight overview of exposure notification app

technology.

Since then, there has been an increase in the development and use of these apps.

You asked us to conduct an assessment of these technologies. This report discusses (1) the

benefits and design of exposure notification apps, (2) the current level of deployment in the

U.S., (3) challenges affecting their use, and (4) policy options that could help address key

challenges for future use.

To address all of these objectives, we reviewed documentation and met with officials from

selected federal agencies involved in providing guidance, funding research, and directing other

efforts related to exposure notification apps. In addition, we interviewed representatives from

entities involved in the development of exposure notification apps; public health organizations;

federally funded research and development centers; academic researchers; and

nongovernmental organizations. We also conducted a review of literature discussing exposure

notification apps, including their benefits, design, and challenges, as well as relevant policy

options.

To identify the current level of deployment, we developed an inventory of exposure notification

apps that had been deployed by U.S. states, territories, and the District of Columbia (hereafter

referred to as states) as of June 2021. States that had an app in a pilot phase at the time of our

review were included the category of “states that had not deployed an app as of June 2021.”

To obtain additional information associated with the development and use of these apps, we

interviewed state public health officials from a non-generalizable sample of nine states that had

deployed an exposure notification app as of January 1, 2021: Alabama, Colorado, Connecticut,

Minnesota, Nevada, North Carolina, Pennsylvania, Virginia, and Washington. We also reached

out to two other states (Louisiana and Utah) that deployed apps in the later stages of our

evidence collection and received written feedback to structured questions about the status of

their efforts to deploy an app, their rates of adoption, and other topics. We selected the sample

of nine states to obtain a range of views, based on factors such as deployment date,

geographical distribution, number of COVID-19 cases, and app developer. Because the selection

was based on a non-generalizable sample, the results were not used to make inferences about

all states that had deployed an app. In addition to our interviews with officials from the selected

states, we conducted a review of each of the selected states apps, including the key functions,

features, and privacy use policies for those apps.

To obtain perspectives from states that had not deployed an app, we collected information from

a non-generalizable selection of seven states that had not deployed an app at the time of our

review (Montana, Nebraska, Oregon, Rhode Island, South Carolina, Texas, and West Virginia).

GAO, Science & Tech Spotlight: Contact Tracing Apps, GAO-20-666SP (Washington, D.C.: July 28, 2020). While the terms exposure

notification apps and contact tracing apps have been used to describe these technologies, we will use the term exposure notification

apps in this review.

Exposure Notification GAO-21-104622 3

For these states, we conducted an interview with officials from one state and obtained written

responses to a semi-structured set of questions for the other six.

We identified policy options that may address the identified challenges based on our literature

review and interviews. We assessed each policy option by identifying potential benefits and

considerations of implementing them, as identified over the course of our review. See appendix

I for a detailed description of our objectives, scope, and methodology.

We conducted our work from November 2020 to September 2021 in accordance with all

sections of GAO’s Quality Assurance Framework that are relevant to technology assessments.

The framework requires that we plan and perform the engagement to obtain sufficient and

appropriate evidence to meet our stated objectives and to discuss any limitations to our work.

We believe that the information and data obtained, and the analysis conducted, provide a

reasonable basis for any findings and conclusions in this product.

Exposure Notification GAO-21-104622 4

1 Background

1.1 COVID-19

The outbreak of COVID-19 was first

reported on December 31, 2019, in Wuhan,

China.

In the weeks that followed, the virus

quickly spread around the globe. On

January 31, 2020, the Secretary of Health

and Human Services declared a public

health emergency for the U.S., retroactive

to January 27, which followed a World

Health Organization (WHO) declaration on

January 30 that the outbreak constituted a

public health emergency of international

concern. On March 11, 2020, WHO

characterized the COVID-19 outbreak as a

global pandemic due to its levels of spread

and its severity. COVID-19 is highly

contagious and may be spread by people

who are not showing symptoms (i.e.,

“asymptomatic”) or before symptoms

appear (“pre-symptomatic”).

More than a year later, as we have

previously reported, the pandemic has

resulted in catastrophic loss of life and

substantial damage to the global economy,

and to the stability and security of our

This disease is caused by SARS-CoV-2 (Severe Acute

Respiratory Syndrome, Coronavirus 2).

GAO, COVID-19: Key Insights from the GAO’s Oversight of

the Federal Public Health Response, GAO-21-396T

(Washington, D.C.: Feb. 24, 2021).

CDC’s National Center for Health Statistics COVID-19 death

counts in the U.S. are based on provisional counts from

death certificate data, which do not distinguish between

laboratory-confirmed and probable COVID-19 deaths.

Provisional death counts are incomplete due to an average

delay of 2 weeks (a range of 1–8 weeks or longer) for death

certificate processing. Data include deaths occurring from

January 2020 through the week ending on July 3, 2021.

Centers for Disease Control and Prevention, National Center

for Health Statistics, (Atlanta, Ga.), accessed July 7, 2021,

https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm.

nation.

In the U.S., there have been more

than 596,000 reported deaths

and 32

million reported confirmed and probable

cases as of July 2021.

In addition, despite

strides made in getting people vaccinated,

the threat of variants is growing, including

evidence of increased transmissibility. As a

result, uncertainty about the future of the

COVID-19 pandemic remains.

1.2 Manual contact tracing for

COVID-19

Contact tracing is a key component in

controlling the transmission and spread of

infectious diseases, according to the

Centers for Disease Control and Prevention

(CDC).

Contact tracing is intended to

separate the people who have (or may

have) an infectious disease from those who

do not and provide information on other

measures the potentially exposed contacts

should take, such as being tested for the

disease or self-isolating. Together, the test,

trace, and isolate strategy is part of the

Data on COVID-19 cases in the U.S. are based on aggregate

case reporting to the Centers for Disease Control and

Prevention, COVID Data Tracker, (Atlanta, Ga.), accessed July

7, 2021,

https://covid.cdc.gov/covid-data-tracker/#datatracker-home

, and include probable and confirmed cases as reported by

states and jurisdictions. Centers for Disease Control and

Prevention (CDC) COVID-19 counts are subject to change

due to delays or updates in reported data from states and

territories. According to CDC, the actual number of COVID-

19 cases is unknown for a variety of reasons, including that

people who have been infected may not have been tested or

may not have sought medical care.

Centers for Disease Control and Prevention, Operational

Consideration for Adapting a Contact Tracing Program to

Respond to the COVID-19 Pandemic in non-US Settings,

(Atlanta, Ga.: June 23, 2021).

Exposure Notification GAO-21-104622 5

broader effort to limit the transmission of

infectious diseases such as COVID-19.

Contact tracers are the people who

manually trace the contacts of each person

who has tested positive for COVID-19.

Contact tracers begin the process by

interviewing the person with the positive

test result in order to identify others whom

that person might have contacted. Next,

the tracer advises the person and the

contacts to take containment measures

(e.g., a 14-day quarantine for COVID-19),

and coordinates or provides information on

any needed care, testing recommendations,

and resources.

For COVID-19, CDC defines a close contact

as anyone who has been within 6 feet of an

infected person for a total of 15 minutes or

more over a 24-hour period (for example,

three individual 5-minute exposures for a

total of 15 minutes).

According to CDC,

infected persons can spread COVID-19

starting from 48 hours (or 2 days) before

they have symptoms or test positive for

COVID-19.

In a public health emergency such as the

COVID-19 pandemic, it is critical that each

state has a sufficient workforce of contact

tracers in order to contain the disease.

Although state and local public health

Consistent with CDC guidance, except in certain

circumstances, people who have been in close contact with

someone who has COVID-19 should quarantine. However,

people who have been fully vaccinated and people who

were previously diagnosed with COVID-19 within the last 3

months may not need to quarantine.

See Centers for Disease Control and Prevention,

“Appendices,” COVID-19, (Atlanta, Ga.: Updated July 2,

2021), accessed July 2, 2021,

https://www.cdc.gov/coronavirus/2019-ncov/php/contact-t

racing/contact-tracing-plan/appendix.html#contact.

agencies typically maintain an existing

capacity to conduct contact tracing for

infectious diseases, this capacity is generally

sufficient only to respond to relatively small

or isolated outbreaks.

Contact tracing is resource intensive,

because, as cases increase, the contact

tracer will need more time to contact all

potentially exposed persons. Hence, more

and more tracers will be needed to ensure

comprehensive contact tracing of all

diagnosed cases and potentially exposed

persons. The particular features of the

COVID-19 pandemic—asymptomatic

persons and the ability to spread rapidly—

require a significantly large workforce of

contact tracers. According to the National

Association of County and City Health

Officials, the benchmark rate is 30 contact

tracers per 100,000 people. This equates to

about 98,460 contact tracers needed to

cover the entire U.S. population.

According to CDC, state health departments

reported a total of 51,855 employed

contact tracers for the month of December

2020, which was about one month before

the peak of reported new cases in the U.S.

To supplement the capabilities of manual

contact tracing, several states have used

smartphone apps. These apps include those

that help people monitor their COVID-19

National Association of County and City Health Officials,

Position Statement: Building COVID-19 Contact Tracing

Capacity in Health Departments to Support Reopening

American Society Safely (Washington, D.C.: Apr. 16, 2020).

This number represents a best estimate for a 1-month

snapshot and may not include contact tracers employed at

the local health department or community level, according

to CDC documentation. Data are reported monthly, and

estimates will continue, and be updated regularly, according

to CDC. We used the reported estimates from December

2020 to illustrate capacity just before the peak cases

observed in January 2021.

Exposure Notification GAO-21-104622 6

symptoms, assist people in recalling the

places they had visited when providing that

information to a contact tracer, and

exposure notification apps.

1.3 Roles of states, federal agencies,

and other stakeholders

Various entities have a role in the

deployment and use of exposure

notification apps within the U.S. These

entities include states, federal agencies, and

other stakeholders, such as national public

health organizations and organizations

involved in research and development of

the apps.

States

In the U.S., public health authorities at the

state, territorial, and local levels plan and

coordinate pandemic response actions

within their jurisdictions. In addition, these

authorities generally lead contact tracing

efforts, including the implementation of

related technologies, such as exposure

notification apps.

For example, in April 2020, Utah deployed an app which

allowed residents to check their symptoms, and privately

share a subset of their location information with public

health officials to aid in the contact tracing process. In

summer 2020, Utah disabled the location–based services in

this app.

Federal agencies

Federal agencies—including CDC, the

Department of Homeland Security (DHS),

the National Institutes of Health (NIH), and

the National Institute of Standards and

Technology (NIST)—have taken various

steps to assist states in the development

and use of exposure notification apps,

including issuing guidance, distributing

funds to states, and funding research.

Specifically, in May 2020 and December

2020, CDC issued two guidance documents

on digital contact tracing tools, which

included discussion of exposure notification

apps.

The guidance is intended to provide

health departments with minimum and

preferred characteristics of the apps,

including those for contact notification and

data security.

In addition, CDC distributes

funds to states—through established

mechanisms such as its Epidemiology and

Laboratory Capacity for Prevention and

Control of Emerging Infectious Diseases

cooperative agreement, which currently

provides funds to 64 jurisdictions to detect,

prevent, and respond to the growing

threats posed by infectious diseases,

including for the development and use of

exposure notification apps, according to

Centers for Disease Control and Prevention, Preliminary

Criteria for the Evaluation of Digital Contact Tracing Tools

for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and

Guidelines for the Implementation and Use of Digital Tools to

Augment Traditional Contact Tracing, version 1.0 (Atlanta,

Ga: Dec. 15, 2020).

These characteristics included that the apps should enable

health departments to define different exposure risk levels

used to identify contacts based on how close and how long

their exposure was and to require user consent before their

data are shared with a health department.

Exposure Notification GAO-21-104622 7

CDC documentation.

(See ch. 3 for

additional information on uses of this

funding.)

CDC has also funded research on exposure

notification apps, including research

performed by the Massachusetts Institute

of Technology’s Lincoln Laboratory, to

examine barriers to adoption and the

efficacy of the underlying technologies used

by various apps. Further, CDC officials

stated that the agency has provided

ongoing support and consultation to states

interested in implementing exposure

notification apps. For example, since early

August 2020, CDC has coordinated with

Lincoln Laboratory to host meetings with

state public health authorities where they

can discuss issues related to app

development and deployment.

In addition, DHS’s Science and Technology

Directorate provided funding to two

projects through its Silicon Valley

Innovation Program.

These projects are

intended to develop criteria the apps can be

tested against and to enable the capability

to test the apps using the criteria. According

to DHS officials, they expect that these

projects will be completed in the next 2

The 64 jurisdictions receiving awards under the

Epidemiology and Laboratory Capacity for Prevention and

Control of Emerging Infectious Diseases cooperative

agreements include all 50 states, several large metro areas,

and U.S. territories and affiliates. A full list of recipients is

provided on the Centers for Disease Control and

Prevention’s website. See Centers for Disease Control and

Prevention, National Center for Emerging and Zoonotic

Infectious Diseases, Division of Preparedness and Emerging

Infections, Recipients, Project Officers, and Jurisdictional

Assignment Listing, (Atlanta, Ga.: last reviewed June 16,

2021), accessed June 30, 2021,

https://www.cdc.gov/ncezid/dpei/elc/advisor-list.html.

years. NIH has also funded various projects

related to contact tracing tools.

In early 2020, NIST began work on a project,

which is currently ongoing, to study and

develop exposure notification systems with

strong privacy and cybersecurity

protocols.

As a part of this project, NIST

held an event in June 2020 to help facilitate

research aimed at improving the

performance of these kinds of apps.

Further, in January 2021, NIST held a

workshop on challenges associated with

exposure notification apps.

Other stakeholders

National public health organizations have

issued guidance and provided other support

to state public health authorities to assist in

the development and deployment of

exposure notification apps. These

organizations include the Association of

Public Health Laboratories (APHL),

Association of State and Territorial Health

Officials, Council of State and Territorial

Epidemiologists, National Association of

County and City Health Officials, Linux

Foundation Public Health, the Public Health

Informatics Institute, and others. Other key

stakeholders include entities involved in the

research and development of exposure

DHS intends for this program to find new technologies

that strengthen national security.

These projects included tools to identify businesses and

hot spots visited by people with COVID-19 and development

of a digital health pass to enable businesses to verify health

credentials.

A system (or protocol) provides a framework that

determines the function of a particular software application,

like an app on a smartphone.

Exposure Notification GAO-21-104622 8

notification apps or analysis of their

performance. Specifically, researchers,

organizations, and technology companies

have played a key role in the design of the

systems used by exposure notification

apps.

In May 2020, Google and Apple—the two

primary developers of operating systems

for smartphones—collaborated on the

development of an exposure notification

system used by the states discussed later in

the report. According to Google and Apple,

they developed this system to help

governments and the global community

slow the spread of the COVID-19 pandemic.

In addition, Google and Apple collaborated

with Microsoft and APHL to establish and

host servers to facilitate the system.

Further, Google and Apple collaborated

with the MITRE Corporation to deploy the

Exposure Notification Private Analytics

portal, which provides public health

authorities with data on the performance of

the states’ apps. This effort involves several

other partners, including the Internet

Security Research Group and NIH.

These include, for example, the TCN Coalition and

Massachusetts Institute of Technology Pact developed

systems—referred to as the Temporary Contact Numbers

Protocol, or TCN Protocol; and the Private Automated

Contact Tracing (PACT) protocol, respectively.

Exposure Notification GAO-21-104622 9

2 Benefits and Design of Exposure Notification Apps

Exposure notification apps are intended to

automate and augment the manual contact

tracing process, with enhanced speed and

reach being among the expected benefits,

according to scientific literature, state

officials, federal agency documents, and

representatives of stakeholder

organizations we interviewed. They work by

using proximity detection to determine

when two smartphone users are in close

contact, then notifying all contacts of a user

who later reports a positive test result for

COVID-19. The apps can use a centralized,

decentralized, or hybrid system to collect,

store, and manage data. Many states within

the U.S. are using apps based on a

decentralized system that was developed

jointly by Google and Apple.

2.1 Exposure notification apps are

expected to provide enhanced

speed and reach

Exposure notification apps are expected to

provide two key benefits—speed and reach.

Specifically, they are expected to allow for

more timely identification and notification

of contacts and greater (more complete)

coverage of contacts, according to the

majority of the selected states, CDC

John Hopkins University and Association of State and

Territorial Health Officials, A National Plan to Enable

Comprehensive COVID-19 Case Finding and Contact Tracing

in the US (Baltimore, Md.: Johns Hopkins University, Apr. 10,

2020), Massachusetts Institute of Technology Lincoln

Laboratory, Realizing the Promise of Automated Contact

Tracing Technology to Control the Spread of COVID-19:

Recommendations for Smartphone App Deployment, Use,

and Iterative Assessment (Cambridge, Mass.: Massachusetts

documentation, and publications that we

reviewed.

Speed. Apps are expected to allow for

faster identification and notification of

contacts. After a positive test result is

received, apps automate the process of

identifying and notifying contacts. This

automation can lead to faster notification,

which in turn can lead to faster changes in

individual behavior aimed at helping slow

disease transmission, namely testing and

quarantine, according to selected studies.

Reach. Apps are also expected to provide

more complete and faster identification of

contacts because, unlike manual contact

tracing, they do not rely on a person’s

memory to identify the people they came

into contact with, according to CDC

documents. In addition, according to a Pew

Research Center report, 41 percent of

Americans asked about their views on

speaking with a public health official

reported that they are unlikely to talk with

contact tracers. The report also noted that

younger adults, those with lower incomes,

and those with less formal education are

especially unlikely to engage with manual

contact tracers.

Apps may provide a way

to increase coverage of these populations.

In addition, apps can reach people even

Institute of Technology, Oct. 29, 2020), and J.A. Moreno

Lopez et al., “Anatomy of Digital Contact Tracing: Role of

Age, Transmission, Setting, Adoption, And Case Detection,”

Science Advances, vol. 7, no. 15 (2020): eabd8750.

Pew Research Center, The Challenges of Contact Tracing

as U.S. Battles COVID-19, (last updated Oct. 30, 2020),

accessed July 2, 2021,

https://www.pewresearch.org/fact-tank/2020/10/30/key-fi

ndings-about-americans-views-on-covid-19-contact-tracing/.

Exposure Notification GAO-21-104622 10

when manual contact tracing resources are

limited or overwhelmed.

2.2 How exposure notification apps

work

Exposure notification apps use proximity

detection to determine whether two app

users are in close contact. The app then

notifies a person if they had been in close

contact with another user who was likely

infectious at the time, and who voluntarily

confirmed their diagnosis in the app.

Proximity detection involves a series of

automated actions that determine the

proximity of two persons, notify them of

potential exposure, and provide guidance in

the case of exposure.

The proximity detection steps are described

more fully here.

· An exposure notification app

periodically broadcasts messages

(referred to as encounter messages)

using a wireless radio transmission

technology—Bluetooth Low Energy

(BLE)

—that contain, among other

things, a random identifier and the

strength of the signal sent (i.e.,

transmitted power).

Any other phone

that has the same or similar app

installed and is in range of the user’s

BLE is a wireless radio transmission technology with a

range of around 30 feet. BLE started to be included in

smartphones in 2011, and is now included on most

smartphones to enable communication between devices,

such as smart watches and wireless headphones.

To help preserve user privacy and to limit the ability to

track the movements of other users, the random identifiers

are changed on a periodic basis (e.g., every 10 minutes).

Further, the identifiers do not reveal any personal

information about other users.

signal can receive and store these

encounter messages. The distance

between two phones can be estimated

by comparing the strength of the BLE

signal when it was sent with its strength

when it is received.

· If one or more of the messages later

turns out to have been from a contact

who tested positive for COVID-19, a

central server or a user’s smartphone

analyzes the encounter message to

determine whether the user’s risk of

exposure exceeds a predetermined

threshold. The risk analysis includes

factors such as the time spent at

various distances and when the contact

occurred in relation to when the

contact was most infectious.

The formula used to calculate the level of

risk, including the specific risk factors, can

be set by the public health agency. The

assessment generally involves determining

whether the encounter meets the CDC’s

definition of a close contact (i.e., at least 15

minutes within 6 feet within 24 hours). The

apps do not consider other factors that

affect the risk of infection. For example,

they do not consider whether the users

were wearing masks, or whether the

encounter occurred in a well ventilated

location (e.g., indoors or outdoors).

The BLE signal will weaken as the distance between two

smartphones increases. The strength of the signal when it is

received is referred to as a received signal strength

indication measurement.

This can be determined based on when the person first

had symptoms or was tested.

Exposure Notification GAO-21-104622 11

If a user’s risk of exposure exceeds the risk

threshold, the user receives an exposure

notification from the app.

The exposure

notification can also include other

information, such as when the exposure

occurred and the next steps the person

should take, such as getting tested,

monitoring symptoms, and self-

quarantining. See figure 1 for an example of

an exposure notification message.

The distance measured between two

phones using BLE is only an estimate, and

its accuracy can be affected by various

factors. See section 4.1 for additional

discussion of factors affecting the accuracy

In certain systems, a public health provider could provide

the notification in lieu of an app notification.

Other methods to determine a smartphone’s location

include assisted-GPS, the triangulation of cell towers, and

Wi-Fi access point identification.

of measurements and other potential

technologies.

Location data can also be used instead of,

or in addition to, the data gathered using

the BLE messages. Location data are not

currently used by U.S. states. However,

other nations (e.g., Israel) have apps that

use Global Positioning System (GPS) data to

track and record a person’s location,

including the date and time.

Further, apps

can track user locations by having the user

scan a quick response (QR) code at a

specific location (e.g., venue, restaurant).

The app then records the location, date,

A QR code is a barcode with the ability to encode diﬀerent

types of information. Each location needs to have a unique

QR code and it must be accessible (e.g., posted at the

entrance to a building).

Exposure Notification GAO-21-104622 12

and time.

See figure 2 for an example of a

QR code.

A user’s recent locations can be compared

with a list of locations of people who have

tested positive for COVID-19 to determine

the risk of exposure.

However, GPS

location estimates are only accurate within

about a 16-foot radius outdoors. In

addition, the accuracy decreases near

buildings, bridges, and trees, and indoors or

underground.

Thus, the location estimates

may not always be reliable in determining

whether two people were in close contact.

The app uses the phone’s camera to scan the QR code.

The list of locations from infected persons can also include

locations obtained through manual contact tracing.

National Coordination Office for Space-Based Positioning,

Navigation, and Timing, GPS Accuracy, (Washington, D.C.:

last update Apr. 22, 2020), accessed May 17, 2020,

https://www.gps.gov/systems/gps/performance/accuracy.

2.3 Apps can use either a

centralized, decentralized, or hybrid

system to manage data

Exposure notification apps can use a

centralized, decentralized, or hybrid

architecture for collecting, storing, and

analyzing data.

The main difference

between these types of architecture is the

extent to which the information used to

determine exposure is stored and analyzed

on a central server or on an individual

smartphone. These differences affect the

privacy protections built into the system. A

decentralized architecture may help

preserve a user’s privacy more than a

centralized architecture. These types of

architecture are described more fully here.

· In a centralized architecture, most of

the data are stored on a central server,

which also analyzes the data to

determine which users may have been

exposed. For example, the central

server collects personal information as

a part of users’ registration and

generates the random identifiers used

for the encounter messages. A public

health authority can access this

information (including information on

which users were in contact) and

aggregate it to perform further analyses

of the data to identify additional

potential exposures and to identify

potential surges in cases to help inform

Data architecture is a framework that comprises of

models, policies, rules, and standards that govern the

collection, storage, arrangement, integration, and use of

data in organizations.

Exposure Notification GAO-21-104622 13

broader mitigation and response

efforts. The central server can also

incorporate data from other sources,

such as manual contact tracing (e.g.,

locations an infected person visited).

However, storing data on a centralized

server can also reveal potentially

sensitive information to governmental

organizations, or others who gain

access to the server. An example of a

centralized app is one used by the

nation of Singapore, which it deployed

in March 2020.

· In a decentralized architecture, most of

the data are located on users’

smartphones, with only limited data on

a central server. Each user’s device

analyzes the data to determine whether

an exposure has occurred. This

approach may help preserve personal

privacy; however, it also limits the data

that are available to public health

authorities for determining the

effectiveness of the apps, informing

contact tracing efforts, and identifying

where infections may be occurring. For

example, this architecture does not

allow authorities to know who received

an exposure notification.

· A hybrid architecture incorporates

aspects of both architectures.

Specifically, the random identifier

generation for encounter messages

remains decentralized (i.e., handled by

user smartphones) to help preserve

privacy, while the risk analysis and

notifications are handled by the central

server. Hybrid systems have been

developed by researchers, but we are

not aware of their use at a national or

state level.

Table 1 provides a comparison of the

different architectures, including how data

are managed and the key advantages and

disadvantages to each approach.

Exposure Notification GAO-21-104622 14

Table 1: Advantages and disadvantages of centralized, decentralized, and hybrid data architectures

used in exposure notification apps

Source: Based on GAO review of technical and industry documentation. I GAO-21-104622

The central server can also incorporate data from other sources, such as manual contact tracing (which can provide

the location of an infected person among other things).

2.4 States are widely using the Google

and Apple Exposure Notifications

system

U.S. states with apps are using the Google and

Apple Exposure Notifications system.

(See

ch. 3 for additional information on

deployment of apps by state public health

authorities.) The Google Apple system was

released in May 2020 as an application-

programming interface to be used by public

health authorities in developing and

For the purposes of this report we refer to the Google and

Apple Exposure Notifications system as the Google Apple

system.

customizing their own exposure notification

apps.

In September 2020, Google and Apple

provided public health authorities with an

additional option (referred to as the Express

option), which was intended to make it easier

for authorities to use the Google Apple

system by removing the need for the

authorities to build their own custom apps. In

this option, Google developed an app for

Android-based phones, and Apple deployed

app-less functionality, such that a person can

Centralized

Decentralized

Hybrid

Where most information is

stored

central server

smartphone device

Where random identifiers

are generated

central server

smartphone device

Where exposure data are

analyzed

central server

smartphone device

central server

Level of data access by

public health authorities

higher

lower

moderate

Key advantages

Public health authorities

can access data to

perform analysis,

identify additional

exposure and potential

surges, and inform

response efforts.

Seeks to preserve

individual privacy by

limiting data accessible to

entities (e.g., public health

authorities).

Seeks to preserve

individual privacy and

provides health

authorities with useful

data.

Key disadvantages

Data could reveal

potentially sensitive

information to public

health authorities or

other entities that gain

access to the server.

Limits the data that are

available to public health

authorities for determining

how well the app works

and to inform response

efforts.

Data could reveal

potentially sensitive

information to public

health authorities or

other entities that gain

access to the server.

Exposure Notification GAO-21-104622 15

enable the system for their area (if available)

through the settings on an iPhone.

The Google Apple system uses BLE and a

decentralized architecture. In this system,

each user’s app creates a temporary key

(changes every 24 hours) that the app uses to

generate random identifiers, and to encrypt

information provided in the encounter

messages. The app then exchanges a random

identifier with other users’ apps, and

maintains a list of the encounter messages

that the user has received. To help preserve

the privacy of the users, the encounter

messages do not include personal information

or location data.

If the user tests positive for COVID-19, a

public health authority uses a verification

server to generate a verification code, and

then sends the user the code to verify the

positive test result. A user can then

voluntarily input this code in the app to

submit their recent temporary keys (e.g.,

prior 14 days) to a key server. If a user

chooses not to input the code in the app, the

user will not enable the notification of other

recent close contacts who were also using the

app of potential exposure.

Each exposure notification app periodically

downloads the temporary keys from people

who had recently tested positive from the key

server and then compares it with its list of

The Express option works differently on iPhones and Android

phones. For iPhones, the Express option is built into the

operating system and can be activated by users in the iPhone

settings. To receive exposure notifications, no app is required

and therefore it is said to have “app-less” functionality. For

Android smartphones, states develop the configuration

settings (e.g., risk parameters), and Google then develops the

app, which states can then use. For Apple devices, the Google

Apple system works on iPhones running at least iOS 12.5. For

Android, the system works for any smartphone capable of

running Android version 6.

encounter messages. If there is a match, the

app analyzes the risk of exposure based on

the method and parameters established by

the public health authority. If the risk exceeds

a predetermined threshold, the app displays

an exposure notification to the app user. The

notification can include guidance and

instructions. Figure 3 provides an overview of

this process.

In August 2020, Microsoft partnered with

APHL to establish a key server that could be

used by all U.S. states—the National Key

Server. With its launch, apps from different

U.S. states could be interoperable, so that app

users can find out if they have been exposed

without needing to download and use apps

from multiple states.

This feature is

particularly important in regions where

commuters regularly cross jurisdictional

boundaries (e.g., in neighboring areas of

Washington D.C., Maryland, and Virginia).

Further, according to APHL, the server also

reduces the burden of each state’s public

health agency needing to build and host its

own key server.

In addition to the National Key Server, APHL

manages a central verification server,

referred to as the Multi-tenant Verification

Server, which was launched in September

2020. APHL made the verification server

available to reduce the effort needed by

public health agencies to bring an exposure

App interoperability means that a person using an app from

one state could receive an exposure notification based on an

encounter with any other person who had an app that also

used this server, such as a person from another state.

Exposure Notification GAO-21-104622 16

notification app to their jurisdiction. As a part

of the Google Apple system, APHL noted that

a verification server is necessary to ensure a

user has received a positive test result before

uploading their temporary keys to the

National Key Server. APHL also noted that

rather than each public health agency

standing up its own verification server and

deciding on a verification approach, providing

one verification server reduces the time and

cost to deploy the Google Apple system. For

the states and territories with apps, nearly all

were using the National Key Server, while

over two-thirds were using the Multi-tenant

Verification Server as of August 2021,

according to APHL.

Although a verification server is necessary, it does not have

to be the Multi-tenant Verification Server, so some states

elected to use their own verification servers.

Exposure Notification GAO-21-104622 17

Exposure Notification GAO-21-104622 18

3 Deployment and Adoption of Exposure Notification Apps

Almost half of U.S. states have deployed an

exposure notification app. In the absence of a

national exposure notification app, states

have independently launched their own apps

at different times, resulting in a staggered

rollout. States have developed and deployed

apps using the Google Apple system and

about half have customized their apps, which

provides the apps with more flexibility and

functionality. According to officials from

selected states, development time, costs, and

levels of adoption have varied.

3.1 About half of the states have

deployed an exposure notification

app

As of June 2021, 26 of 56 U.S. states

(including territories and the District of

Columbia) have deployed an app (see fig. 4).

Unlike other countries, the U.S. does not have

a national exposure notification app; instead,

states have independently deployed

individual apps.

Exposure Notification GAO-21-104622 19

The patchwork of app deployment shown in

figure 4 arises from the fact that public health

authorities at the state and territorial levels

decide whether and when to deploy an app.

Furthermore, there was no existing option for

a national app that states could use,

according to CDC documentation. As of June

2021, 26 out of 56 states had deployed apps.

Officials from seven selected states that had

not deployed an app cited several reasons for

that decision, including limited cell phone

coverage in rural areas or other challenges

related to the deployment and use of an app

(see ch. 4 for additional detail). They also

cited competing priorities, such as natural

disaster response or vaccine distribution

efforts, and were concerned that exposure

notification app development would divert

limited resources away from other priorities.

The 26 states deployed apps over a span of 10

months, in a staggered rollout beginning in

August 2020. Figure 5 provides a timeline of

app deployment and related events. Virginia

was the first state to deploy an exposure

notification app, in August 2020, and

Massachusetts was the most recent, in June

2021. Seventeen of the states deployed an

app between October 2020 and June 2021,

which was after the Express option was made

available (see fig. 5).

Exposure Notification GAO-21-104622 20

Note: The timeline indicates the Google Apple exposure notification option that was initially deployed by the state

(i.e., custom or Express). Seven states deployed the Express option after initially deploying a custom app (Minnesota,

Nevada, Hawaii, New York, Virginia, Louisiana, and New Jersey) between January and April 2021. In addition to

exposure notification apps based on the Google Apple system, a few states developed smartphone apps to help

people monitor their COVID-19 symptoms or assist in recalling the places they had visited when providing that

information to a contact tracer. For example, in April 2020, Utah deployed an app which allowed residents to check

their symptoms, and privately share a subset of their location information with public health officials to aid in the

contact tracing process. In summer 2020, Utah disabled the location-based services in this app.

Exposure Notification GAO-21-104622 21

3.2 About half of the states use a

customized app

Of the 26 U.S. states that had deployed

apps as of June 2021, all are using a version

of the Google Apple system.

With this

system, public health authorities can

choose to develop customized apps, use the

Express option, or use both in tandem (see

table 2). States could use, for example, a

customized app for Android and the Express

option for iOS smartphones.

In addition to the U.S., most countries with an app use BLE

(primarily using the Google Apple system); approximately

one-third use GPS. Some countries use both BLE and QR

codes; for example, the United Kingdom’s National Health

Service’s app uses both the Google Apple system and QR

codes to check-in to locations.

One of the selected states uses a customized app for both

Android and iOS and also enabled the Express option for iOS.

Exposure Notification GAO-21-104622 22

Table 2: U.S. states deployment of apps using the Google Apple Exposure Notifications system

Legend: ● = State or territory that is using a version of the Google and Apple Exposure Notifications system; ○ =

State or territory that is not using the identified version of the Google Apple system.

Source: GAO compilation of data from selected states, related documents, interviews, and other sources. I GAO-21-104622

Note: State public health authorities can deploy customized exposure notification apps, which may offer unique

functions and features. State public health authorities can also elect to use the Express option of the Google Apple

system. The Express option provides convenience and efficiency but potentially less flexibility to tailor exposure

notification functionality. The total in the table (33) does not equal the number of states with apps (26) because some

states use both a customized app and the Express option.

To build a customized exposure notification

app, public health authorities can seek

external technical support (e.g., third-party

developers, nonprofit organizations, or

university partners). For the Express option,

public health authorities can provide Google

and Apple an electronic configuration file that

includes instructions and content, including

States

Customized app

Express option

Alabama

●

○

Arizona

●

○

California

○

●

Colorado

○

●

Connecticut

○

●

Delaware

●

○

District of Columbia

○

●

Guam

●

○

Hawaii

●

Louisiana

●

Maryland

○

●

Massachusetts

○

●

Michigan

●

○

Minnesota

●

Nevada

●

New Jersey

●

New Mexico

○

●

New York

●

North Carolina

●

○

North Dakota

●

○

Pennsylvania

●

○

Utah

○

●

Virginia

●

Washington

○

●

Wisconsin

○

●

Wyoming

●

○

Total

Exposure Notification GAO-21-104622 23

the risk parameters for enabling an exposure

notification and messaging for app users.

While states can use their own internal

technical team or seek outside help to

develop their app, officials from all nine of the

selected states we interviewed said they had

limited technical expertise and resources and

received varying levels of external support to

deploy their apps, regardless of whether they

used customized apps or the Express option.

According to Google and Apple

representatives, the Express option was

developed to help states quickly and easily

deploy their app. However, the Express

option does not offer states the same

flexibility to tailor the functions and features

of their app as do customized exposure

notification apps.

For example, a customized app may help

users identify testing facilities and access

state-level statistics about COVID-19

infections and death rates. State officials

noted that they included these features to

provide information to the public outside of

their agencies’ websites, which they hoped

would encourage people to download and

use their app. Based on our observations of

exposure notification apps for the selected

states, a common customized function was

the ability to share the app with others.

Figure 6 shows screenshots for two

customized apps. These images illustrate how

an app can be tailored to offer unique

functions in the user interface. For example,

one screenshot illustrates a unique function,

“Healthcheck,” which allows app users to

report any COVID-19 related symptoms,

exposure history, and testing history to their

public health authority.

Exposure Notification GAO-21-104622 24

Note: The number in the image on the left (490238) is an illustration of a verification code that would be provided by

a public health authority to an app user to verify the positive results of a COVID-19 test. A user can then voluntarily

input this code in the app to submit their recent temporary keys.

Exposure Notification GAO-21-104622 25

Based on our review of the apps for the nine

selected states, among other qualitative

differences between state apps, we noted

variation in the depth and scope of guidance

information provided to app users.

Specifically, some states provided more

detailed information on symptoms, testing,

and quarantine. We also found that the apps’

privacy use agreements provided varying

details on how the apps protect privacy,

including how users can delete their data.

3.3 States reported that app

development time and cost varied

based on several factors

Officials from each of the nine selected states

we interviewed and the two additional states

that provided written information varied in

their reported app development time frames

and costs.

Some public health authorities

from these states attributed these variations

to several factors, including legal review and

marketing efforts. Officials from nine of the

11 states reported that the time to develop

their apps ranged from less than 2 months to

over 5 months. This time included the

development of the apps, as well as

conducting legal reviews of contracts,

Some of the exposure notification apps had embedded links

to the public health authorities’ websites, which provided

access to the state’s privacy use agreement or other

information.

We interviewed state public health officials from a non-

generalizable sample of nine states that had deployed an

exposure notification app as of January 1, 2020: Alabama,

Colorado, Connecticut, Minnesota, Nevada, North Carolina,

Pennsylvania, Virginia, and Washington. We also reached out

to two other states, Louisiana and Utah, which deployed apps

in the later stages of our evidence collection about the status

of their efforts to deploy an app and received written feedback

to our structured questions.

This includes the nine selected states we interviewed and the

other two states that provided written responses to our

questions.

preparing marketing campaigns, and choosing

to pilot the app prior to the full release. States

that chose the Express option generally noted

shorter development times.

The cost of app development also varied

according to the information reported to us

by officials from the 11 states.

One state

reported zero development costs because a

nonprofit organization developed the state’s

app, while another state reported

development costs of $700,000.

State

officials noted that their marketing costs also

varied; costs ranged from $380,000 to $3.2

million, as of June 2021.

States used federal funding for development

and marketing costs; some used state funding

as well. Six of the nine states reported that

they used CARES Act funding to support the

development of their apps or marketing costs.

However, according to the CDC, which

distributes certain CARES Act and

supplemental COVID-19 relief funds through

its Epidemiology and Laboratory Capacity for

Prevention and Control of Emerging Infectious

Diseases cooperative agreement,

exposure

notification apps are allowable expenses

through these awards, but the agency does

We did not independently verify the states’ reported costs.

As part of the Coronavirus Aid, Relief, and Economic Security

Act (CARES), Coronavirus Preparedness and Response

Supplemental Act, and Paycheck Protection Program and

Health Care Enhancement Act supplements, the cooperative

agreement awarded approximately $11 billion to support the

domestic response to COVID-19. See CARES Act, Pub. L. No.

116-136, 134 Stat. 281 (2020); Coronavirus Preparedness and

Response Supplemental Appropriations Act, 2020, Pub. L. No.

116-123, 134 Stat. 146 (2020); Paycheck Protection Program

and Health Care Enhancement Act, Pub. L. No. 116-139, 134

Stat. 620 (2020). An additional award of $19.11 billion from the

Coronavirus Response and Relief Supplemental Appropriations

Act of 2021, Pub. L. No. 116-260, Div. M was awarded to

continue to shore up domestic response efforts to COVID-19.

See Consolidated Appropriations Act, 2021, Div. M, Pub. L. No.

116-260, 134 Stat. 1182 (2020).

Exposure Notification GAO-21-104622 26

not require recipients to report on use of

funds to support exposure notification apps.

3.4 Officials reported varying

download levels and use

Different app download levels (or activations

for states using the Express option) were

reported by officials from the nine selected

states we interviewed and the two additional

states that provided written information.

Specifically, four states reported less than 1

million, four states reported 1 to 2 million,

and two states reported more than 2 million

downloads (or activations), as of June 2021.

The other state does not track these data.

According to Google and Apple

representatives, states that initially deployed

a custom app and then later added the

Express option, experienced a significant

increase in activations. Specifically,

representatives stated that the adoption rate

quadrupled for four states that added the

Express option (Nevada, New Jersey, New

York, and Virginia). However, there may be

other factors that affect adoption rates.

Further, the number of downloads and

activations is not an accurate reflection of the

number of people using the app. For example,

a person could download or activate the app

and not use it, or could download the app

multiple times. See section 4.5 for additional

information on this issue and appendix II for

additional information on state app adoption

rates.

Download data includes Android and iOS phones in states

with customized apps; downloads for Android phones in states

using the Express option; or “activations” for iOS phones in

states using the Express (“app-less”) option.

In addition, different levels of app use were

reported by officials from the nine selected

states we interviewed and the two additional

states that provided written information.

Specifically, for two states the number of

times their app users received exposure

notifications as of June 2021 were above

30,000 (31,000 for one state and 42,000 for

the other), while four states reported

notifications that ranged from about 900 to

3,800; the remaining five states did not track

these data. However, the number of

notifications depends on a variety of factors,

including the extent of the app users’

contacts.

Further, limited data are available on the

extent to which exposure notifications

affected people’s behavior, according to

public health officials and studies we

reviewed. For example, public health

authorities do not know whether app users

are actually using the app and following

instructions for next steps contained in the

alerts. Seven of the nine selected states that

we interviewed said that they did not track

whether app users actually sought testing or

medical care based on the receipt of an

exposure notification from an app; one state

said it was done inconsistently and the other

remaining state did not respond to our

request for these data.

States with customized apps can calculate download levels

for Android and iOS smartphones from data obtained from

Google and Apple apps stores. However, for states using the

Express option, they can determine downloads for Android

devices but must estimate the number of users that activated

the app on iOS smartphone.

Exposure Notification GAO-21-104622 27

4 Challenges Associated with Exposure Notification Apps

We identified the following five categories

of challenges associated with these apps:

· Accuracy of measurements

· Privacy and security concerns

· Adoption and use of apps

· Verification code delays

· Evidence of effectiveness

4.1 Accuracy of measurements

The techniques that exposure notification

apps use to measure distance have

technical limitations that can result in users

receiving false exposure notifications. For

example, BLE wireless radio technology,

used to measure the distance between two

smartphones, cannot always reliably

measure whether two smartphones are

within 6 feet of each other. In addition,

research has demonstrated that the BLE

signal strength does not always decrease

with distance, and can even increase with

distance under certain conditions.

For

example, objects in the environment

between a sender and a receiver (e.g.,

furniture, walls, people) can impact the

signal, causing the received signal strength

to vary substantially. Other factors include

the type of phone and antenna, whether

See Douglas Leith & Stephen Farrell, “Coronavirus Contact

Tracing: Evaluating the Potential of Using Bluetooth

Received Signal Strength for Proximity Detection,” ACM

SIGCOMM Computer Communication Review, vol. 50, no.4

(2020); 1-11.

Like BLE, ultra-wideband is a wireless radio transmission

technology, but it could provide measurements that are

more accurate. However, ultra-wideband is only available on

certain newer smartphones. Ultrasound refers to the

transmission of inaudible acoustic pulses in the ultrasonic

frequency range between phones.

the phone is being held or is in a pocket or

otherwise obstructed location, and the

position of one phone with respect to the

other phone (e.g., if it has a 90 degree

rotation or is facing down).

As a result, a person could receive a

notification even if that person was far

away or separated by a physical barrier

from an infected person. Such a result,

known as a false positive, can lead a person

who has been notified to take unnecessary

steps, such as getting tested, or self-

quarantining. Further, false positives could

reduce that person’s confidence in the app,

which could lead to them not using the app

or using it less often. In addition, without

accurate measurements, an app could fail

to detect that two people are in close

proximity for a certain amount of time,

leaving the potentially exposed person with

a false sense of security—a false negative.

To help address these limitations, various

industry experts have highlighted the

potential of using other technologies to

perform measurements instead of or in

addition to BLE, including ultra-wideband

signals and ultrasound.

Several studies

have found that these other technologies

may be more accurate than BLE.

See, for example, N. Ahmed et al., “A Survey of COVID-19

Contact Tracing Apps,” IEEE Access, vol. 8 (July 2020):

134577-134601, accessed December 1, 2020,

https://doi.org/10.1109/ACCESS.2020.3010226; and J.

Meklenburg et al., “SonicPACT: An Ultrasonic Ranging

Method for the Private Automated Contact Tracing (PACT)

Protocol,” arXiv.org (Dec. 2020): 1-14, accessed November

24, 2020, https://arxiv.org/abs/2012.04770.

Exposure Notification GAO-21-104622 28

addition, researchers have suggested that

exposure notification apps could use sensor

technologies to improve distance

estimation based on BLE measurements,

such as by using a gyroscope, an

accelerometer, or a magnetometer.

For

example, these technologies could help to

detect the position of the phone. However,

thus far, there has been only limited use of

these technologies. Specifically, while some

entities with an exposure notification app

use ultrasound technology for distance

estimation, including several U.S.

universities, as of June 2021, we did not

find any exposure notification apps that use

ultra-wideband signals, a gyroscope,

accelerometer, or a magnetometer.

4.2 Privacy and security concerns

Privacy

Officials from all nine of the selected states

identified privacy as an important factor in

determining whether to implement an

exposure notification app and in selecting

the system used by the app (i.e., Google

Apple system). In particular, officials stated

that users would likely not adopt an app

that collected their personal information,

including location data.

A gyroscope is a device used for measuring or maintaining

orientation and angular velocity. An accelerometer is a

device used to measure acceleration forces. A

magnetometer is a device that measures the strength and

sometimes the direction of magnetic fields.

An example of such an assessment is a privacy impact

assessment that is used by federal agencies in response to

requirements in the E-Government Act of 2002. Among

other things, the assessment is an analysis of how personally

identifiable information is handled to ensure compliance

with applicable privacy requirements and manage privacy

risks. Also, a privacy impact assessment includes a formal

Despite the privacy protections built into

the apps by Google and Apple, the public

may lack confidence that their privacy is

protected, in part, due to a lack of

independent assessments and federal legal

protections for the privacy of app data. In

particular, CDC’s guidance on the

implementation and use of exposure

notification apps recommends that the

apps go through independent security and

privacy assessments,

and that the results

be made publicly available.

However, we

found that none of the nine selected states

had fully implemented this guidance.

Specifically, officials from five of the nine

selected states reported that security and

privacy assessments were performed;

however, the results were not made

publicly available. The remaining four states

reported that these assessments were not

performed.

Currently there is no federal law that

provides the public with clearly applicable

privacy protections for the information that

exposure notification apps gather.

Specifically, in January 2019, we reported

that the U.S. did not have a comprehensive

internet privacy law governing the

collection, use, and sale or other disclosure

of consumers’ personal information.

Accordingly, we recommended that

Congress consider developing legislation on

document detailing the process and the outcome of the

analysis. See Office of Management and Budget, Managing

Information as a Strategic Resource, Circular A-130

(Washington, D.C.: July 2016).

Centers for Disease Control and Prevention, Guidelines for

the Implementation and Use of Digital Tools to Augment

Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec.

15, 2020) and Preliminary Criteria for the Evaluation of

Digital Contact Tracing Tools for COVID-19, version 1.2

(Atlanta, Ga.: May 17, 2020).

Exposure Notification GAO-21-104622 29

internet privacy that, among other things,

would enhance consumer protections.

Legislation governing the collection and use

of consumers’ personal information—in

particular for exposure notification apps—

could help to safeguard their privacy, and

provide the public with greater assurance

that its privacy is protected. However, such

legislation has not been enacted.

In the absence of such legislation, individual

companies have set their own privacy

requirements for exposure notification

apps, including requirements on the

collection and use of the data. For example,

Google and Apple have each established

requirements for their exposure notification

system and their respective app stores

regarding data collection and privacy. These

requirements specify that only the

minimum amount of user data that is

necessary for response efforts should be

collected, and that the data may only be

used for such efforts.

In addition, the

requirements state that the apps cannot

collect any information to identify or track

the precise location of users.

GAO, Internet Privacy: Additional Federal Authority Could

Enhance Consumer Protection and Provide Flexibility,

GAO-19-52 (Washington, D.C.: Jan. 15, 2019). Other federal

laws governing health information, including the Health

Insurance Portability and Accountability Act, may not

provide consistent, clearly-applicable privacy protections for

the information that likely would be gathered and used in

digital contact tracing activities. See Congressional Research

Service, COVID-19: Digital Contact Tracing and Privacy Law,

LSB10511 (Washington, D.C.: July 9, 2020).

Congress has introduced several bills over its last two

sessions that address aspects of exposure notification apps

or digital contact tracing tools. Of the bills that have been

introduced, one that was enacted into law related to

implementing a national strategy for contact tracing and

enhancing information technology and data modernization

capabilities (American Rescue Plan Act of 2021, Pub. L. No.

117-2, § 2401, 135 Stat. 4, 40 (2021)). However, it does not

Security

To ensure that exposure notification apps

function as intended and that user privacy

is protected, it is important that developers

build in security protections. However,

security assessments of these apps are

limited.

Security considerations should include the

supporting infrastructure—such as central

servers—and address how the data are

stored and maintained, including

appropriate authentication and access

controls. Security incidents could lead to

privacy violations (e.g., identifying or

tracking users) or disrupt the functioning of

the app (e.g., inserting false data). This

would likely result in the public’s loss of

confidence in the apps, potentially leading

to decreased use. Researchers have

identified a variety of potential threats for

specifically address exposure notification apps or the

associated privacy issues. In addition, at least one state

passed a law regarding the use of location data for contact

tracing. Specifically, in June 2020 Kansas passed a law stating

that contact tracing shall not be conducted through the use

of any service or means that uses cell phone location data to

identify or track, directly or indirectly, the movement of

persons. See K.S.A. § 48-961 (2021).

See, for example, Google, Google COVID-19 Exposure

Notifications Service Additional Terms, (last modified May 4,

2020), accessed May 16, 2021,

https://blog.google/documents/72/Exposure_Notifications_

Service_Additional_Terms.pdf; and Apple, Exposure

Notification APIs Addendum, (last revised May 4, 2020),

accessed May 16, 2021,

https://developer.apple.com/contact/request/download/Ex

posure_Notification_Addendum.pdf.

Exposure Notification GAO-21-104622 30

exposure notification apps.

Table 3

identifies several examples of these threats

and their potential effects.

Table 3: Examples of threats and their effects for exposure notification apps

Threat

Description

Potential Effect

Re-

identification

Comparing exposure notifications with

personal logs of a phone owner’s recent

contacts.

The identity of an infected app user is revealed.

Denial of

service

Broadcasting fake encounter messages to

consume resources of other smartphones.

Loss of availability of a smartphone due to the

additional battery power, storage, and processing

time required to store and process the fake

messages.

Phone

tracking

Tracking a user’s location by analyzing the

information sent in encounter messages,

such as the random identifiers.

A user’s location and movements is revealed.

Relay (or

replay)

Re-transmitting captured encounter

messages at the same or a different

location.

Smartphone receives exposure notification despite

not coming in close contact with an infected person

(i.e., false positive).

Source: GAO review of selected literature. | GAO-21-104622

The Google Apple system includes features

intended to mitigate these threats. In

addition, according to representatives from

these companies, they had a third party

perform a security assessment of the

system. There are also ongoing assessments

on the security of exposure notification

apps, including on the Google Apple

system.

However, as exposure notification

See, for example, N. Ahmed et al., “A Survey of COVID-19

Contact Tracing Apps”; Massachusetts Institute of

Technology Lincoln Laboratory, “Exposure Notification

Security Assessment Considerations,” Lexington,

Massachusetts. Unpublished Article; and M. Chowdhury et

al., “COVID-19 Contact Tracing: Challenges and Future

Directions,” IEEE Access, vol. 8 (Nov. 2020): 225703-225729,

accessed February 16, 2021,

https://ieeexplore.ieee.org/document/9252092.

For example, the Massachusetts Institute of Technology’s

Lincoln Laboratory has developed security assessment

considerations for the Google Apple system and conducted a

security assessment of one of the apps used in the states. In

addition, in February 2021, DHS’s Science and Technology

Directorate’s Silicon Valley Innovation Program awarded

funding to a company (AppCensus) for a project to develop

apps are a relatively new technology, these

assessments have, as of now, limited data

and results. Further, as previously stated,

the selected states have not provided the

results of independent security assessments

in a public format, as recommended by CDC

guidance.

testing and validation services. In April 2021, the company

reported that it had identified a vulnerability with apps using

the Google Apple system. Specifically, the company reported

that preinstalled apps could gain access to system logs made

by exposure notification apps on Android devices. According

to the company, these logs could include information, such

as whether a person had received an exposure notification

and the random identifiers that a smartphone device had

sent and received. Google representatives stated that a fix

for this vulnerability was available as of May 5, 2021, and

that there is no evidence that it was exploited.

Centers for Disease Control and Prevention, Guidelines for

the Implementation and Use of Digital Tools to Augment

Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec.

15, 2020).

Exposure Notification GAO-21-104622 31

4.3 Adoption and use of apps

States have also faced challenges attracting

public interest in downloading (or

activating) and using an exposure app. State

public health officials told us that, in spite

of their marketing and outreach efforts,

getting people to download (or activate)

and use their app is difficult for the

following reasons.

Lack of trust. Mistrust of governmental

health authorities and technology

companies can lead people to forgo using

apps, according to literature and state

officials. For example, officials from six of

the 11 states cited public concerns about

the use of apps for government surveillance

(e.g., using the apps to track users’ location)

as a leading obstacle to app adoption,

even though these apps do not collect

location data. In addition, the public may

not trust big technology companies with

their data. These concerns may be

exacerbated by reported vulnerabilities

with apps using the Google Apple system.

Specifically, as previously stated, a company

reported that other apps on a phone could

potentially gain access to sensitive

information, including whether a person

had received an exposure notification. In

addition, the lack of trust regarding the use

of apps may be intensified by other

incidents where technology companies

potentially misused consumers’ personal

information. For instance, in April 2018,

Facebook disclosed that a Cambridge

University researcher may have improperly

shared the personal data of 87 million of its

The 11 states include nine from our selected sample plus

two additional states.

GAO-19-52.

users, which followed other incidents of

misuse of personal information.

Also,

multiple officials noted that the public was

skeptical of the Google Apple system, since

it is a joint initiative between U.S.

technology companies and the government

that involves personal health information.

In particular, officials from three states

indicated that the public expressed

concerns about the perceived “big brother”

nature of exposure notification.

Lack of understanding of how apps

function. Multiple officials said they

frequently had to counter misinformation

about how the apps work and the data they

collect. For instance, officials from one state

reported that they emphasized the app's

use as a public health communication tool

because of misinformation describing the

exposure notification app as a data

collection tool used to surveil and track the

public.

Also, multiple officials said the public had a

limited understanding about the apps’

privacy-preserving features. Officials from

three states said that they believed the

public did not understand the technical

aspects of the apps, which may include how

the random identifiers do not reveal

personal information.

Such

misunderstandings may contribute to public

unwillingness to download or use the apps.

Further, such misunderstandings may also

contribute to hesitance to enter verification

codes for people who had downloaded an

app on their device. Specifically, sometimes

people receive positive COVID-19 test

According to Google and Apple documentation, the

random identifiers exchanged with other smartphones are

not linked to the app user’s identity or phone number and

change on a periodic basis (e.g., every 10 minutes).

Exposure Notification GAO-21-104622 32

results long after the app has been

downloaded. Because people may not have

initially understood (or have forgotten) how

the app works, including the apps’ built-in

privacy preserving features, some app users

many not want to input their verification

codes to prompt exposure notifications to

other people, according to officials from

selected states.

Lack of awareness of the availability of the

apps. Officials from several states noted

that it was difficult to make people aware of

the apps. For example, officials from

multiple states noted that they thought

building awareness in closely connected

communities with influential leaders would

encourage people to download and use an

app. However, one state tried, but was

unsuccessful, in recruiting support from

some of these groups, including churches

and a college football program. One official

said the lack of support was a lost

opportunity to build awareness and

increase that state’s app adoption rate.

Further, some states reported having

minimal resources for marketing, which one

official said resulted in low awareness of

the state’s app. Three states reported that a

federally led national marketing campaign

would have helped promote their app and

drive higher rates of adoption. Similarly,

officials from a national health organization

reported that a national public awareness

campaign led by the CDC would help

encourage adoption and be more cost-

effective than individual state campaigns.

Limited access. Another reason it can be

difficult to get people to download and use

an exposure notifications app is lack of

access to a smartphone, reliable cellular

coverage, and broadband internet service,

according to selected states and literature.

Indeed, officials in a few states told us one

reason they chose not to deploy an app was

the lack of necessary supporting

infrastructure or internet service in rural

areas. To be effective, exposure notification

apps need to be downloaded and used by a

critical mass of the general public. While

the levels of adoption needed to achieve

certain measures of effectiveness are not

well established (see section 4.5), increasing

the number of people using the app should

result in a greater likelihood that users who

come in close contact with an infectious

person will be notified of potential

exposure, according to CDC and the

Massachusetts Institute of Technology’s

Lincoln Laboratory.

4.4 Verification code delays

States were challenged in distributing

verification codes quickly to the public. As

previously stated, these codes are used to

confirm that a person had a positive test or

diagnosis before they are able to upload

their recent temporary keys to the National

Key Server. For people to be notified of

potential exposure quickly, these

verification codes need to be distributed in

a timely manner and users need to

voluntarily decide to use them to notify

recent contacts.

Officials from several of the selected states

reported that their initial process for

distributing the codes required a public

health official, such as a contact tracer, to

provide a person with the verification code

via phone after the person had received a

positive test result or a confirmed diagnosis.

However, states reported that, due to

staffing shortages, in particular as cases

surged, it sometimes took several days to

provide the code. As a result, some app

users who had tested positive for COVID-19

Exposure Notification GAO-21-104622 33

were delayed in submitting their

verification codes to notify others of

possible exposure, according to officials

from selected states.

In addition, there

can be delays in test results being available

and provided to health care providers who

report the results to local or state health

officials. In particular, earlier in the

pandemic, testing availability and

turnaround time for results could take a

week or more. Following the receipt of test

results, the health care provider or

laboratory then reports the results to local

or state health officials. Any delays in this

process could also contribute to delays in

public health authorities’ distribution of

verification codes to individuals after the

individual has received the confirmed

diagnosis.

In addition, a few states noted that contact

tracers did not always follow the state’s

processes for providing app users with a

verification code to enter COVID-19 test

results in the app. For example, when a

contact tracer was conducting an interview

with a person, they were supposed to ask if

the person had downloaded the state’s app,

and if so, to provide them with a

verification code. However, officials from

one state stated that this was not always

performed.

To help address this challenge, five of the

nine selected states implemented an

automated process to distribute the codes.

Instead of providing the codes entirely

through phone calls, some states also send

text messages with the code or a link to a

If the user tests positive for COVID-19, a public health

authority generates a verification code and then sends the

user the code to verify the positive test results. A user can

then voluntarily input this code in the app to submit recent

temporary keys.

website with instructions for how to obtain

the code. State officials reported that this

automated distribution resulted in an

increase in the number of verification codes

disseminated to app users. For example,

following implementation of the new

process, one state's average distribution of

verification codes increased from 15 to 85 a

day, according to public health officials. In

addition, officials from a different state

reported that they had seen an increase in

the number of codes redeemed, and

improved the timeliness of code

redemption following implementation of

the new system. However, another state

noted that, even after automating the

process, it still took 4 days, on average, for

a person to receive a verification code

following a positive test result.

4.5 Limited evidence of

effectiveness

We found limited evidence that exposure

notification apps are effective at enhancing

the speed or reach of manual contact

tracing or at reducing the spread of disease.

One reason for the dearth of evidence is

that states collect limited data from

exposure notification apps due to the

emphasis on data privacy. In addition, little

to no guidance exists on what data to

collect and how to collect the data. As for

slowing the spread of COVID-19, studies

have not yet sufficiently demonstrated that

exposure notification apps are having an

effect. CDC and others have, therefore,

Exposure Notification GAO-21-104622 34

highlighted the need for additional research

into the effectiveness of exposure

notification apps in preventing the spread

of disease.

4.5.1 States collect limited data from

exposure notification apps

The privacy protections that are

incorporated into the functionality of the

existing apps limit the data available to

public health authorities, which reduces the

ability to measure and improve the

effectiveness of the apps. For example,

officials from all nine selected states said to

preserve personal privacy, they do not

collect data on who has installed an app,

including who has received an exposure

notification. In addition, states have limited

data on how well the apps are working,

including how changes to the formula used

to calculate the level of risk affects the

number of people provided with exposure

notifications.

In addition, states do not

collect location data, so they are unable to

identify where disease spread is occurring.

Furthermore, states do not collect data on

the speed of exposure notification,

according to our review of information

provided by selected states. Metrics on the

speed of notification are not provided as

part of the Google Apple system, though

these data could be collected by the states

individually, should they choose to,

according to Google and Apple

representatives. With fast-spreading

MITRE is planning to enhance the Exposure Notification

Private Analytics portal with additional features, including

the ability to analyze how changes in risk parameters affect

the number of exposure notifications. The portal and the

exposure notification analytics data it provides are only

available to states using the Express option.

diseases like COVID-19, the speed of

contact tracing plays a critical role in

reducing disease spread, allowing those

who may have been exposed to take action

more quickly. States could use the speed of

notification as one indicator or metric of

app effectiveness. According to Google and

Apple representatives, the time between

when a user submits their temporary keys

to the key server and when a person would

be notified is estimated to be between 4

and 10 hours. However, a few factors can

delay the submission, including the amount

of time it takes to receive a test result (e.g.,

time for test processing and reporting to

the health care provider and to the local

health officials) and when that person

receives and uses the verification code.

4.5.2 States lack guidance on measuring

effectiveness

States lack guidance for measuring the

effectiveness of exposure notification apps.

Officials from nearly all the selected states

told us they wanted to gauge the impact of

their apps and assess effectiveness, such as

the enhanced reach through electronic

notification. Officials from several of the

selected states said that they had reached

out to CDC for guidance regarding

recommended approaches and indicators

for measuring app effectiveness, which was

confirmed by CDC officials. However, the

requested information was not available.

CDC officials indicated that they considered

developing additional guidance to evaluate

Exposure Notification GAO-21-104622 35

app effectiveness. However, they

acknowledged there are limited app

evaluation strategies available due to the

lack of data from exposure notification apps

and, as a result, they are not planning to

develop additional guidance. Because of the

lack of federal guidance, officials from

states said they were “on their own” and

began reaching out to other states and

countries that had deployed apps for advice

and best practices, such as metrics for

measuring effectiveness. Officials from the

majority of the selected states said they

wished there had been additional guidance

available, including how to measure app

effectiveness; officials from three states

used the analogy of “building the plane

while flying it,” to describe their experience

deploying their apps with limited direction.

CDC has developed general guidance on

exposure notification apps, such as

minimum and preferred characteristics.

CDC also developed guidance to measure

the success of manual contact tracing

efforts, including both process and outcome

metrics, but has not developed specific

guidance on criteria to use in measuring

app effectiveness, such as increased speed

Centers for Disease Control and Prevention, Preliminary

Criteria for the Evaluation of Digital Contact Tracing Tools

for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and

Guidelines for the Implementation and Use of Digital Tools to

Augment Traditional Contact Tracing, version 1.0 (Atlanta,

Ga.: Dec. 15, 2020). CDC’s May 2020 and December 2020

guidance and website information on digital contact tracing

tools did not include a definition for effectiveness or any

standardized metrics for states or indicators for measuring

app effectiveness.

Centers for Disease Control and Prevention, Evaluating

Case Investigation and Contact Tracing Success, (Atlanta,

Ga.: May 26, 2020), accessed June 9, 2021,

https://www.cdc.gov/coronavirus/2019-ncov/php/contact-t

racing/contact-tracing-plan/evaluating-success.html.

or reach compared to manual contact

tracing.

The lack of standardized metrics was

identified as a challenge in President

Biden’s National Strategy for the COVID-19

Response and Pandemic Preparedness in

January 2021. The strategy noted that

states use and report different metrics for

tracking COVID-19 response activities,

including contact tracing, and called for

common federal metrics to evaluate

progress and the identification of areas

where additional federal resources should

be directed.

In part due to the lack of federal guidance,

selected states varied in the types of data

that they are collecting to measure the

overall effectiveness of their own apps and

have developed their own metrics and

indicators for determining how well the

apps are working. States are using one or

more of these metrics:

· App downloads or activations

· Verification codes issued

· Verification codes claimed

The White House, National Strategy for the COVID-19

Response and Pandemic Preparedness, (Washington, D.C.,

Jan. 21, 2021).

As previously mentioned, download data includes Android

and iOS phones in states with customized apps; downloads

for Android phones in states using the Express option; or

“activations” for iOS phones in states using the Express

(“app-less”) option.

Verification codes issued refers to the codes that are

disseminated to app users with a positive COVID-19 test

result. The codes may be provided on the phone by contact

tracers or through other methods, such as text message.

Verification codes claimed refers to the codes entered by

app users with a positive COVID-19 test result that enables

them to send the recent temporary keys to the National Key

Server to notify others that they may be at risk.

Exposure Notification GAO-21-104622 36

· Exposure notifications generated

Yet, officials from eight of the nine selected

states noted that some of these metrics

provide a limited understanding of app

effectiveness. Data on downloads,

verification codes, and exposure

notifications provide public health

authorities some information to gauge how

effective they are. However, these metrics

do not indicate how quickly people were

notified of exposure or timeliness relative

to manual contact tracing alone. With

Exposure notifications are alerts provided to close

contacts of the app users who confirm a positive test or

diagnosis using a verification code they enter into the app.

However, they are only an estimate, as app users voluntarily

provide this information.

regard to reach, the number of downloads

gives an approximate, but not actual sense

of the number of app users.

The number of downloads is not an

accurate reflection of app usage. Officials

from one state speculated that some

people were downloading the apps out of

curiosity but never enabling them on their

smartphone. Further, after a person

downloads the app, they need to perform a

series of actions for it to be used as

intended (see fig. 7). Therefore, a user

Exposure Notification GAO-21-104622 37

could download but not use the app or

download it more than once, according to

public health officials from several selected

states. Also, people could choose to not

enable receiving exposure notifications;

ignore notifications; and if they do test

positive, not voluntarily provide that

information. While download totals may

represent the possible population of app

users that could receive benefits (i.e.,

notification) from these apps, these

limitations hinder states from

understanding the effectiveness of using

exposure notification apps.

Verification codes claimed by the people

who have tested positive for COVID-19 and

who entered the information in the app

(which prompts exposure notifications to be

sent to others) may also provide an

indication of app use. Similarly, the number

of exposure notifications gives a sense of

how many people are notified of potential

exposure to COVID-19, but it may provide

limited insight into the effectiveness of the

public health intervention because users

need to voluntarily provide this information

in their apps. Finally, none of the data

indicate whether people changed behavior

as a result of the notification.

While people seeking testing or medical

care could be asked whether they sought

testing or care due to an exposure

notification from an app, such information

may violate a user's expectations of privacy,

Infectious disease models are simplified versions of reality

that help to characterize disease spread (see GAO-20-372

for an overview of infectious disease modeling). Other types

of epidemiological studies of contact tracing apps that could

be conducted in real world settings, rather than via

modeling, face methodological, logistical, and ethical

challenges, including the lack of empirical data, confounding

factors that affect disease spread, and other issues.

according to officials from one state.

Further, because states do not track who

receives an exposure notification, states

have a limited understanding of what

impact, if any, these notifications have on

disease spread and the overall effectiveness

of their apps.

Nonetheless, public health officials from

seven of the nine selected states said they

believe the exposure notification apps have

been effective and that their apps had been

worthwhile. Officials from two states said

they think adoption even at relatively low

levels would help slow disease spread.

Furthermore, exposure notification apps

provided a new tool for states to use—at a

time of urgent need—to limit the spread of

COVID-19.

4.5.3 Evidence of reduced disease

spread has been limited but additional

studies are underway

We reviewed seven selected modeling

studies that have sought to measure the

effects of the use of apps on the spread of

COVID-19. However, there are important

limitations to these studies—such as limited

evidence to support assumptions about

behavioral changes—which hinder the

ability to draw high-confidence conclusions

about the apps effectiveness.

The studies

we reviewed generally suggested that the

use of exposure notification apps can

reduce disease transmission.

In general,

The selected studies we reviewed covered a range of

geographic areas, including Washington State, the United

Kingdom, Spain, and Switzerland, and were published

between April 2020 and May 2021. Five of the seven papers

are peer-reviewed publications. We identified the papers

from our interviews with subject matter experts and a

search of the literature. Studies we reviewed include:

Exposure Notification GAO-21-104622 38

the studies suggested that app usage can

decrease COVID-19 infections and deaths,

with the size of the estimated effects

depending on the level of app adoption,

among other things. For example, one peer-

reviewed study estimated that, when 15

percent of the population used an exposure

notification app, infections could be

reduced by approximately 8 percent and

deaths by about 6 percent. Another peer-

reviewed study in the United Kingdom

estimated that a 30 percent app uptake

averted approximately one infection for

every four infections that arose over a 4½-

month period.

However, there are significant limitations to

these modeling studies. For example, the

models estimated outcomes by relying on

assumptions about app usage and

behavioral changes associated with

notifications. These assumptions covered

factors such as how many people used an

app, how many app users had a positive

test result, and how many app users self-

R. Hinch, et al., Effective Configurations of a Digital Contact

Tracing App: A Report to NHSX, (April 16, 2020), accessed

December 9, 2021,

https://cdn.theconversation.com/static_files/files/1009/Rep

ort_-_Effective_App_Configurations.pdf?1587531217.

M. Abueg, et al., “Modeling the Effect of Exposure

Notification and Non-pharmaceutical Interventions on

COVID-19 Transmission in Washington State,” npj Digital

Medicine, (4, 49), (March 12, 2021) accessed March 12,

2021, https://www.nature.com/articles/s41746-021-00422-

C. Wymant, et al., “The Epidemiological Impact of the NHS

COVID-19 App,” Nature, Vol. 594, no. 7863 (2021).pp. 408-

412, accessed February 25, 2021.

P. Rodríguez, et al., “A Population-Based Controlled

Experiment Assessing the Epidemiological Impact of Digital

Contact Tracing,” Nature Communications, (January 26,

2021), accessed February 22, 2021,

https://www.nature.com/articles/s41467-020-20817-6.

S. Marcel, et al., “Early Evidence of Effectiveness of Digital

Contact Tracing for SARS-CoV-2 in Switzerland,” Swiss

Medical Weekly, (December 16, 2020), accessed March 9,

2021, https://smw.ch/article/doi/smw.2020.20457.

isolated. In some cases, particularly with

studies earlier in the pandemic, these

assumptions were not grounded in research

and were not otherwise well supported. For

example:

· Assumptions in one study were that

everyone notified of a potential

exposure would self-isolate, with a 2

percent drop-out rate each day, and

that 18 percent of infected people

remained asymptomatic, with no

variation in this rate across age

groups.

These assumptions were not

grounded in evidence because little to

none was available at the time.

· A study of three counties in Washington

State assumed in its simulations that it

would take 2 days from symptom onset

to receive a COVID-19 test result, which

the authors characterized as a key

assumption underlying the findings.

However, in the earlier months of the

D. Menges, et al., “A Data-Driven Simulation of the Exposure

Notification Cascade for Digital Contact Tracing of SARS-CoV-

2 in Zurich, Switzerland,” JAMA Network Open, (4

(4):e218184), (April 30, 2021), accessed July 13, 2021,

https://jamanetwork.com/journals/jamanetworkopen/fullar

ticle/2779376.

Massachusetts Institute of Technology Lincoln Laboratory,

“Simulated Automatic Exposure Notification (SimAEN):

Exploring the Effects of Interventions on the Spread of

COVID,” Private Automated Contact Tracing (PACT)

Technical Report #3, (December 8, 2020), accessed March 1,

2021, https://pact.mit.edu/simulated-automatic-exposure-

notification-simaen-exploring-the-effects-of-interventions-

on-the-spread-of-covid-wlogos/.

Estimates of the COVID-19 asymptomatic rates vary

widely by age group, according to information from CDC.

Centers for Disease Control and Prevention, “Estimated

Disease Burden of COVID-19,” COVID-19, (Atlanta, Ga.:

updated May 19, 2021), accessed July 13, 2021,

https://www.cdc.gov/coronavirus/2019-ncov/cases-updates

/burden.html.

Exposure Notification GAO-21-104622 39

pandemic, wait times for test results in

U.S. could be a week or more.

· Oxford University researchers

associated with the United Kingdom

studies told us that the recent rise in

variant strains and vaccinations has

increased uncertainty in assumptions

about disease transmission.

In addition to the studies on disease spread,

some studies have estimated shorter-term

outcomes, such as the number of close

contacts detected by exposure notification

apps. In one simulation study, the findings

implied that the app prompted quarantine

recommendations for, at most, an

estimated 5 percent more exposed contacts

than manual contact tracing. However, as

with the modeling studies we reviewed on

disease spread, these studies of shorter-

term outcomes are also subject to

important limitations, such as model inputs

derived from studies with limited sample

sizes or national estimates applied to states

or local regions.

Since we originally identified papers for our

review, additional studies are now

D. Lazer, et al., “Report #8: Failing the Test: Waiting Times

for COVID Diagnostic Tests Across the U.S.” in The State of

the Nation: A 50-State COVID-19 Survey, (OSF Preprints,

August 2020), accessed July 13, 2021,

https://doi.org/10.31219/osf.io/gj9x8.

See, for example, C. Segal, et al., Early Epidemiological

Evidence of Public Health Value of WA Notify, a Smartphone-

based Exposure Notification Tool: Modeling COVID-19 Cases

Averted in Washington State (June 2021), accessed July 1,

2021,

https://www.medrxiv.org/content/10.1101/2021.06.04.212

57951v4; and J. Masel, et al., Quantifying meaningful

adoption of a SARS-CoV-2 exposure notification app at the

campus of the University of Arizona (June 2021), accessed

June 1, 2021,

https://www.medrxiv.org/content/10.1101/2021.02.02.212

51022v6.

underway that suggest the use of apps can

help mitigate the spread of COVID-19.

addition, some states are conducting their

own evaluations of the effectiveness of

exposure notification apps in reducing

disease spread.

4.5.4 CDC and others have highlighted

the need for additional research and

data

Exposure notification apps are a relatively

recent public health intervention. As a

result, additional primary research on the

benefits and effectiveness of exposure

notification apps is needed, according to

CDC and other public health researchers.

This includes a need for primary research

into the use of digital tools in conjunction

with manual systems, since public health

authorities are unlikely to use digital tools

in isolation, according to all selected states

and most literature we reviewed.

Specifically:

· CDC has noted that more data are

needed from preliminary

implementation efforts to quantify the

public health value of these apps.

A. Anglemyer, et al., “Digital contact tracing technologies

in epidemics: a rapid review.” Cochrane Database of

Systematic Reviews. (2020).

Centers for Disease Control and Prevention, Guidelines for

the Implementation and Use of Digital Tools to Augment

Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec.

15, 2020).

Exposure Notification GAO-21-104622 40

Also, the agency has highlighted the

need for more studies on the

effectiveness of digital tools, including

exposure notification apps, to support

contact tracing and reduce the spread

of infectious disease. In addition, CDC

has identified a specific research need

to comprehensively compare the

effectiveness of manual contact tracing

with exposure notification apps and has

initiated work to study these issues with

the Massachusetts Institute of

Technology’s Lincoln Laboratory,

according to CDC officials.

· WHO has called for additional research

on the minimum adoption levels

required for these apps to be effective

in light of the limited evidence to

date.

In addition, since some

populations have limited access to

digital technology, WHO identified the

potential for the systematic exclusion of

individuals who cannot access such

technologies. It called for additional

research and sufficient regulatory

oversight of these issues.

· Linux Foundation Public Health noted

that additional research on the

effectiveness of exposure notification

apps is needed, since there is currently

a limited understanding of the extent to

which apps may have changed the

course of the pandemic in the U.S.

The organization called for a data

driven approach and research on app

efficacy to help app developers and

others, including the federal

government, decide whether to

improve apps for potential future use or

abandon the approach if research

showed that desired outcomes had not

been achieved.

Moreover, the need for additional research

on the effectiveness was identified as a

challenge in President Biden’s National

Strategy, issued in January 2021. The

strategy notes that the federal government

should work with public health authorities

and the private sector to collect COVID-19

data on a range of issues, including the

effectiveness of contact tracing.

World Health Organization, Contact Tracing in the Context

of COVID-19, Interim Guidance, February 1, 2021.

Linux Foundation Public Health was founded in summer

2020 with an initial focus on helping public health

authorities deploy apps based on the Google Apple system.

The White House, National Strategy for the COVID-19

Response and Pandemic Preparedness, (Washington, D.C.,

Jan. 21, 2021).

Exposure Notification GAO-21-104622 41

5 Policy Options That Could Help Address Challenges for Future Use

We identified four policy options that, when

implemented, could help address the

challenges we have identified for both

current and future use of exposure

notification apps.

Policymakers could also

choose to maintain the status quo—that is,

allow current efforts to proceed without

intervention. The relevant policymakers

could include Congress, other elected

officials, federal agencies, state and local

governments, academic research

institutions, and industry. While some

challenges described in this report may be

addressed through current efforts, other

challenges may not be resolved, may be

exacerbated, or may take longer to resolve

without intervention. The four policy

options are in the following areas: research

and development, privacy and security,

data collection and measurement, and

national strategy.

5.1 Policy option: Research and

development

Policymakers could promote research and

development to address technological

limitations.

Description

Research could seek to address the

technical limitations we identified that can

We present policy options that were within the scope of

this technology assessment. This is not an exhaustive list of

all potential policy options, nor are policy options intended

to be recommendations to federal agencies or matters for

congressional consideration. They are not listed in a specific

rank or order, and we are not suggesting that they be

completed individually or combined in any particular

fashion. We did not conduct the detailed additional analysis

result in users receiving false exposure

notifications, such as by improving the

accuracy of the distance measurements

performed by exposure notification apps.

For example, apps could use additional

sensors (e.g., the gyroscope and

magnetometer that certain smartphones

already have) or other technologies, such as

ultra-wideband and ultrasound. In addition,

research could examine methods for

evaluating other factors that affect the risk

of disease transmission, such as whether

the encounter occurred indoors or

outdoors.

Policymakers could promote research in

multiple ways, including by providing grants

to academic and research institutions or by

setting up a public-private partnership.

Further, the research could build off of prior

and ongoing research by various entities.

Opportunities

· Research on technological limitations,

such as inaccurate distance

measurements, could help increase the

accuracy and speed of exposure

notification apps, incentivizing users to

download and use them.

· Research on technologies and

architectures other than those used by

U.S. states could also improve the apps,

for example by increasing the speed

that would be needed to fully implement a specific policy

option or combination of options—for instance, on potential

design and legal issues—nor did we assess how effective the

options may be. We express no view regarding the extent to

which legal changes would be necessary to implement them.

Exposure Notification GAO-21-104622 42

and reach of notifications. Such

alternatives include GPS and centralized

or hybrid data architecture.

· Partnerships with technology

companies could help with integrating

improvements into smartphone

operating systems. These collaborations

could spur further technological

innovation.

Considerations

· Research into new technologies could

be costly and is generally considered a

long-term investment with uncertain

benefits.

· The roles for government, the private

sector, and academia in researching

new technologies for exposure

notification apps would need to be

defined, planned, and coordinated to

ensure that research and costs are not

duplicative.

· Research may not produce cost-

effective improvements, because

existing apps may still be sufficiently

accurate for notifying a person of

potential exposure. Moreover, other

alternative technologies also have

accuracy limitations, and other data

architectures may increase the risk of

revealing sensitive user information.

· Research into new technologies based

on the COVID-19 pandemic may also

result in apps that are not functional for

future outbreaks or pandemics.

Diseases that are not transmitted

through the air, such as sexually

transmitted diseases, would require

apps that use different methods to

determine potential exposure. In

addition, the continuous changes in

smartphone technology would require

ongoing research.

5.2 Policy option: Privacy and

security standards and best

practices

Policymakers could promote uniform

privacy and security standards and best

practices for exposure notification apps.

Description

Policymakers could support the

development of privacy and security

standards and best practices for exposure

notification apps to ensure that these apps

function as intended and that user privacy

is protected. One way to do this would be

to specify standards for public health

authorities to ensure that personal data are

encrypted when stored, and to specify

limits on the types of data that can be

collected and how the data may be used

and disclosed. In addition, the standards

could specify that the data can only be used

for disease response efforts and that

personal data cannot be shared with other

agencies, law enforcement, or immigration

authorities without a user’s consent.

Another action policymakers could take is

to require that standards be developed and

agreed on by a broad coalition of

stakeholders. Best practices could also be

developed by government agencies (e.g.,

NIST) or the private sector.

Opportunities

· Developing and adopting uniform

privacy and security standards and

related best practices could help

address real and perceived risks that

Exposure Notification GAO-21-104622 43

the public’s data might be misused or

otherwise not appropriately protected.

· Standards developed and agreed on by

a broad coalition of stakeholders could

increase the likelihood of still broader

stakeholder agreement and buy-in.

· Independent security and privacy

assessments could evaluate apps based

on these standards and best practices,

and these assessments could be made

publicly available.

Considerations

· Policymakers would need to balance

the need for privacy and security with

the direct and indirect costs of

developing and implementing these

standards and practices.

· Implementing these privacy

requirements may require flexibility

because different jurisdictions could

use different technologies (e.g., BLE or

GPS) and data architectures to collect

and use the data.

· It could be challenging to determine

how to oversee and enforce the privacy

and security standards and practices.

5.3 Policy option: Best practices to

measure effectiveness

Policymakers could promote best practices

to increase adoption and measure the

effectiveness of exposure notification apps.

The National Academies of Sciences, Engineering, and

Medicine, Encouraging Participation and Cooperation in

Contact Tracing: Lessons from Survey Research,

(Washington, D.C.: Aug. 2020).

Description

Policymakers could assess the approaches

that states have used to increase adoption

and then develop best practices based on

those results. Best practices could also

include standardization of the metrics

collected and reported to measure

effectiveness as well as the procedures for

verification code distribution. The

development of best practices could be led

by a broad coalition of stakeholders and

result in guidance to states. These efforts

could help address the challenges we

identified related to app adoption,

verification code delays, and efficacy

determination.

Opportunities

· Best practices could help state public

health authorities share strategies to

improve app adoption. For example, if a

state found that translating the app into

multiple languages improved adoption

among non-English speaking people,

this information could be shared with

other states. Also, understanding and

appealing to user motivations could

promote app adoption. Further,

partnering with trusted sponsors could

encourage cooperation in COVID-19

contact tracing, as published by the

National Academies of Sciences,

Engineering, and Medicine.

· Such practices could help state public

health authorities by providing

information on potential methods and

Exposure Notification GAO-21-104622 44

processes for distributing verification

codes in a timely manner.

· In addition, best practices can help

states to measure the effectiveness and

impact of these apps. Best practices

could also leverage outside knowledge

to promote app adoption. More

accurate measurement of app

effectiveness would help public health

authorities identify opportunities for

improvement, both to the technology’s

function and to its widespread use.

Considerations

· The creation of best practices could

require consensus from many public

and private sector stakeholders, which

can be time- and resource-intensive.

· If the best practices are not updated,

they may not be relevant or useful in a

future pandemic.

· In some cases, stakeholders may lack

sufficient or complete information or

the experience to develop best

practices. If best practices are put in

place without sufficient basis, it could

limit further innovation.

5.4 Policy option: Enhance the

national strategy

Policymakers could collaborate to enhance

the national strategy and promote a

coordinated approach to the development,

The White House, National Strategy for the COVID-19

Response and Pandemic Preparedness, (Washington, D.C.,

Jan. 21, 2021).

deployment, and use of exposure

notification apps.

Description

Policymakers could evaluate whether to

enhance the current national strategy or a

future pandemic response strategy to

enable a coordinated nationwide approach

to the development and deployment of

exposure notification apps.

This could

help address the challenges we identified

related to the adoption of these apps and

evidence of their effectiveness. An

enhanced strategy could include specifying

federal, state, and local roles and

coordination efforts. Further, an enhanced

strategy could identify what other

infectious diseases (e.g., tuberculosis,

measles) may be applicable to exposure

notification apps in the future.

As part of this strategy, the federal

government could decide to repurpose

apps that were developed for state use

during the COVID-19 pandemic, or use

comparable technology to develop new

contact tracing solutions. Policymakers

could recommend a national exposure

notification app that public health

authorities could decide to use based on

their individual needs, resulting in a generic

exposure notification app tailored to state

needs or a federally managed exposure

notification app that is made available for

states to use.

Exposure Notification GAO-21-104622 45

Opportunities

· Enhanced national coordination could

prompt faster deployment of apps in

the future if that coordination builds

upon the underlying infrastructure and

leverages the lessons learned from

COVID-19.

· A federally led national marketing

campaign with cohesive and coherent

messaging could result in wider

adoption of exposure notification apps.

Increased federal promotion and

support of the exposure notification

apps could potentially help with

increasing public trust in the apps.

· A national app could allow integration

of exposure notification capabilities

with other disease prevention and

response activities, such as test

scheduling or vaccine delivery

coordination.

Considerations

· Implementing a coordinated national

strategy would likely have associated

costs and require a source of sustained

funding during and after the pandemic.

· Without clear roles and responsibilities,

coordination culd be challenging. For

example, coordination of groups with

divergent perspectives and interests

may pose challenges to defining

outcomes and measuring performance

and effectiveness of apps.

· It is unclear whether the public would

be more or les likely to trust and use a

national exposre notification app than

one developed by their state

government. Some states, including

Virginia, Colorado, and California, have

passed state-wide privacy laws. Due to

the absence of a federal privacy law in

the U.S., the public may be less likely to

trust the federal government’s privacy

protections.

Exposure Notification GAO-21-104622 46

6 Agency and Expert Comments

We provided a draft of this report to the Departments of Health and Human Services (including

CDC and NIH), Homeland Security, and Commerce, Federal Communications Commission, and

Federal Trade Commission for their review. The agencies provided technical comments, which

we incorporated as appropriate. Representatives from Apple, the Association of Public Health

Laboratories, Google, Massachusetts Institute of Technology’s Lincoln Laboratory, and MITRE

Corporation also reviewed a draft of this product; we incorporated their technical comments as

appropriate.

We are sending copies of this report to the appropriate congressional committees and other

interested parties. In addition, the report is available at no charge on the GAO website at

https://www.gao.gov.

If you or your staff have any questions about this report, please contact Karen L. Howard at

(202) 512-6888 or how[email protected] or Vijay A. D’Souza at (202) 512-6240 or

[email protected]. Contact points for our Offices of Congressional Relations and Public Affairs

may be found on the last page of this report. GAO staff who made key contributions to this

report are listed in appendix III.

Karen L. Howard, PhD

Director,

Science, Technology Assessment, and Analytics

Vijay A. D’Souza

Director,

Information Technology and Cybersecurity

Exposure Notification GAO-21-104622 47

List of Congressional Addressees

The Honorable Patrick Leahy

Chairman

The Honorable Richard Shelby

Vice Chairman

Committee on Appropriations

United States Senate

The Honorable Ron Wyden

Chairman

The Honorable Mike Crapo

Ranking Member

Committee on Finance

United States Senate

The Honorable Patty Murray

Chair

The Honorable Richard Burr

Ranking Member

Committee on Health, Education, Labor,

and Pensions

United States Senate

The Honorable Gary C. Peters

Chairman

The Honorable Rob Portman

Ranking Member

Committee on Homeland Security and

Governmental Affairs

United States Senate

The Honorable Rosa L. DeLauro

Chairwoman

The Honorable Kay Granger

Ranking Member

Committee on Appropriations

House of Representatives

The Honorable Frank Pallone, Jr.

Chairman

The Honorable Cathy McMorris Rodgers

Republican Leader

Committee on Energy and Commerce

House of Representatives

The Honorable Bennie G. Thompson

Chairman

The Honorable John Katko

Ranking Member

Committee on Homeland Security

House of Representatives

The Honorable Carolyn B. Maloney

Chairwoman

The Honorable James Comer

Ranking Member

Committee on Oversight and Reform

House of Representatives

The Honorable Richard Neal

Chairman

The Honorable Kevin Brady

Republican Leader

Committee on Ways and Means

House of Representatives

The Honorable Morgan Griffith

Republican Leader

Subcommittee on Oversight and

Investigations

Committee on Energy and Commerce

House of Representatives

The Honorable Brett Guthrie

Republican Leader

Subcommittee on Health

Committee on Energy and Commerce

House of Representatives

The Honorable Michael C. Burgess

House of Representatives

Exposure Notification GAO-21-104622 48

Appendix I: Objectives, Scope, and Methodology

Objectives

We were asked to assess smartphone

applications (apps)—commonly referred to as

exposure notification apps—that are intended

to notify persons of potential exposure to

infectious diseases. This report discusses:

· the benefits and design of exposure

notification apps;

· the current level of deployment in the

U.S.;

· challenges affecting their use; and

· policy options that could help address key

challenges for future use.

Scope and methodology

To address these objectives, we reviewed

documentation and met with officials from

selected federal agencies and entities

involved in providing guidance, funding

research, and other efforts related to

exposure notification apps. These agencies

were:

· Centers for Disease Control and

Prevention (CDC) and National Institutes

of Health (NIH) within the Department of

Health and Human Services (HHS),

· Cybersecurity and Infrastructure Security

Agency, Science and Technology

Directorate, within the Department of

Homeland Security (DHS),

· National Institute of Standards and

Technology (NIST), Federal Trade

Commission (FTC), within the Department

of Commerce, and

· Federal Communications Commission

(FCC).

In addition, we interviewed representatives

from companies involved in the development

of exposure notification apps (Google, Apple,

and PathCheck Foundation); public health

organizations (Association of Public Health

Laboratories, Public Health Informatics

Institute, Council of State and Territorial

Epidemiologists, Association of State and

Territorial Health Officials); federally funded

research and development centers

(Massachusetts Institute of Technology’s

Lincoln Laboratory and MITRE Corporation);

and academic researchers from Oxford

University’s Big Data Institute, Nuffield

Department of Medicine. We identified these

entities through our interviews and document

reviews. During our interviews with officials

and representatives, we discussed topics such

as exposure notification app functionality;

benefits of its use; levels of deployment in the

U.S.; technological limitations; and challenges

to its development, deployment, and use. We

also obtained written responses from two

organizations: the National Association of

County and City Health Officials and Linux

Foundation Public Health.

Further, we conducted a literature search for

articles regarding exposure notification apps,

including their benefits, capabilities, and

challenges, as well as policy options

associated with the apps. A research librarian

conducted searches of various databases

including Inspec, Scopus, Policy File,

ProQuest’s COVID-19 Research Database, and

the Harvard Kennedy School’s Custom Google

Think Tank Search. We used synonyms of the

Exposure Notification GAO-21-104622 49

following search terms to identify relevant

articles: contact tracing, exposure

notification, Google Apple exposure,

application, app, system, platform, digital,

mobile, smartphone. We paired these search

terms with additional synonyms for privacy,

security, policy, legislation, opportunities, and

challenges. We considered articles that met

the following criteria: published from 2016

through January 2021 in academic journals,

working papers, trade journals, legislative

materials, and reports by government

agencies and nonprofit organizations. From

the results produced by this search, we

reviewed a selection of articles to provide an

overview and additional context for our

research objectives. We also used the results

to help inform our development of an

inventory of states by app deployment status,

among other sources noted below.

To identify the current level of deployment in

the U.S., we developed an inventory of

exposure notification apps that had been

deployed by U.S. states and territories as of

June 2021. We developed the inventory by:

· reviewing inventories that had been

developed by other organizations;

· reviewing state health department

websites related to COVID-19 to identify

whether they identified an available app,

or plans to deploy one;

· conducting Google searches; and

· reviewing Android and iPhone app stores.

The inventories included those developed by MIT Technology

Review, the Association of State and Territorial Health Officials,

and the Ada Lovelace Institute.

Specifically, whether states used the Association of Public

Health Laboratories’ National Key Server and Multi-tenant

Verification Server.

We also contacted several individual states to

verify their status in deploying an app. We

analyzed the inventory to identify the extent

to which states and territories had deployed

apps, the underlying technologies used (e.g.,

Bluetooth Low Energy), and the use of

national servers.

States that had an app in a

pilot phase at the time of our review were

included in the category of “states that had

not deployed an app as of June 2021.”

To obtain additional information associated

with the development and use of the apps,

we interviewed state public health officials

from a non-generalizable sample of nine

states that had an exposure notification app

as of January 1, 2021: Alabama, Colorado,

Connecticut, Minnesota, Nevada, North

Carolina, Pennsylvania, Virginia, and

Washington.

We selected this sample based

on deployment date, geographical

distribution, the number of COVID-19 cases

and deaths, and app developer. We aimed for

a selection of states that would allow for the

selected states to provide a broad overview

and context for assessing our engagement’s

research objectives. Because the selection

was based on a non-generalizable sample, the

results cannot be used to make inferences

about all states that had deployed an app. We

also received written feedback to structured

questions from two additional states

(Louisiana and Utah) that deployed apps in

the later stages of our evidence collection.

We also conducted a review of each of the

selected states apps on both a phone using

We considered a state as having an app if it had an official

application available for download or the state officially

supported an exposure notification system that exists in a

smartphone operating system. As of January 1, 2021, we had

identified 20 states with an exposure notification app. After we

selected our sample, we learned that one additional state—

Wisconsin—had deployed an app in late December 2020.

Exposure Notification GAO-21-104622 50

the iOS (iPhone 6) and Android (Samsung

Galaxy S9) operating systems. As a part of this

review, we reviewed the general functions,

features, and usability of the apps. To help

understand how privacy considerations

applied to the apps, we examined and

compared each state’s privacy policies with

recommended practices identified in federal

guidance, such as CDC’s guidelines for digital

tools.

In addition, to obtain perspectives from states

that had not deployed an app, we collected

information from a non-generalizable

selection of seven states that had not

deployed an app at the time of our review

(Montana, Nebraska, Oregon, Rhode Island,

South Carolina, Texas, and West Virginia),

which included an interview with one state

and written responses to a semi-structured

set of questions for the other six. We selected

these states based on geographical

distribution, suggestions from stakeholders

we interviewed, and information we gathered

during our review regarding challenges

certain states may have faced.

We identified policy options based on our

literature review and interviews with federal

agencies, the selected states, and other

stakeholders, including national health

organizations and researchers. We assessed

each policy option by identifying potential

benefits and considerations of implementing

them, as identified over the course of our

review. Based on the evidence collected, we

identified four policy options. The list is not

Centers for Disease Control and Prevention, Preliminary

Criteria for the Evaluation of Digital Contact Tracing Tools for

COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and

Guidelines for the Implementation and Use of Digital Tools to

Augment Traditional Contact Tracing, version 1.0 (Atlanta, Ga.:

Dec. 15, 2020).

intended to be inclusive of all potential policy

options and are neither recommendations to

federal agencies nor matters for

congressional consideration. They are also not

listed in any specific rank or order. We are not

suggesting that they be done individually or

combined in any particular fashion.

Additionally, we did not conduct work to

assess how well they may lead to a particular

outcome.

We conducted our work from November 2020

to September 2021 in accordance with all

sections of GAO’s Quality Assurance

Framework that are relevant to technology

assessments. The framework requires that we

plan and perform the engagement to obtain

sufficient and appropriate evidence to meet

our stated objectives and to discuss any

limitations to our work. We believe that the

information and data obtained, and the

analysis conducted, provide a reasonable

basis for any findings and conclusions in this

product.

Exposure Notification GAO-21-104622 51

Appendix II: Exposure Notification App Adoption Rates for Selected

U.S. States

As previously discussed, states may use the rate of adoption to measure the success of their

efforts to promote exposure notification apps. However, officials from the 11 states in our

review reported that they used differing methods to calculate adoption rates. App adoption

rates can be determined by dividing the total number of smartphone downloads, or

activations, (numerator) by the size of a given population (denominator). However, states use

different methods for determining the denominator, which affects the adoption rates.

Specifically, two states used the total state population, three used populations aged 18 or

older, and three used the percent of the population with a smartphone (either age 18 or older

or 18 to 65), according to state officials. These inconsistent methods impede comparative

assessments across states. The following table provides information on the apps deployed by

the nine U.S. states selected for our review, plus Louisiana and Utah.

Table 4: Reported exposure notification app adoption rates for 11 U.S. states, as of June 2021

State

Type

(custom or

Express)

App name

Launch

date

Reported

downloads and/or

estimated

activations

(numerator)

Reported

population

(denominator)

Reported

adoption

rate

Alabama

Custom

GuideSafe™

Aug.

2020

280,000

Not provided

20%

Colorado

Express

COVID

Exposure

Notifications

Oct.

2020

2,511,070

The combined

number of iOS and

Android

activations taken

from several

sources.

Not provided

42%

Connecticut

Express

COVID Alert

Nov.

2020

N/A

Louisiana

Custom and

Express

COVID

Defense

Jan.

2021

663,379

This includes the

combined number

of iOS and Android

app downloads for

the custom app, as

well as the

Android app

downloads and

the estimated

number of

activations for iOS

for the Express

option.

3,560,976

(18 and older)

Based on U.S.

Census Bureau

estimate for the

adult population.

19%

Exposure Notification GAO-21-104622 52

State

Type

(custom or

Express)

App name

Launch

date

Reported

downloads and/or

estimated

activations

(numerator)

Reported

population

(denominator)

Reported

adoption

rate

Minnesota

Custom and

Express

COVIDAware

Nov.

2020

1,419,232

This includes the

combined number

of iOS and Android

app downloads for

the custom app, as

well as the

Android app

downloads and

the estimated

number of

activations for iOS

for the Express

option.

5,600,000

(Total population)

Based on U.S.

Census Bureau

estimates.

25%

Nevada

Custom and

Express

COVID Trace

Aug.

2020

1,202,874

This includes the

combined number

of iOS and Android

app downloads for

the custom app, as

well as the

Android app

downloads and

the estimated

number of

activations for iOS

for the Express

option.

3,100,000

(80% of state

population, a

proxy for the

number of people

with

smartphones)

Based on U.S.

Census Bureau

estimates.

49%

North

Carolina

Custom

SlowCOVIDNC

Sept.

2020

854,802

The combined

number of iOS and

Android app

downloads.

9,472,502

(18 and older)

Based on North

Carolina’s Office

of State Budget

and Management

estimates for the

adult population.

Pennsylvania

Custom

COVID Alert

Sept.

2020

912,863

The combined

number of iOS and

Android app

downloads.

10,880,000

(18-65 years old)

Exposure Notification GAO-21-104622 53

State

Type

(custom or

Express)

App name

Launch

date

Reported

downloads and/or

estimated

activations

(numerator)

Reported

population

(denominator)

Reported

adoption

rate

Based on U.S.

Census Bureau

estimates for the

adult population

and Pew Research

Center estimates

that 85% of the

adult population

has a smartphone.

Utah

Express

UT Exposure

Notifications

Feb.

2021

N/A

Data are not

tracked. However,

officials estimated

there were

600,000

activations in the

initial phase of app

deployment.

N/A

Virginia

Custom and

Express

COVIDWISE

Aug.

2020

1,100,338

This includes the

combined number

of iOS and Android

app downloads for

the custom app, as

well as the

Android app

downloads and

the estimated

number of

activations for iOS

for the Express

option.

4,253,335

(18-65 years old)

Based on U.S.

Census Bureau

estimates for the

adult population

and Google and

Apple statements

that 80% of the

adult population

has a smartphone.

26%

Washington

Express

WA Notify

Nov.

2020

2,068,916

The number of

Android app

downloads and

the estimated

number of

activations for iOS.

6,115,500

(18 and older)

Based on

Washington’s

Office of Financial

Management

estimates for the

adult population

and Pew Research

Center estimates

that 85% of the

adult population

has a smartphone.

34%

Legend: N/A = A state that does not track the number of app downloads or its adoption rate.

Exposure Notification GAO-21-104622 54

Source: GAO compilation of data from selected states, related documents, interviews and other sources. I GAO-21-104622

Notes: The “as of “date for the data provided by the states ranged from May 31, 2021 to June 11, 2021.

For states using the Express option, daily activations for Android are from the Google Play app store, while the

Apple activations numbers are an estimate based on counting the number of times people accessed the app’s logo

image and using a multiplier provided by Google and Apple to estimate the number of people who go on to install

the Express option.

The adoption rates were provided to us by the states; we rounded the rates up to the nearest whole number. We

did not calculate the adoption rates; however, the rates can be calculated with the provided numerators and

denominators for the states that provided this information.

Exposure Notification GAO-21-104622 55

Appendix III: GAO Contacts and Staff Acknowledgments

GAO contacts

Karen L. Howard, PhD, (202) 512-6888 or [email protected]

Vijay A. D’Souza, (202) 512-6240 or [email protected]

Staff acknowledgments

In addition to the contacts named above, Sushil Sharma (Assistant Director), Neela Lakhmani

(Assistant Director), Eric Bachhuber (Analyst in Charge), Scott Borre (Analyst in Charge), Nora

Adkins, Daniel Emirkhanian, Donna Epler, Nancy Glover, Anika McMillon, Melissa Melvin,

Monica Perez-Nelson, Ben Shouse, Amber Sinclair, and Umesh Thakkar made key

contributions to this report. Rebecca Gertler, Tim Kinoshita, Eleni Orphanides, and Ethiene

Salgado-Rodriguez also contributed to this report.

(104622)

GAO’s Mission

The Government Accountability Office, the audit, evaluation, and investigative arm of Congress,

exists to support Congress in meeting its constitutional responsibilities and to help improve the

performance and accountability of the federal government for the American people. GAO

examines the use of public funds; evaluates federal programs and policies; and provides analyses,

recommendations, and other assistance to help Congress make informed oversight, policy, and

funding decisions. GAO’s commitment to good government is reflected in its core values of

accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s

website (https://www.gao.gov). Each weekday afternoon, GAO posts on its website newly

released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted

products, go to https://www.gao.gov and select “E-mail Updates.”

Order by Phone

The price of each GAO publication reflects GAO’s actual cost of production and distribution and

depends on the number of pages in the publication and whether the publication is printed in color

or black and white. Pricing and ordering information is posted on GAO’s website,

https://www.gao.gov/ordering.htm.

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money

order. Call for additional information.

Connect with GAO

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates.

Listen to our Podcasts and read The Watchblog.

Visit GAO on the web at https://www.gao.gov.

To Report Fraud, Waste, and Abuse in Federal Programs

Contact: Website: https://www.gao.gov/fraudnet/fraudnet.htm

Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional Relations

Nikki Clowers, Managing Director, [email protected], (202) 512-4400,

U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548

Public Affairs

Chuck Young, Managing Director, Yo[email protected], (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149, Washington, DC 20548

Strategic Planning and External Liaison

Stephen Sanford, Managing Director, [email protected], (202) 512-9715

U.S. Government Accountability Office, 441 G Street NW, Room 7B37N, Washington, DC 20548