Public Health Lessons Learned From Biases in Coronavirus Mortality Overestimation

This accepted version of the article may differ from the final published version. This is an Accepted Manuscript for Disaster Medicine and Public Health Preparedness as part of the Cambridge Coronavirus Collection DOI: 10.1017/dmp.2020.298 Public health lessons learned from biases in coronavirus mortality overestimation Ronald B. Brown, PhD School of Public Health and Health Systems University of Waterloo, Waterloo, ON N2L 3G1, Canada r26brown@uwaterloo.ca

Public health lessons learned from biases in coronavirus mortality overestimation

 

Ronald B. Brown, PhD | PDF here or here.

School of Public Health and Health Systems

University of Waterloo, Waterloo, ON N2L 3G1, Canada

r26brown@uwaterloo.ca

August 12, 2020

Abstract

In testimony before U.S. Congress on March 11, 2020, members of the House Oversight and Reform Committee were informed that estimated mortality for the novel coronavirus was ten times higher than for seasonal influenza. Additional evidence, however, suggests the validity of this estimation could benefit from vetting for biases and miscalculations.

The main objective of this article is to critically appraise the coronavirus mortality estimation presented to Congress. Informational texts from the World Health Organization and the Centers for Disease Control and Prevention are compared with coronavirus mortality calculations in Congressional testimony. Results of this critical appraisal reveal information bias and selection bias in coronavirus mortality overestimation, most likely caused by misclassifying an influenza infection fatality rate as a case fatality rate.

Public health lessons learned for future infectious disease pandemics include: safeguarding against research biases that may underestimate or overestimate an associated risk of disease and mortality; reassessing the ethics of fear-based public health campaigns; and providing full public disclosure of adverse effects from severe mitigation measures to contain viral transmission.

Keywords: coronavirus mortality overestimation; COVID-19; sampling bias; case fatality rate; infection fatality rate

Introduction

On September 23, 1998, the U.S. National Aeronautics and Space Administration (NASA) permanently lost contact with the $125 million Mars Climate Orbiter.1 A simple miscalculation, failure to convert English measurements to metric measurements, doomed the Mars space mission.2

A later investigation found that backup quality assurance procedures were not in place at NASA to catch and correct this simple miscalculation. Fast forward 22 years to another crisis involving a U.S. government agency:

On March 11, 2020, the U.S. Congress House Oversight and Reform Committee received information from the National Institute of Allergy and Infectious Diseases (NIAID) concerning the novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and coronavirus-disease 2019 (COVID-19).3 Based on the data available at the time, Congress was informed that the estimated mortality rate for the coronavirus was ten-times higher than for seasonal influenza, which helped launch a campaign of

  • social distancing,
  • organizational and
  • business lockdowns, and
  • shelter-in-place orders.

Previous to the Congressional hearing, a less severe estimation of coronavirus mortality appeared in a February 28, 2020 editorial released by NIAID and the Centers for Disease Control and Prevention (CDC). Published online in the New England Journal of Medicine (NEJM.org), the editorial stated:

“…the overall clinical consequences of Covid-19 may ultimately be more akin to those of a severe seasonal influenza (which has a case fatality rate of approximately 0.1%).”4

Almost as a parenthetical afterthought, the NEJM editorial inaccurately stated that 0.1% is the approximate case fatality rate of seasonal influenza.

By contrast, the World Health (WHO) reported that 0.1% or lower is the approximate influenza infection fatality rate,5 not the case fatality rate. To fully appreciate the significance of discrepancies in fatality rate usage by NIAID, the CDC, and the WHO, brief definitions of relevant epidemiological terms follow.

  • Case fatality rates (CFRs), infection fatality rates (IFRs), and mortality rates are used by epidemiologists to describe deaths during and after an infectious disease outbreak.
  • The CDC defined a mortality rate as the frequency of deaths within a time period relative to the size of a well-defined population.6

  • Patients may be classified as having an influenza-like illness (ILI) such as COVID-19 according to standard criteria in a case definition. 7

  • A case fatality rate (CFR) is defined as the proportion of deaths among confirmed cases of the disease. CFRs indicate the disease severity, while an infection fatality rate (IFR) is defined as the proportion of deaths relative to the prevalence of infections within a population.8

  • IFRs are estimated following an outbreak, often based on representative samples of blood tests of the immune system in individuals exposed to a virus. Estimation of the IFR in COVID-19 is urgently needed to assess the scale of the coronavirus pandemic.9

    • Because different types of fatality rates can vary widely, it is imperative to not confuse fatality rates with one another; else misleading calculations with significant consequences could result. Interestingly, a search of the keyword term “infection fatality rate” on the CDC website returned no matching results, nor was the epidemiological term located in the 511-page CDC publication, Principles of Epidemiology in Public Health Practice.

    This terminology omission, in conjunction with questionable use of fatality rate terminology in the NEJM editorial, raises red flags — warning of possible inaccuracies in the coronavirus mortality estimation presented to Congress. Similar to the need to vet for miscalculations that might have rescued NASA’s 1998 Mars mission, vetting the coronavirus mortality estimation for miscalculations and biases may benefit the validity of mortality conclusions. Therefore, the purpose of this article is to present an ad hoc critical appraisal of the coronavirus mortality estimation presented to U.S. Congress on March 11, 2020.

    Main

    Findings from a comparative analysis of selected video and texts are used in this article to critically appraise the validity of coronavirus mortality calculations presented in U.S. Congressional testimony. Critical appraisal is a process that judges the validity of scientific research evidence.10 Comparative analysis is a tool used in a grounded theory methodology to investigate an unexplored area through logical induction of coherent themes and explanations that are grounded in empirical evidence.11 Text from the February 2020 NEJM.org editorial and video of Congressional testimony are compared with reliable informational texts from the WHO and CDC. Inconsistencies, inaccuracies, biases, utilization, and consequences of the coronavirus mortality estimation are discussed.

    In NIAID testimony before the House Oversight and Reform Committee Hearing on Coronavirus response, Day 1,3 the Committee learned that mortality from seasonal influenza is 0.1%. Additionally, it was reported to Congress that the overall coronavirus mortality of about 2–3% had been reduced to 1% to take into account infected people who are asymptomatic or have mild symptoms. The adjusted mortality rate from coronavirus of 1% was then compared with the 0.1% mortality rate from seasonal influenza, and the conclusion was reported to the House Committee that the coronavirus was ten-times more lethal than seasonal influenza.

    In a comparative analysis with WHO and CDC documents, the coronavirus mortality rate of 2–3% that was adjusted to 1% in Congressional testimony is consistent with the coronavirus CFR of 1.8–3.4% (median 2.6%) reported by the CDC.12 Furthermore, the World Health Organization reported that the CFR of the H1N1 influenza virus is also 2–3%,13 similar to the unadjusted 2–3% CFR of the coronavirus reported in Congressional testimony, with no meaningful difference in mortality. As previously mentioned, the World Health Organization also reported that 0.1% is the IFR of seasonal influenza, not the CFR of seasonal influenza as reported in the NEJM editorial.

    Discussion

    Confusion between case fatality rates and infection fatality rates may seem trivial, and it is easy to overlook at first, but this confusion may have ultimately led to an unintentional miscalculation in coronavirus mortality estimation.

    IFRs from samples across the population include undiagnosed, asymptomatic, and mild infections, and are often lower compared to CFRs, which are based exclusively on relatively smaller groups of moderately to severely ill diagnosed cases at the beginning of an outbreak.

    Due to host defense mechanisms and autoimmunity provided by innate and adaptive immune responses,14 asymptomatic infections are often prevalent in influenza.15 With many asymptomatic infections already identified in COVID-19,16 it appears unlikely that the IFR in an influenza-like illness like COVID-19 would approximate the disease’s CFR. Presymptomatic infections can also lower the proportion of asymptomatic infections. For example, a CDC report found that asymptomatic individuals identified through reverse transcriptase-polymerase chain reaction (PCR) testing developed symptoms a week later, and those individuals were re-classified as having been presymptomatic at the time of testing. 17

    In Figure 1, four cases grouped in the dotted-line box are also included among seven infections, illustrating that all cases are infections but not all infections are cases, a potential point of confusion in media reports of COVID-19. For example, a high number of coronavirus infections were discovered in U.S. meat-packing plants in Iowa,17 but these infections were reported as cases in the media,18 potentially causing a type of information bias known as misclassification.19 Misclassification refers to “the erroneous classification of an individual, a value, or an attribute into a category other than that to which it should be assigned."20 This type of information bias in epidemiological research can lead to underestimates or overestimates of associated disease and mortality risks.[^20]

    CFRs and IFRs represent different segments of a targeted population and contain widely different proportions of nonfatal infections; therefore, misapplying findings or generalizing inferences between these two groups can cause a type of selection bias known as sampling bias21 or ascertainment bias.22 In this type of bias, people do not represent segments of the population to whom findings apply. Furthermore, “…comparisons of the CFR of one disease with the IFR of another are mostly useless,” 23 and sampling bias can lead to serious inaccuracies, as when Congress was informed that the coronavirus is ten-times more lethal than seasonal influenza.

    A comparison of coronavirus and seasonal influenza CFRs may have been intended during Congressional testimony, but due to misclassifying an infection fatality rate as a case fatality rate, the comparison turned out to be between an adjusted coronavirus CFR of 1% and an influenza IFR of 0.1%. Had the adjusted coronavirus mortality rate not been lowered from 3% to 1%, fatality comparisons of the coronavirus to the IFR of seasonal influenza would have increased from 10-times higher to 20- to 30-times higher. By then, epidemiologists might have been alerted to the possibility of a miscalculation in such an alarming estimation.

    Quality assurance

    Most people rely on trusted public health experts from organizations like the CDC to disseminate vital information on infectious diseases.24 Unfortunately, even experts can make simple miscalculations which can lead to catastrophic results. In the example of NASA’s lost Mars Climate Orbiter, the NASA board investigating the failed mission recognized that mistakes happen on projects, "However, sufficient processes are usually in place on projects to catch these mistakes before they become critical to mission success." The NASA board also recognized the importance of quality assurance procedures to prevent future failures. Of relevance, in 2018, the National Institute of Neurological Disorders and Stroke (NINDS) provided an exemplary definition of quality assurance (QA) in clinical and health sciences:

    "The objectives of QA procedures are to assure the accuracy and consistency of study data, from the original observations through the reporting of results and to ensure that study results are considered valid and credible within the scientific and clinical communities."25

    Similar to NASA’s quality assurance problems in 1998, quality assurance procedures at U.S. national public health organizations in 2020 may benefit from review and revision to prevent crucial mortality miscalculations of infectious diseases in the future. As a safeguard against misuse of fatality rates, and protection in the event of non-standardized or inter-organizational discrepancies in terminology, every fatality rate should clearly define the denominator of the rate as the specific group to whom fatalities apply, either to

    1. the total population in mortality rates, confirmed cases of a disease in CFRs, or
    2. individuals exposed to a viral infection in IFRs.

    Mitigation measures

    As the campaign to mitigate coronavirus transmission was implemented from March into May, 2020, expected coronavirus mortality totals in the United States appeared much lower than the overestimation reported in Congressional testimony on March 11.

    Compared to the most recent season of severe influenza A (H3N2) in 2017-2018,26 with 80,000 U.S. deaths reported by CDC officials,27 U.S. coronavirus mortality totals had just reached 80,000 on May 9, 2020.28

    By then, relative to the 2017-2018 influenza, it was clear that the coronavirus mortality total for the season would be nowhere near 800,000 deaths inferred from the ten-fold mortality overestimation reported to Congress. Even after adjusting for the effect of successful mitigation measures that may have slowed down the rate of coronavirus transmission, it seems unlikely that so many deaths were completely eliminated by a non-pharmaceutical intervention like social distancing, which was only intended to contain infection transmission, not suppress infections and related fatalities.29

    Also in early May, 2020, a New York State survey of 1,269 COVID-19 patients recently admitted to 113 hospitals found that most of the patients had been following shelter-in-place orders for six weeks, which raised state officials’ suspicions about social distancing effectiveness.30 Still, polls showed the public credited social distancing and other mitigation measures for reducing predicted COVID-19 deaths, and for keeping people safe from the coronavirus.31,32

    Surprisingly, disproportionate mortality increases in Italian and American healthcare facilities during the height of the COVID-19 outbreak were not unique; similar healthcare facility crises occurred during the 2016-2017 influenza season in Italy,33 and during the 2017- 2018 influenza season in the United States.34 Yet, these earlier outbreaks did not appear to receive the same intensive media coverage as COVID-19.

    Although media reports of new coronavirus infections reinforced the public’s belief that the virus was continuing to spread, greater levels of testing may have increased detection of infections that were already prevalent throughout the population. In addition, the accuracy of coronavirus tests rushed into production during the pandemic were unknown. 35

    PCR testing has been in use since the detection of the A (H5N1) influenza virus in 2005,36 but a serious limitation of PCR testing is that nucleic acid detection is not capable of determining the difference between infective and non-infective viruses.37 Moreover, the CDC modified criteria to record coronavirus mortality by including “probable” and “likely” deaths in the International Classification of Diseases code (ICD) for COVID-19. 38

    By June 21, new daily deaths from the coronavirus dropped to 267 in the United States, a 90% decrease from 2,693 daily deaths reported on April 21. However, confirmed cases in some areas increased as lockdowns lifted, 39 and total U.S. infections had reached 1,254,055 by June 21.[^29] Several reasons in addition to increased viral transmission could account for case increases. For example,

    • ill people may no longer fear going to hospitals as society reopens, 40 and
    • coronavirus testing may also result in greater differential diagnosis of SARS-CoV-2 infections from other common respiratory viral infections. 41

    With more reported cases of COVID-19 in younger people following reopening,42 case fatality rates could actually decline due to lower associated mortality risk in this age group. Furthermore, country comparisons of coronavirus case fatality rates are often confounded by numerous factors,43 including:

    • healthcare differences in case definitions,
    • access to quality treatment and reliable testing,
    • compliance with mitigation measures,
    • underlying health conditions;
    • demographic differences in age, race, socioeconomic status, and
    • population density;
    • geo-political differences including

      • climate,
      • seasonality,
      • environmental pollution,
      • social inequities and unrest,
      • personal liberties,
      • public health policies,
      • reliability in reporting valid government statistics of disease, and
      • lifestyle customs that affect physical and mental health, public sanitation, and personal hygiene.

    Ultimately, with a myriad of uncontrolled confounding factors, a serosurvey of representative samples of a population is a more reliable method to determine the true prevalence of coronavirus infections.

    Emerging confounding factors in the United States have also contributed to a rising mortality trend in ILIs such as COVID-19. For example, each year surviving members of the aging Baby-Boomer cohort of 76 million people born between 1946 and 1964 enter the hig-hrisk category for ILIs, increasing the burden placed on health care systems.44 Also, research shows that a warming trend in the Artic can lead to more extreme winter weather conditions, especially in the Eastern United States,45 which may play a role in rising mortality rates from ILIs during the influenza season.

    As health authorities responded to the COVID-19 pandemic by implementing lockdowns and other mitigation measures with minimal supporting evidence, scientists warned of “a fiasco in the making,”46 Caution was also raised against violations of fundamental principles of science and logic, such as the mistaken assumption that correlation implies causation.[^44]

    For example, the public’s belief that mitigation measures were responsible for reducing coronavirus mortality may be a post hoc fallacy if lower mortality was actually due to the overestimation of coronavirus deaths.

    Furthermore, implementing the unconfirmed hypothesis that mitigation measures save lives in vulnerable populations, and rejecting the null hypothesis that assumes no life-saving effect exists, is a type I error in hypothesis testing.47 The null hypothesis does not assume a priori knowledge. Therefore, before implementing mitigation measures that incur severe costs, the onus is on mitigation proponents to formally reject the null hypothesis by justifying claims of lifesaving benefits.

    Additionally, education in principles of basic research methods is essential for consumers of public health research, and there is a need to increase instruction in the science and logic of research methods in general education curricula.48 More research of non-drug mitigation interventions is also urgently needed to prevent COVID-19, especially in vulnerable populations.49

    Scientists also warned of public health decisions made without reliable data of infection prevalence within the population. Lacking valid input data due to insufficient testing for disease prevalence, statistical modelling methods often relied on speculative assumptions, producing fearful predictions of increased mortality which have often proved unreliable.50 A systematic review found that most diagnostic and predictive models for COVID-19 lack rigor, have a high risk of selection bias, and are likely to have lower predictive performance in actual practice compared to optimistic reports published in the research literature.51

    A revised version of a non-peer-reviewed study on COVID-19 antibody seroprevalence in Santa Clara County, California found that infections were many times more prevalent than confirmed cases.52 As more serosurveys are conducted throughout the country, a nationally coordinated COVID-19 serosurvey of a representative sample of the population is urgently needed,53 which can determine if the national infection fatality rate is low enough to expedite an across-the-board end to restrictive mitigating measures.

    Plans for a national U.S. serosurvey were announced in April 2020 by the National Institutes of Health, to be conducted by NIAID and the National Institute of Biomedical Imaging and Bioengineering (NIBIB), with the assistance of the National Center for Advancing Translational Sciences (NCATS) and the National Cancer Institute (NCI).54

    Of relevance, nationwide mitigation measures such as lockdowns, social distancing, and shelter-in-place orders were not implemented during the 2017- 2018 influenza with 45 million U.S. illnesses reported by the CDC. 55 Neither were mitigation measures implemented during the 2009 influenza, with reported estimates adjusted for underreported hospitalizations of approximately 60.8 million U.S. cases, ranging between 43.3 million to 89.3 million cases. 56

    Fear and collateral damage

    Psychological adverse effects such as anxiety, anger, and post-traumatic stress have been linked to restrictive public health mitigation measures due to isolation, frustration, financial loss, and fear of infection. 57,58 A June 8, 2020 survey from the Association for Canadian Studies found that fear of contracting the coronavirus affected 51% of the Canadian population, compared to 56% of the U.S. population.59 Venturing out into public during the reopening phase of the lockdown was stressful to 50% of Canadians compared to 56% of Americans. A second wave of the virus was also expected by 76% of Canadians and 64% of Americans. Furthermore, the possibility exists that yet another novel virus could emerge, potentially reigniting a perpetual process of unfounded fear and unnecessary lockdowns if mortality estimations are not properly vetted.

    Fear, in contrast to moral civic duty and political orientation, was shown to be a more powerful predictor of compliance with mitigating behavior in response to a viral pandemic, but with decreasing wellbeing and poorer decision making.60 Studies have shown that fear impairs performance of cognitive tasks through debilitating anxiety and worry. 61 Even if a threat ceases to exist, prolonged fearful avoidance of threats is maladaptive and restricts a return to normal social interaction and productivity.62 For example, after the outbreak of severe acute respiratory syndrome (SARS) had ended in 2004, avoidance behavior continued to restrict people’s social interactions and prevented people from returning to work.63

    Exaggerated levels of fear were driven by sensationalist media coverage during the COVID19 pandemic.64,65 And yet, while the public was ordered to lockdown, overall costs and benefits to society from severe mitigation measures had not been assessed. Fear of infection also prevented people from seeking needed healthcare services in hospitals during the pandemic.66 The ethics of implementing fear-based public health campaigns needs to be reevaluated for the potential harm these strategies can cause.67 Dissemination of vital health information to the public should employ emotionally persuasive messaging without exploiting and encouraging overreactions based on fear.

    In addition, legal and ethical violations associated with mitigation of pandemic diseases were previously investigated by the Institute of Medicine in 2007.68

    People should have the right to full disclosure of all information pertinent to adverse impacts of mitigation measures during a pandemic, including information on legal and constitutional human rights issues, and the public should be guaranteed a voice in a transparent process as authorities establish public health policy.

    Lastly, severe mitigating measures during the COVID-19 pandemic caused considerable global social and economic disruption.69 Enforced lockdowns increased domestic violence, closed businesses and schools, laid off workers, restricted travel, affected capital markets, threatened the security of low-income families, and saddled governments with massive debt.

    Between February and April 2020, U.S. unemployment rose from 3.5%, the lowest in 50 years to 14.7%.70 A recession in the United States was also officially declared in June 2020 by the National Bureau of Economic Research, ending 128 months of historic economic expansion. Of relevance, economic downturns are associated with higher suicide rates compared to times of prosperity, and increased suicide risk may be associated with economic stress as a consequence of severe mitigation measures during a pandemic.71 Relapses and newly diagnosed cases of alcohol use disorder were also predicted to increase due to social isolation, and harmful drinking in China increased two-fold following the COVID-19 outbreak.72 As a global natural experiment, psychological outcomes from restrictive interventions in the COVID-19 pandemic require further investigations. 73

    Public health lessons learned during the COVID-19 pandemic contribute knowledge and insights that can be applied to prevent future public health crises.74 Figure 2 shows a flow chart that summarizes biases and potential effects of viral mortality overestimation observed in a pandemic. Failure to intervene at the source of the problem, at the upstream levels of information bias and sampling bias, can allow fear to rapidly escalate and may cause an overactive response that produces severely harmful collateral damage to society.

    Conclusion

    Sampling bias in coronavirus mortality calculations led to a ten-fold increased mortality overestimation in March 11, 2020 U.S. Congressional testimony. This bias most likely followed from information bias due to misclassifying a seasonal influenza infection fatality rate as a case fatality rate, evident in a NEJM.org editorial. Evidence from the World Health Organization confirmed that the approximate case fatality rate of the coronavirus is generally no higher than that of seasonal influenza.

    By early May, 2020, mortality levels from COVID-19 were considerably below predicted overestimations, a result which the public attributed to successful mitigating measures to contain the spread of the novel coronavirus.

    This article presented important public health lessons learned from the COVID-19 pandemic. Reliable safeguards are needed in epidemiological research to prevent seemingly minor miscalculations from developing into disasters. Sufficient organizational quality assurance procedures should be implemented in public health institutions to check, catch, and correct research biases and mistakes that underestimate or overestimate associated risks of disease and mortality. Particularly, the denominator of fatality rates should clearly define the group to whom fatalities apply.

    Public health campaigns based on fear can have harmful effects, and the ethics of such campaigns should be reevaluated. People need to have a greater voice in a transparent process that influences public health policy during an outbreak, and educational curricula should include basic research methods to teach people how to be better consumers of public health information. The public should also be fully informed of the adverse impacts on psychological well-being, human rights issues, social disruption, and economic costs associated with restrictive public health interventions during a pandemic.

    In closing, nations across the globe may fearfully anticipate future waves of the coronavirus pandemic, and look bleakly toward outbreaks of other novel viral infections with a return to severe mitigation measures. However, well-worn advice from a famous aphorism by the poet philosopher George Santayana should be borne in mind, which is relevant to public health lessons learned in this article: “Those who cannot remember the past are condemned to repeat it.” 75

    Funding: This research received no external funding.

    Conflicts of Interest: The author declares no conflict of interest.

    Acknowledgments: Thank you to the following people who assisted in reviewing the material for this article: Nicole Richards, Mark Oremus, and Robin Taylor Wilson.

    References

    1. NASA. Mars Climate Orbiter. NASA Science Web site. https://solarsystem.nasa.gov/missions/mars-climate-orbiter/in-depth/. Published July 25, 2019. Accessed May 11, 2020.  ↩

    2. Isbell D, Savage D. Mars climate orbiter failure board releases report, numerous NASA actions underway in response. NASA Web site. https://mars.nasa.gov/msp98/news/mco991110.html. Published November 10, 1999. Accessed May 8, 2020.  ↩

    3. Fifer J. House Oversight and Reform Committee Hearing on Coronavirus Response, Day 1 C-SPAN Web site. https://www.c-span.org/video/?c4860450/user-clip-dr-anthonyfauci-addresses-covid-19-mortality-rate%20. Published March 11, 2020. Accessed April 30, 2020.  ↩

    4. Fauci AS, Lane HC, Redfield RR. Covid-19—navigating the uncharted. New England Journal of Medicine Web site. https://www.nejm.org/doi/full/10.1056/nejme2002387. Published February 28, 2020. Accessed May 8, 2020.  ↩

    5. who.int. Coronavirus disease 01 (C ID-1 ): situation report, 46. World Health Organization. https://www.who.int/docs/default-source/coronaviruse/situationreports/20200306-sitrep-46-covid-19.pdf?sfvrsn=96b04adf_4. Published 2020. Accessed April 30, 2020.  ↩

    6. Dicker R, Coronado F, Koo D, Parrish RG. Principles of epidemiology in public health practice. Atlanta GA: US Department of Health and Human Services. 2006.  ↩

    7. who.int. WHO surveillance case definitions for ILI and SARI. World Health Organization Web site. <https://www.who.int/influenza/surveillance_monitoring/ili_sari_surveillance_case_definit ion/en/. Published January 2014. Accessed April 26, 2020.  ↩

    8. Quoctrung Bui MS-K, Sarah Kliff. How deadly is coronavirus? What we know and what we don’t. The New York Times Web site. https://www.nytimes.com/interactive/2020/03/07/upshot/how-deadly-is-coronaviruswhat-we-know.html. Published 2020. Updated March 7. Accessed April 30, 2020.  ↩

    9. Mallapaty S. How deadly is the coronavirus? Scientists are close to an answer. nature Web site. https://www.nature.com/articles/d41586-020-01738-2. Published June 16, 2020. Accessed June 18, 2020.  ↩

    10. Mhaskar R, Emmanuel P, Mishra S, Patel S, Naik E, Kumar A. Critical appraisal skills are essential to informed decision-making. Indian journal of sexually transmitted diseases and AIDS. 2009;30(2):112.  ↩

    11. Chun Tie Y, Birks M, Francis K. Grounded theory research: A design framework for novice researchers. SAGE open medicine. 2019;7:2050312118822927.  ↩

    12. cdc.gov. Severe Outcomes Among Patients with Coronavirus Disease 2019 (COVID19)—United States, February 12–March 16, 2020. Morbidity and Mortality Weekly Report: Centers for Disease Control and Prevention Web site. https://www.cdc.gov/mmwr/volumes/69/wr/mm6912e2.htm. Published March 27, 2020. Accessed April 30, 2020.  ↩

    13. apps.who.int. Pandemic influenza risk management: a WHO guide to inform and harmonize national and international pandemic preparedness and response. World Health Organization. World Health Organization. https://apps.who.int/iris/handle/10665/259893. Published 2017. Accessed April 30, 2020.  ↩

    14. Aoshi T, Koyama S, Kobiyama K, Akira S, Ishii KJ. Innate and adaptive immune responses to viral infection and vaccination. Current opinion in virology. 2011;1(4):226- 232.  ↩

    15. Furuya-Kanamori L, Cox M, Milinovich GJ, Magalhaes RJS, Mackay IM, Yakob L. Heterogeneous and dynamic prevalence of asymptomatic influenza virus infections. Emerging infectious diseases. 2016;22(6):1052.  ↩

    16. Gao Z, Xu Y, Sun C, et al. A systematic review of asymptomatic infections with COVID19. Journal of Microbiology, Immunology and Infection. 2020.  ↩

    17. Leys T. Coronavirus infects more than 1,600 workers at four Iowa meatpacking plants. Des Moines Register Web site. https://www.desmoinesregister.com/story/news/health/2020/05/05/coronavirus-infectsthousands-iowa-meatpacking-plant-workers-covid-19-waterloo-perry/5170796002/. Published May 5, 2020. Accessed May 5, 2020.  ↩

    18. Maddow R. HHS Statement TRANSCRIPT: 5/5/20. The Rachel Maddow Show Web site. http://www.msnbc.com/transcripts/rachel-maddow-show/2020-05-05. Published May 5, 2020. Accessed May 5, 2020.  ↩

    19. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. Journal of multidisciplinary healthcare. 2016;9:211.  ↩

    20. Misclassification. In: Porta M, ed. A Dictionary of Epidemiology. 5 ed: Oxford University Press: New York, NY; 2008.  ↩

    21. Johnson LC, Beaton R, Murphy S, Pike K. Sampling bias and other methodological threats to the validity of health survey research. International Journal of Stress Management. 2000;7(4):247-267.  ↩

    22. Delgado-Rodriguez M, Llorca J. Bias. Journal of Epidemiology & Community Health. 2004;58(8):635-641.  ↩

    23. Roth S, Clausen L, Mueller S. COVID-19. Scenarios of a Superfluous Crisis. SSRN Web site. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3564920. Published 2020 March 30. AccessedApril 30, 2020.  ↩

    24. Fingerhut H. AP-NORC poll: High use, mild trust of news media on COVID-19. AP Web site. https://apnews.com/4e2a20bd01bd2352009c3281b657375d. Published April 30, 2020. Accessed June 9, 2020.  ↩

    25. NINDS. Quality Assurance Guidelines. National Institute of Neurological Disorders and Stroke Web site. https://www.ninds.nih.gov/Funding/Apply-Funding/ApplicationSupport-Library/Quality-Assurance-Guidelines. Published August 13, 2018. Accessed May 11, 2020.  ↩

    26. cdc.gov. Summary of the 2017–2018 influenza season. Centers for Disease Control and Prevention Web site. https://www.cdc.gov/flu/about/season/flu-season-2017-2018.htm. Published September 5, 2019. Accessed May 5, 2020.  ↩

    27. AP. CDC: 80,000 people died of flu last winter in U.S., highest death toll in 40 years. 2018. STAT Web site. https://www.statnews.com/2018/09/26/cdc-us-flu-deaths-winter/. Published September 26, 2018. Accessed May 12, 2020.  ↩

    28. worldometers.info. United States. Worldometer Web site. https://www.worldometers.info/coronavirus/country/us/. Published June 21, 2020. Accessed June 21,, 2020.  ↩

    29. Ferguson N, Laydon D, Nedjati Gilani G, et al. Report 9: Impact of non-pharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcare demand. 2020.  ↩

    30. Matthews K, Villeneuve M, Hill M. Most new COVID-19 patients in NY not working, older: survey. Associated Press Web site. https://apnews.com/29033e79ac5a3cf66b0c1fd2422e729d. Published May 7, 2020. Accessed May 7, 2020.  ↩

    31. Benzie R. Most Canadians support COVID-19 measures, but the pandemic is wearing us down, poll finds. The Star Web site. https://www.thestar.com/politics/provincial/2020/05/05/most-canadians-support-covid19-measures-but-the-pandemic-is-wearing-us-down-poll-finds.html. Published 2020. Accessed May 6, 2020.  ↩

    32. Chappell B. 8 in 10 Americans support COVID-19 shutdown, Kaiser Health Poll finds. NPR Web site. https://www.npr.org/sections/coronavirus-liveupdates/2020/04/23/843175656/8-in-10-americans-support-covid-19-shutdown-kaiserhealth-poll-finds. Published 2020. Accessed May 6, 2020.  ↩

    33. Rosano A, Bella A, Gesualdo F, et al. Investigating the impact of influenza on excess mortality in all ages in Italy during recent seasons (2013/14–2016/17 seasons). International Journal of Infectious Diseases. 2019;88:127-134.  ↩

    34. Macmillan A. Hospitals Overwhelmed by Flu Patients Are Treating Them in Tents. TIME Web site. https://time.com/5107984/hospitals-handling-burden-flu-patients/. Published January 8, 2018. Accessed May 7, 2020.  ↩

    35. Perrone M. Accuracy still unknown for many coronavirus tests rushed out. AP Web site. https://apnews.com/38967d1644ef374d1bcbc7806b383b8e. Published June 14, 2020. Accessed June 15, 2020.  ↩

    36. WHO. The use of PCR in the surveillance and diagnosis of influenza. Paper presented at: Report of the 4th meeting of the WHO working group on polymerase chain reaction protocols for detecting subtype influenza A viruses2011  ↩

    37. Joynt GM, Wu WK. Understanding COVID-19: what does viral RNA load really mean? The Lancet Infectious Diseases. 2020;20(6):635-636. ↩

    38. Schwartz S. COVID-19 Alert No. 2. Centers for Disease Control and Prevention Web site. https://www.cdc.gov/nchs/data/nvss/coronavirus/Alert-2-New-ICD-code-introducedfor-COVID-19-deaths.pdf. Published March 24, 2020. Accessed June 17, 2020.  ↩

    39. Cohen H. With 2,581 new coronavirus cases, Florida hits record high for the third day in a row. Miami Herald Web site. https://www.miamiherald.com/news/coronavirus/article243513417.html. Published June 13, 2020. Accessed June 15, 2020.  ↩

    40. White E. Patients slowly returning to hospital emergeny rooms in northeastern Ontario. CBC Web site. https://www.cbc.ca/news/canada/sudbury/emergency-room-hospitalscovid-northeastern-ontario-1.5584581. Published May 27, 2020. AccessedJune 14, 2020.  ↩

    41. Singhal T. A review of coronavirus disease-2019 (COVID-19). The Indian Journal of Pediatrics. 2020:1-6. ↩

    42. Samore P. Arizona hospitals getting fuller as COVID-19 patients getting younger. KTAR NEWS Web site. https://ktar.com/story/3291284/arizona-hospitals-getting-fuller-ascovid-19-patients-getting-younger/. Published June 17, 2020. Accessed June 17, 2020.  ↩

    43. James JJ. Lockdown or Lockup. Disaster Medicine and Public Health Preparedness. 2020:1-4. ↩

    44. Henderson L, Maniam B, Leavell H. The Silver Tsunami: Evaluating the Impact of Population Aging in the US. Journal of Business and Behavioral Sciences. 2017;29(2):153-169.  ↩

    45. Cohen J, Pfeiffer K, Francis JA. Warm Arctic episodes linked with increased frequency of extreme winter weather in the United States. Nature communications. 2018;9(1):1-12.  ↩

    46. Ioannidis JP. A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data. Stat Web site. https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronaviruspandemic-takes-hold-we-are-making-decisions-without-reliable-data/. Published 2020. Accessed March 17, 2020.  ↩

    47. Randolph C. Remember the null hypothesis? In: BMJ Publishing Group Ltd; 2020 ↩

    48. Aguado NA. Teaching research methods: Learning by doing. Journal of Public Affairs Education. 2009;15(2):251-260. ↩

    49. Glasziou PP, Sanders S, Hoffmann T. Waste in covid-19 research. BMJ. 2020;369:m1847.  ↩

    50. Begley S. Influential Covid-1 model uses flawed methods and shouldn’t guide U.S. policies, critics say. Stat Web site. https://www.statnews.com/2020/04/17/influentialcovid-19-model-uses-flawed-methods-shouldnt-guide-policies-critics-say/. Published 2020. Accessed May 6, 2020.  ↩

    51. Wynants L, Van Calster B, Bonten MM, et al. Prediction models for diagnosis and
      prognosis of covid-19 infection: systematic review and critical appraisal. bmj. 2020;369. ↩

    52. Bendavid E, Mulaney B, Sood N, et al. COVID-19 Antibody Seroprevalence in Santa Clara County, California. medRxiv. 2020.  ↩

    53. McClellan M, Gottlieb S, Mostashari F, Rivers C, Silvis L. A national COVID-19 surveillance system: Achieving containment. Margolis Center for Health Policy Web site. https://healthpolicy.duke.edu/sites/default/files/atoms/files/a_national_covid_surveillance_system_withcover.pdf. Published April 7, 2020. Accessed May 8, 2020.  ↩

    54. nih.gov. NIH begins study to quantify undetected cases of coronavirus infection. National Institutes of Health Web site. https://www.nih.gov/news-events/news-releases/nihbegins-study-quantify-undetected-cases-coronavirus-infection. Published April 10, 2020. Accessed June 10, 2020.  ↩

    55. cdc.gov. Estimated influenza illnesses, medical visits, hospitalizations, and deaths in the United States—2017–2018 influenza season. Centers for Disease Control and Prevention, Atlanta, GA. Web site. https://www.cdc.gov/flu/about/burden/2017-2018.htm. Published 2018. Accessed May 21, 2020.  ↩

    56. Shrestha SS, Swerdlow DL, Borse RH, et al. Estimating the burden of 2009 pandemic influenza A (H1N1) in the United States (April 2009–April 2010). Clinical Infectious Diseases. 2011;52(suppl_1):S75-S82.  ↩

    57. Brooks SK, Webster RK, Smith LE, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. The Lancet. 2020.  ↩

    58. Hoof EV. COVID-19 lockdown is the world’s biggest psychological experiment – and we will pay the price. World Economic Forum Web site. https://www.weforum.org/agenda/2020/04/this-is-the-psychological-side-of-the-covid19-pandemic-that-were-ignoring/. Published April 9, 2020. Accessed May 5, 2020.  ↩

    59. acs-aec.ca. Leger National Weekly Pandemic Tracker. Association for Canadian Studies Web site. https://acs-aec.ca/wp-content/uploads/2020/06/Leger-National-weeklypandemic-tracker-june-8th-2020.pdf. Published June 8, 2020. Accessed June 10, 2020.  ↩

    60. Harper CA, Satchell LP, Fido D, Latzman RD. Functional fear predicts public health compliance in the COVID-19 pandemic. International Journal of Mental Health and Addiction Web site. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7185265/. Published April 27, 2020. Accessed May 8, 2020.  ↩

    61. Zhuang Q, Wang L, Tang Y, Chen A. Translation of fear reflex into impaired cognitive function mediated by worry. Science Bulletin. 2016;61(24):1841-1843.  ↩

    62. Sangha S, Diehl MM, Bergstrom HC, Drew MR. Know safety, no fear. Neuroscience & Biobehavioral Reviews. 2020;108:218-230.  ↩

    63. Usher K, Bhullar N, Jackson D. Life in the pandemic: Social isolation and mental health. Journal of Clinical Nursing. 2020.  ↩

    64. Oprysko C. Fauci: Americans should be prepared to ‘hunker down’ even more. POLITICO Web site. https://www.politico.com/news/2020/03/15/coronavirus-fauciamericans-hunker-down-130035. Published March 15, 2020. AccessedMay 11, 2020. ↩

    65. Budrick Z. Fauci: ‘If it looks like you’re overreacting, you’re probably doing the right thing’. The Hill Web site. https://thehill.com/homenews/sunday-talk-shows/487639-fauciif-it-looks-like-youre-overreacting-youre-probably-doing-the. Published March 15, 2020. Accessed April 30, 2020.  ↩

    66. McFarling UL. ‘Where are all our patients?’: Covid phobia is keeping people with serious heart symptoms away from ERs. STAT Web site. https://www.statnews.com/2020/04/23/coronavirus-phobia-keeping-heart-patients-awayfrom-er/. Published April 23, 2020. Accessed May 12, 2020.  ↩

    67. Bayer R, Fairchild AL. Means, ends and the ethics of fear-based public health campaigns. Journal of medical ethics. 2016;42(6):391-396.  ↩

    68. Gostin L, Berkman B. Preparing for the pandemic influenza: legal and ethical challenges. In: Ethical and Legal Considerations in Mitigating Pandemic Diseases: Workshop Summary. Washington, DC: Institute of Medicine. Vol 80.2007.  ↩

    69. Nicola M, Alsafi Z, Sohrabi C, et al. The Socio-Economic Implications of the Coronavirus and COVID-19 Pandemic: A Review. International Journal of Surgery. 2020.  ↩

    70. Davidson P. It’s official: The US is in a recession, ending longest expansion in history. USA Today Web site. https://www.usatoday.com/story/money/2020/06/08/recessionbegins-us-ending-longest-expansion-history/5320335002/. Published June 8, 2020. Accessed June 10, 2020.  ↩

    71. Reger MA, Stanley IH, Joiner TE. Suicide mortality and coronavirus disease 2019—a perfect storm? JAMA psychiatry. 2020.  ↩

    72. Da BL, Im GY, Schiano TD. C ID‐1 Hangover: A Rising Tide of Alcohol Use Disorder and Alcohol‐Associated Liver Disease. Hepatology. 2020.  ↩

    73. Patrick SL, Cormier HC. Are Our Lives the Experiment? COVID-19 Lessons during a Chaotic Natural Experiment–A Commentary. Health Behavior and Policy Review. 2020;7(2):165-169. ↩

    74. Secchi P, Ciaschi R, Spence D. A concept for an ESA lessons learned system. Paper presented at: Proceedings of Alerts and LL: An Effective way to prevent failures and problems1999.  ↩

    75. Santayana G. The life of reason: Vol 1: Reason in common sense. Dover Publications Incorporated; 1905. ↩