What to know about the study on lockdowns and COVID-19 deaths by economists

Fox News and other outlets are touting a new study that looked at how early COVID-19 lockdowns impacted mortality of the disease. The stories come with bombshell headlines that say research conducted by Johns Hopkins University found that lockdowns barely reduced deaths.

"Johns Hopkins Study: Lockdowns Had ‘Little To No Effect On COVID-19 Mortality’ But Had ‘Devastating’ Effects On Society," read a headline by the Daily Wire.

After publishing a similar story about the study, Fox News shared a second article berating news outlets such as the New York Times, CNN, MSNBC and the Washington Post for ignoring it, calling it "a full-on media blackout."

But the study is not definitive. The research represents a non-peer-reviewed "working paper" by three economists affiliated with Johns Hopkins University. The institution did not endorse the study.

"The university hasn’t taken a position on this and typically wouldn’t in such a situation," university spokesperson Jill Rosen wrote in an email. 

"The working paper is not a peer-reviewed scientific study, and its authors are not medical or public health researchers," Joshua Sharfstein, vice dean of the Johns Hopkins Bloomberg School of Public Health, told PolitiFact in an emailed statement. "To reach their conclusion that ‘lockdowns’ had a small effect on mortality, the authors redefined the term ‘lockdown’ and disregarded many peer-reviewed studies. The working paper did not include new data, and serious questions have already been raised about its methodology."

He added: "COVID-19 is caused by SARS-CoV-2, a respiratory virus transmitted between people. Reducing transmission of the virus leads to fewer cases, hospitalizations, and deaths. Early in the pandemic, when so little was known about COVID-19, stay-at-home policies kept the virus from infecting people and saved many lives."

The paper’s authors — Jonas Herby, Lars Jonung and Steve H. Hanke — all come from an economics background. Hanke, a senior fellow at the CATO Institute, has aired opinions about lockdowns and "fascist" vaccine policies on Twitter and has repeatedly elevated false claims about the pandemic. Hanke is the only one affiliated with the university.

In June 2020, 285 public health researchers, professionals and citizens signed an open letter to Johns Hopkins University that demanded Hanke apologize after he wrongly tweeted that Vietnam hadn’t reported any COVID-19 data. The letter claimed his post was more politically driven than evidence-based. He later deleted the tweet. 

Once the working paper started making headlines for its finding that lockdowns only "reduced COVID-19 deaths by 0.2%," multiple public health experts and researchers released statements about the paper's methodology being flawed.

They pointed to issues such as the authors’ broad definition of lockdown, a limited focus on the first wave of the pandemic, a comparison of policies that look different from country to country, an exclusion of studies that look at the science of disease transmission, and a failure to consider the lag between infections and deaths.

Bill Hanage, a professor of epidemiology at Harvard T.H. Chan School of Public Health, said the question of whether lockdowns decrease death is in itself not sensible.

"The whole premise of it is wrong, because given enough time and no vaccines, if the virus infects enough people, they will die," Hanage said. "These interventions are designed to try and mitigate that so that they don't all get sick at the same time, so it’s completely mistaken."

The authors of the paper did not respond to a request for comment. 

The dictionary definition of a lockdown is "a mandatory state of isolation or restricted access instituted as a security measure." During the pandemic, a lockdown largely referred to a temporary condition imposed by governmental authorities in which people were required to stay in their homes and refrain from or limit activities outside the home involving public contact, such as dining out or attending large gatherings.

Instead, the authors defined lockdown as "the imposition of at least one compulsory, non-pharmaceutical intervention." This means that a region’s mask-wearing policy would be considered a lockdown, some experts pointed out, even though the policy doesn’t stop people from venturing out and frequenting public places.

"This report on the effect of ‘lockdowns’ does not significantly advance our understanding of the relative effectiveness of the plethora of public health measures adopted by different countries to limit COVID-19 transmission. The policies which comprised ‘lockdown’ varied dramatically between countries, meaning defining the term is problematic," Neil Ferguson, director of the MRC Centre for Global Infectious Disease Analysis at Imperial College London, wrote in a reaction paper for the Science Media Centre.

By the authors’ definition of lockdown, Ferguson noted, "the UK has been in permanent lockdown since 16th of March 2021, and remains in lockdown — given it remain compulsory for people with diagnosed COVID-19 to self-isolate for at least 5 days."

The authors said their paper analyzed and reviewed previous studies that addressed the subject of lockdowns’ impact on COVID-19 mortality. They identified 18,590 potential studies, of which 34 ultimately qualified.

"We only include studies that attempt to establish a relationship (or lack thereof) between lockdown policies and COVID-19 mortality or excess mortality," the paper said. "We exclude studies that use cases, hospitalizations, or other measures."

Many have taken issue with this decision. Dr. Seth Flaxman, an associate professor of computer science at the University of Oxford wrote:

"They systematically excluded from consideration any study based on the science of disease transmission, meaning that the only studies looked at in the analysis are studies using the methods of economics.

"These do not include key facts about disease transmission such as: later lockdowns are less effective than earlier lockdowns, because many people are already infected; lockdowns do not immediately save lives, because there’s a lag from infection to death, so to see the effect of lockdowns on Covid deaths we need to wait about two or three weeks."

While the paper doesn’t account for a lag between infections and deaths, it also limited its review to studies done early in the pandemic, with little to no data related to more recent variants and waves. 

"Two years in, it seems still to focus on the first wave of SARS-CoV-2 and in a very limited number of countries," wrote Samir Bhatt, a professor of statistics and public health at Imperial College London. "As I have mentioned, it looks at a tiny slice of the pandemic, there have been many lockdowns since globally with far better data, there are many prominent studies that cover the period in question looking at infections … The list of such studies is very long and suggests a highly incomplete meta-analysis."

Several papers that looked at the same time period found that lockdown measures have helped save lives during the pandemic.

A June 2020 peer-reviewed study published in the journal Nature found that lockdowns prevented tens of millions of infections and saved millions of lives. 

"Our estimates show that lockdowns had a really dramatic effect in reducing transmission," Bhatt, who worked on the paper, told NPR shortly after it was published. Bhatt’s team analyzed infection and death rates in 11 European nations and estimated that an additional 3.1 million people in those countries would have died if lockdowns had not been put in place.

Nature also published an early study from the Global Policy Lab at the University of California, Berkeley, that analyzed lockdowns in China, South Korea, Iran, France, Italy and the U.S. It found that lockdowns in those countries averted around 62 million cases.

HealthFeedback, a science and fact-checking website, also found evidence in May 2021 that lockdowns saved lives.

The website found that the New York Post misrepresented an analysis by researchers at Rice University’s Baker Institute for Public Policy that used WalletHub’s ranking of U.S. states’ lockdowns from May 2020 to October 2020. It assigned a numerical value to the imposed restrictions and the ranking was correlated with the state’s average death rate over time.

According to the Post, the study found "little correlation at all between the strictness of lockdown measures and death rates." But the ranking had several limitations, including that it didn’t account for aspects like population density, use of public transportation, or the close living situations in urban areas, HealthFeedback found. 

The analysis ultimately reported that lockdowns were effective in both reducing daily deaths in highly infected states and in preventing new spikes in deaths. "Additionally, this trend implies that states that are more open are susceptible to higher COVID-19 death rates," it said.