What are the odds of a breakthrough infection?

Ample evidence shows that the COVID-19 vaccines dramatically reduce the risk of serious illness, but stories about breakthrough infections continue to spark fear among people who are vaccinated — and resistance among those who aren’t. 

President Joe Biden tried to put those fears in perspective by citing a noteworthy statistic in a Sept. 9 speech.

"Recent data indicates there’s only one confirmed positive case per 5,000 fully vaccinated Americans per day," he said. 

Throughout the pandemic, experts have stressed the difficulty of determining the exact number of vaccinated people who contract COVID-19. We wanted to understand where the figure came from, how it was calculated, and what it can tell us about the prevalence of breakthroughs. 

The statistic first appeared in an issue of The Morning, a newsletter published by the New York Times. The issue aimed to debunk widespread misconceptions about the likelihood of breakthrough infections. 

In order to arrive at the figure, Times reporter David Leonhardt calculated the daily average COVID-19 cases per number of residents in three locations: Virginia, Utah and King County in Washington state, which includes Seattle. What Leonhardt found was that the average vaccinated American has a roughly 1-in-5,000 chance of getting COVID-19 on a given day.

In areas with high vaccination rates such as Seattle, the likelihood of a vaccinated person contracting COVID-19 is probably even lower, around 1-in-10,000 per day. Unvaccinated people in King County were about 10 times as likely to contract COVID-19 as their vaccinated counterparts. 

The statistic "reinforces what we know from many other sources: Vaccines dramatically reduce the probability of infection," said Johns Hopkins epidemiologist Nikolas Wada. "The effect is even greater with respect to severe illness and death, and this remains true even with the dominance of the delta variant."

A new analysis from the Centers for Disease Control and Prevention has emerged since the Times’ piece was published, and it suggests an even lower average chance of a breakthrough. 

On Sept. 10, the day after Biden’s address, the CDC published a report analyzing vaccine effectiveness across 13 jurisdictions from April 4 to July 17. The CDC said in May that it's no longer investigating breakthrough cases that aren't severe, but the September analysis matches case surveillance data to vaccine registries from the 13 locations. Together, the jurisdictions accounted for a quarter of the U.S. population, including states with low vaccination rates like Alabama and Louisiana.

Jennifer Kates, Kaiser Family Foundation senior vice president, used the CDC data to calculate the chance of a breakthrough case across all jurisdictions. She found the chance from June 20 to July 17 was lower than what Leonhardt had found: about 1-in-5,000 per week, or about 1-in-35,000 per day. 

The numbers on breakthrough infection risk have limitations. 

"It's important to stress what this statistic doesn't tell you," said Wada. 

The first complicating factor is that breakthrough infections are likely undercounted and underreported. Since the vaccines are so powerful at weakening the symptoms of illness, it’s likely that many people experiencing a breakthrough infection might not even realize it. 

When reached by email, Leonhardt said he saw no reason to believe that health departments were undercounting the number of positive tests. By definition, however, they’re unable to count breakthrough cases that never get identified, a point which his article acknowledged but Biden did not. 

"It's hard to know how many of them there are, but I assume it's a significant number," Leonhardt told PolitiFact. "The good news is that many of the undetected cases are so mild that they are not likely to be contagious."

The difficulty of pinning down the actual number of breakthroughs reflects the fact that the vaccines are achieving their intended purpose: making disease symptoms less severe. 

"Preventing infection (is) a really nice added benefit of the vaccines, but that’s not their primary purpose," said Emily Gurley, an infectious-disease epidemiologist at the Johns Hopkins Bloomberg School of Public Health. "The primary purpose of vaccinations is to prevent severe disease and death should you be infected. We lose sight of that sometimes, particularly in public discussions."

Furthermore, methods of data collection vary across jurisdictions. Virginia, for example, has kept detailed records of coronavirus infections by vaccination status. By contrast, Florida, where cases of the virus have been surging, does not report the number of breakthrough cases. 

The second caveat to the statistic is that it’s an average that doesn’t capture variations within and between communities. This means the chance of experiencing a breakthrough infection is higher in places with the worst COVID-19 outbreaks. People in locations where the disease is more prevalent will simply encounter the virus more often than people in places that have the virus under control. 

On a similar note, the chance of experiencing a breakthrough infection will vary depending on the precautions someone takes, said Joshua Sharfstein, vice dean at the Bloomberg School. 

The breakthrough statistic is "helpful as a very broad ballpark but not something immutable about the virus, and it's not an assurance that behavior doesn't matter," he said. ​