Cleveland Clinic study didn’t find that taking more COVID-19 vaccine doses causes increased COVID-19 risk; association alone doesn’t imply causation

The claim that a Cleveland Clinic study found taking more COVID-19 vaccine doses increases a person’s risk of COVID-19 went viral in early June 2023. An article by the Gateway Pundit containing this claim received more than 5,800 engagements on Facebook and Twitter, according to the social media analytics tool CrowdTangle. The Gateway Pundit has a history of publishing inaccurate information and conspiracy theories, including falsely alleging that the Parkland high school shooting victims were “crisis actors”. Podcaster and comedian Joe Rogan made the same claim on his podcast “The Joe Rogan Experience” on 8 June 2023, citing the Gateway Pundit article as evidence. 

Other websites like Children’s Health Defense and the media outlet The Hill also made the same claim. The Hill’s Facebook video containing the claim received more than 67,000 views to date.

However, the Cleveland Clinic study doesn’t provide sufficient evidence to support the claim and several other published studies contradict the claim, as we will explain below.

What the study did

A group of researchers at the Cleveland Clinic in Ohio set out to evaluate the effectiveness of the bivalent COVID-19 vaccine against disease. The bivalent vaccine targets the original strain of SARS-CoV-2 isolated in Wuhan, China, as well as the more contagious Omicron variant.

To do this, they monitored more than 50,000 employees of the Cleveland Clinic for 26 weeks, starting from when the bivalent vaccine became available in September 2022. Among these employees, there were 4,424 (8.7%) detected COVID-19 cases. By comparing cases in people who received the bivalent vaccine with those who didn’t, they estimated that bivalent vaccine effectiveness against disease was 29% while the BA.4/5 variant was dominant, but just 4% with the later XBB variant^[1].

Both BA.4/5 and XBB are subtypes of the Omicron variant, although the XBB variant carries new mutations that help it to evade the immune system more easily, which may explain the lower vaccine effectiveness against disease observed with this variant.

In addition to this, the researchers also reported that the likelihood of getting COVID-19 was associated with the number of vaccine doses received and that this was statistically significant, a finding that they called “unexpected”. It is this finding that forms the basis of the claim made in the video by The Hill and the articles by Children’s Health Defense and the Gateway Pundit. These posts downplay or omit any mention of the main finding of the study: that bivalent boosters increase protection against COVID-19.

Association alone doesn’t indicate causation

Health Feedback reached out to the lead author of the study, infectious disease physician Nabin Shrestha, who clarified in an email that “association is not causation”, and that “Any claim that our study shows a causal relationship between getting more doses of the COVID-19 vaccine and higher risk of infection is false”.

Indeed, while an association can be a sign that two variables are causally related to each other, there can also be other reasons for why an association is present.

In the study’s Discussion section, the authors suggested that “It is possible that the association of number of prior vaccine doses with increased risk of infection may have been confounded by time since last prior exposure to SARS-CoV-2.” Previous exposure to SARS-CoV-2 could generate some degree of natural immunity, although this wanes over time, thus the time since a person’s last exposure can also affect COVID-19 risk.

Another possible explanation put forward by other scientists is that people who decided to get more doses of COVID-19 vaccine were also more likely to get tested for COVID-19 when they developed symptoms compared to people who chose to get fewer doses or no dose at all. Thus differences in health seeking behavior could give rise to testing bias, leading more cases to be detected among those who received more vaccine doses.

James Lawler, a professor of infectious diseases at the University of Nebraska, told fact-checking group Lead Stories that “Having a positive PCR for COVID is very much dependent on presenting yourself for testing […] the vast majority of COVID infections in late 2022 never presented themselves for testing. That fact presents tremendous potential bias for a retrospective study”.

Jeffrey Morris, a biostatistician and professor at the University of Pennsylvania, also raised similar concerns in this tweet.

That said, the authors of the study stated that “We did not find an association between the number of COVID-19 tests done and the number of prior vaccine doses, however, suggesting that this was not a confounding factor”, although this information didn’t appear in the tables of statistics presented in the study.

Apart from this, people who receive more doses of the vaccine may simply be more likely to have patient-facing jobs—and therefore more often exposed to COVID-19—compared to those who receive fewer doses, a point highlighted by Greg Tucker-Kellogg, a professor of biology at the National University of Singapore.

However, the study didn’t report the proportion of people who received three or more doses who were also holding patient-facing jobs. We also don’t know the proportion of COVID-19 cases that occurred in people holding clinical jobs. 

Thus the observed association between COVID-19 risk and the number of COVID-19 vaccine doses may be the result of people at higher risk of COVID-19 due to their work being more likely to receive more doses. Since the study didn’t examine this, this possibility cannot be ruled out.

In brief, there are several potential confounding factors in the study that aren’t fully accounted for, such as whether those who worked in clinical jobs were also more likely to get more doses and more likely to get tested for COVID-19. Having such information would provide a more reliable basis on which to assess the nature of the association between the number of COVID-19 doses and COVID-19 risk.

Far from asserting that getting more COVID-19 vaccine doses causes a person’s risk of COVID-19 to increase, the authors concluded that “The unexpected finding of increasing risk with increasing number of prior COVID-19 vaccine doses needs further study”, in keeping with the limitations of the study.

It’s not the first time that this study formed the basis of misinformation. Early in 2023, the study, then a preprint (a study that isn’t yet peer-reviewed), was incorrectly cited by Florida Governor Ron DeSantis as evidence that bivalent booster doses increased a person’s risk of COVID-19, a claim that contradicts that primary findings of the study.

Several published studies showed that booster doses enhance protection against COVID-19, not the opposite

The claim that more booster doses increases a person’s risk of COVID-19 is contradicted by several published studies, which found that booster doses enhance protection against COVID-19.

A study by the U.S. Centers for Disease Control and Prevention observed that booster doses improved protection against symptomatic infection caused by the XBB variant in previously vaccinated people in the general population, for at least the first three months post-vaccination^[2]. It observed that vaccine effectiveness against the BA.5 variant was 52% in those aged 18 to 49 and 49% against the XBB variant, although effectiveness declined with age.

Another study published in JAMA Network, which included patients hospitalized between October 2021 and July 2022, found that those who were boosted had their likelihood of hospitalization reduced by about 50% compared to those who received only the primary vaccination series^[3].

In a letter to the New England Journal of Medicine, researchers in North Carolina reported that booster doses increased protection against severe disease^[4], with vaccine effectiveness at 24.9% for a single monovalent booster (a booster targeting only the original strain) and 61.8% for a single bivalent booster dose, although they also noted that the protection waned after four weeks.

While the Cleveland Clinic study’s results being at odds with other published studies doesn’t necessarily mean something is wrong with the study or that it should be dismissed, the fact that its results weren’t reproduced by other studies and are unexpected does call for caution when interpreting its results.

Conclusion

In summary, the claim that a Cleveland Clinic study found more COVID-19 vaccine doses increases a person’s risk of getting COVID-19 is misleading. It fails to take into account several issues, such as the fact that the study was designed to evaluate bivalent vaccine effectiveness. It wasn’t equipped to determine whether the number of COVID-19 vaccine doses influences COVID-19 risk, therefore several potential confounding factors weren’t accounted for. The claim also doesn’t take into account several other published studies which have found that more vaccine doses enhances protection against COVID-19, giving users a one-sided view of our current state of understanding on the subject.

REFERENCES

• 1 – Shrestha et al. (2023) Effectiveness of the Coronavirus Disease 2019 Bivalent Vaccine. Open Forum Infectious Diseases.
• 2 – Link-Gelles et al. (2023) Early Estimates of Bivalent mRNA Booster Dose Vaccine Effectiveness in Preventing Symptomatic SARS-CoV-2 Infection Attributable to Omicron BA.5– and XBB/XBB.1.5–Related Sublineages Among Immunocompetent Adults — Increasing Community Access to Testing Program, United States, December 2022–January 2023. Morbidity and Mortality Weekly Report.
• 3 – Ridgway et al. (2022) Odds of Hospitalization for COVID-19 After 3 vs 2 Doses of mRNA COVID-19 Vaccine by Time Since Booster Dose. JAMA Network.
• 4 – Lin et al. (2023) Effectiveness of Bivalent Boosters against Severe Omicron Infection. New England Journal of Medicine.