COVID-19 vaccines not shown to have “negative efficacy”, contrary to claim by Nicolas Hulscher

In January 2025, a Facebook post claimed that COVID-19 vaccines had “negative efficacy”, suggesting that the vaccines increased the risk of contracting COVID-19 instead of reducing it. The post cited a Japanese study published in 2024 as supporting evidence.

This claim originated from a Substack article by Nicolas Hulscher, who shared inaccurate information on COVID-19 vaccines before. Hulscher is an epidemiologist at the foundation of Peter McCullough, a cardiologist who spread multiple pieces of disinformation about COVID-19 vaccines over the years.

The headline of Hulscher’s article referenced a study by Nakatani et al., which compared the risk of SARS-CoV-2 infection in vaccinated and unvaccinated Japanese workers. Based on the study, he concluded that COVID-19 vaccines acted as “infection promoters”.

This isn’t the first time such allegations have been made. Claims of “negative efficacy” for COVID-19 vaccines have circulated repeatedly during the pandemic. Science Feedback previously debunked similar claims, showing how they misinterpreted scientific data or ignored critical methodological limitations.

Hulscher also argued that the Japanese study confirmed findings from a 2023 Cleveland Clinic study, which reportedly showed an increased risk of SARS-CoV-2 infection among clinic workers who had received more vaccine doses. In a review from 2023, Science Feedback explained that the Cleveland Clinic study didn’t demonstrate negative efficacy. In fact, the study wasn’t designed to investigate vaccine efficacy and its findings were misrepresented.

As with these earlier examples, Hulscher’s claim misinterprets scientific findings by overlooking significant methodological limitations. We reached out to Hulscher regarding these limitations and will update if any relevant information becomes available. Below, we’ll explain in detail why Hulscher’s interpretation is flawed.

What the study by Nakatani et al. did and what it found

The researchers conducted a survey among Japanese workers from small and medium-size companies asking them to self-report their vaccination status and whether they had COVID-19 at any point since the beginning of the pandemic. Researchers also collected information such as age, gender, and hygiene habits, like bathing and mask-wearing.

Nakatani et al. then compared the odds of getting COVID-19 in vaccinated and unvaccinated groups. They also adjusted their calculation to take into account differences between the two groups, such as age, gender, pre-existing chronic health conditions, and hygiene habits. Because these characteristics can influence COVID-19 risk independently of vaccination status (confounding factors), accounting for group differences is important when the goal is to understand the specific effect of vaccination on COVID-19 risk.

They found that the odds ratio of COVID-19 for the vaccinated group, relative to the unvaccinated group, was 1.85, meaning the the probability of getting COVID-19 was higher in those who were vaccinated compared to those who weren’t. The authors also reported that these odds increased when people declared they received more vaccine doses.

The study by Nakatani et al. isn’t designed to determine the efficacy of COVID-19 vaccines

The study’s design includes significant limitations that undermine Hulscher’s interpretation of the results. Notably, the study’s authors acknowledged these limitations in a dedicated section of their paper—details that Hulscher overlooked in his claim.

A key issue in the study is the lack of chronological information with regards to infection. Participants reported whether they were vaccinated and whether they had contracted COVID-19 at any point, but they didn’t specify whether their infection occurred before or after vaccination.

This lack of information creates a critical ambiguity: individuals in the vaccinated category who reported having had COVID-19 could have been infected prior to receiving their vaccine. This issue is especially relevant because COVID-19 vaccines were only made widely available roughly nine months after the onset of the pandemic. Consequently, anyone who contracted COVID-19 during that period and later got vaccinated would still be counted in the vaccinated group—even though their infection occurred when vaccination was not yet possible.

Another significant limitation is the risk of survivorship bias. Since the study was based on a questionnaire conducted in December 2023, only individuals who were alive at that time could participate. Those who died before the survey was conducted were excluded from the analysis.

Furthermore, it’s well-established that COVID-19 vaccines reduce the risk of severe illness and death^[1]. Therefore, another explanation for the findings could be that a higher proportion of unvaccinated individuals already succumbed to the disease before the survey was conducted. Thus COVID-19 cases were undercounted in the unvaccinated group compared to the vaccinated group.

A third limitation lies in the study’s lack of random sampling and matching procedures, which are essential to ensure that the vaccinated and unvaccinated groups are similar in all respects except for the variable being tested, which in this case is vaccination status. This limitation could have been addressed by randomization, a cornerstone of randomized controlled trials that helps minimize bias and confounding factors.

While the statistical analysis in Nakatani et al. adjusted for some confounding factors, such as age, it didn’t account for other critical differences, including occupation, socioeconomic status, and health-seeking behaviors, all of which could influence the likelihood of contracting COVID-19, independent of vaccination status.

Finally, the reliance on self-reported data introduces another layer of potential bias. Self-reporting depends on participants’ ability to accurately recall and report whether and when they contracted COVID-19. If inaccuracies occurred at differing rates between the vaccinated and unvaccinated groups, it would introduce a bias that undermines the reliability of the data.

Because of all these limitations in the study’s design, Hulscher’s claim is misleading, since the study cannot convincingly establish that COVID-19 vaccines have a “negative efficacy”.

By contrast, a large number of studies were conducted over the past few years to evaluate COVID-19 vaccine effectiveness.

Several studies conducted in Japan showed the effectiveness of COVID-19 vaccines. A test-negative case-control study of people in the Hiroshima prefecture reported that two doses of COVID-19 vaccines were roughly 26 and 35% effective against infection by the Omicron variant, while receiving three doses was 26% more effective than two doses^[2]. Test-negative studies are useful because they take into account the differences in health-seeking behavior between vaccinated and unvaccinated people.

Another study showed nearly 90% vaccine effectiveness against symptomatic disease in the case of infection by the Delta variant, and about 21% effectiveness in the case of infection by the Omicron variant. A third dose increased the effectiveness against Omicron symptomatic disease to 71%, directly contradicting the claim of a decrease in vaccine effectiveness with more doses of vaccine^[3].

Another test-negative case-control study in several Japanese hospitals found a vaccine effectiveness against symptomatic infection of about 26% against Omicron, increased to about 58% by a third dose of vaccine^[4].

Yet another test-negative case-control study in several Japanese hospitals reported an effectiveness of 88% against infection by the Delta variant, and 49 to 56% against infection by the Omicron variant. Here again, a third dose of vaccine increased the effectiveness against the Omicron variant to 74%^[5].

Percentages of effectiveness vary greatly depending on the type of COVID-19 vaccine, the SARS-CoV-2 variant at the time of study, the population under study, and the elapsed time between vaccination and infection. However, these studies combined showed that COVID-19 vaccines are effective, as indicated in this list of vaccine effectiveness studies by the U.S. Centers for Disease Control and Prevention (CDC).

Conclusion

The claim that COVID-19 vaccines have “negative efficacy”, based on a Japanese study by Nakatani et al., misinterprets the study’s findings. While the study observed higher odds of infection among vaccinated individuals, its design—such as a lack of information regarding whether infections occurred before or after vaccination, and relying on self-reported data—means the study cannot support conclusions about vaccine efficacy against infection. Additionally, factors like survivorship bias and differences between vaccinated and unvaccinated groups further complicate its interpretation.

In contrast, robust research, including studies conducted in Japan, has consistently shown that COVID-19 vaccines reduce the risk of severe illness, hospitalization, and death. These findings demonstrate the effectiveness of COVID-19 vaccines and contradict the claim of “negative efficacy.”

REFERENCES

• 1 – Wu et al. (2023) Long-term effectiveness of COVID-19 vaccines against infections, hospitalisations, and mortality in adults: findings from a rapid living systematic evidence synthesis and meta-analysis up to December, 2022. The Lancet Respiratory Medicine.
• 2 – Yumiya et al. (2024) Effectiveness of COVID-19 mRNA vaccine in preventing infection against Omicron strain: Findings from the Hiroshima Prefecture COVID-19 version J-SPEED for PCR center. PLoS Global Public Health.
• 3 – Mimura et al. (2022) Effectiveness of messenger RNA vaccines against infection with SARS-CoV-2 during the periods of Delta and Omicron variant predominance in Japan: the Vaccine Effectiveness, Networking, and Universal Safety (VENUS) study. International Journal of Infectious Diseases.
• 4 – Maeda et al. (2024) Effectiveness of primary series, first, and second booster vaccination of monovalent mRNA COVID-19 vaccines against symptomatic SARS-CoV-2 infections and severe diseases during the SARS-CoV-2 omicron BA.5 epidemic in Japan: vaccine effectiveness real-time surveillance for SARS-CoV-2 (VERSUS). Expert Review of Vaccines.
• 5 – Arashiro et al. (2023) Coronavirus Disease 19 (COVID-19) Vaccine Effectiveness Against Symptomatic Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Infection During Delta-Dominant and Omicron-Dominant Periods in Japan: A Multicenter Prospective Case-control Study (Factors Associated with SARS-CoV-2 Infection and the Effectiveness of COVID-19 Vaccines Study). Clinical Infectious Diseases.