Analysis of Dutch cities’ mortality doesn’t show that COVID-19 vaccines increase the risk of death, contrary to Alex Berenson’s interpretation

Key takeaway

COVID-19 vaccines have been tested in several large clinical trials. Their effectiveness largely outweigh the risk of rare side effects. A study comparing vaccination coverage and all-cause mortality in Dutch cities doesn’t show that COVID-19 vaccines increase the risk of dying. The study’s design is inadequate for drawing such a conclusion.

Reviewed content

Verdict:

Unsupported

Claim:

“New paper suggests Covid mRNA vaccination rates are linked to increases in overall deaths”

Verdict detail

Misrepresents source: The preprint used to support the claim investigated the correlation between the distribution of mortality across Dutch cities and COVID-19 vaccination coverage. However, Berenson presented some numerical results as if it were an increase in overall mortality. Furthermore, this numerical result wasn’t adjusted for possible bias.
Inadequate support: The preprint used to support the claim used city-level data and not individual-level data which precludes drawing conclusions on the impact of vaccination on the risk of dying. Furthermore, all possible confounding factors aren’t accounted for, which may bias the final results.

See our method of evaluation

Full Claim

“New paper suggests Covid mRNA vaccination rates are linked to increases in overall deaths”; the paper suggests a 5% increase in mortality due to vaccination.

More than 80% of the Dutch population over 12 has been vaccinated against COVID-19, as of August 2022. Large clinical trials and ongoing monitoring of the millions vaccinated worldwide show that COVID-19 vaccines effectiveness largely outweigh the risks of rare side effects and effectively protect against severe forms of the disease.

However, inaccurate claims that these vaccines are dangerous, increase the risk of death, or weaken the immune system frequently circulated on social media, and Health Feedback already debunked several of them.

Journalist Alex Berenson made such a claim in July 2022 when he stated on Substack that new results “suggested Covid mRNA vaccination rates are linked to increases in overall deaths”. Berenson had made multiple inaccurate and misleading claims about COVID-19 and vaccines that Health Feedback previously debunked.

In his article from 28 July 2022, Berenson reported on a preprint (a study not yet peer-reviewed or accepted by a journal) published by André Redert, who previously worked as a researcher in computing sciences but is not trained in virology, immunology, or epidemiology. In that preprint, Redert compared the all-cause mortality and vaccination coverage in Dutch cities.

Although Berenson acknowledged that the preprint doesn’t demonstrate that vaccination caused an increase in all-cause mortality, this is implied by the Substack article’s headline. Berenson also combined his take on Redert’s preprint with observations of increased all-cause mortality in “highly vaccinated countries”. Regardless of whether this last statement is accurate, it is clear that Berenson’s subtext is that mRNA vaccination causes an increase in mortality. The readers’ comments section at the bottom of the article clearly shows that this is how his article is interpreted. For example, one can read “Most of my vaxxed family and friends stopped after the initial series, so are they in the clear, or still ticking time bombs?” or “In my country, Portugal, we are witnessing a silent democide” or “our CDC is flat out covering up the excess deaths because they know there is a problem.”

As we explain below, however, Berenson’s interpretation is incorrect and based on a flawed interpretation of the preprint’s results. Redert’s preprint itself suffers from limitations that weaken the conclusions that can be made about vaccine safety.

There is no established causal relationship between all-cause mortality and vaccination coverage

In his paper, Redert looked at the distribution of mortality and vaccination coverage between Dutch cities. Some cities presented an all-cause mortality above average, and others, below. Similarly, some cities had vaccination coverage above the national average and other cities below. Redert then tried to determine if there was a correlation between the two and, if so, what proportion of the mortality was distributed across cities in the same way as vaccination coverage. In other words, Redert asked to what extent a city with an above average vaccination coverage would see its all-cause mortality also rise above average or, conversely, drop below average.

Redert observed that the weekly mortality partially correlated with vaccine coverage. In other words, cities with an all-cause mortality above average tended to also exhibit vaccination coverage above average.

However, it is important to stress that Redert’s preprint only established correlations between vaccination coverage and mortality, and didn’t demonstrate any causal association. Correlation by itself doesn’t prove that one event is the cause of another. Redert’s preprint also doesn’t provide any biological explanation or medical evidence showing that vaccination increases the likelihood of dying.

Furthermore, Redert used city-level data: all-cause mortality and vaccination coverage in that same city. Drawing conclusions on the risk of individuals dying from average data obtained at the city level is known as an ecological fallacy. In our case, city-level data tell us nothing as to whether vaccinated people were overrepresented among those who died, for instance.

Therefore, data from Redert’s preprint aren’t suitable to support Berenson’s suggestion that vaccination is linked to an increase in all-cause mortality.

Berenson’s interpretation of data is flawed

In his account of Redert’s preprint, Berenson misinterpreted some of the results and used them to draw ill-founded conclusions.

Redert provided the example of a given week where 5% of the total 4,000 deaths from all causes of that week—that is, 200 deaths—were distributed between cities in the same way the vaccination coverage was. The remaining 3,800 deaths were distributed across cities in a manner uncorrelated with vaccination coverage.

Berenson apparently interpreted this 5% of “vaccination-correlated mortality” as an increase in the total number of deaths, because he also claimed that his 5% figure was consistent with other data reporting a 5% excess mortality in several countries. He then went on to comment that a 5% increase would actually represent many more dead people in the long run.

However, Berenson’s interpretation of this piece of data is erroneous. This 5% figure from Redert’s preprint is not related to the number of all-cause deaths that Berenson cited. Redert’s analysis focused on the distribution of deaths across all cities, not on the total number of deaths. In fact, Redert warned: “These [5% deaths] are not 200 additional deaths related to vaccination, it means that 200 out of 4k deaths were distributed over municipalities in the same pattern as vaccination coverage”.

Also, this 5% figure is the result of Redert’s analysis for one week—week 50 of 2021—used as an example and isn’t necessarily representative of the entire pandemic, contrary to what Berenson suggested.

Third, many confounding factors bias the direct comparison of all-cause mortality and vaccination coverage. Confounding factors are variables that affect the outcome of an experiment, but aren’t the variables being studied. If scientists don’t factor in the influence of confounding factors in their study, they may draw erroneous conclusions about causality. For example, older people are more likely to get vaccinated against COVID-19 and to die from any cause. Therefore, one would expect that cities with an older population will exhibit a higher than average vaccination coverage and a higher all-cause mortality without any causal association between the two.

Redert was well aware of these confounding factors and proposed a way to take them into account by normalizing his results to pre-pandemic mortality data. However, this 5% figure Berenson extracted from the preprint comes from the raw data and not from the normalized data adjusted for confounding factors. Considering all the above, it is clear that Berenson’s take on Redert’s paper is thus flawed and his conclusions are inaccurate.

Additional limitations of the Dutch cities study preclude drawing conclusions

Even when considering the adjustments for confounding factors proposed by Redert, the study’s design limits what can be concluded from it.

First, Redert used weekly mortality data, but these data were compared with the final vaccination coverage of November 2021. In order to really compare if the mortality correlated with the vaccination coverage, at least partially, it would be better to use mortality and vaccination coverage data from the same week. Given that the study aimed to investigate whether vaccination impacted all-cause mortality, one could even imagine comparing the mortality of a given week with the vaccination coverage of an earlier time period, but not with the final vaccine coverage.

Second, Redert took into account some important confounding factors by normalizing his results by the mortality in 2019, before the COVID-19 pandemic. While this can correct confounding factors due to demographic, such as age or wealth, it would not correct for any other possible confounding factors that would affect both all-cause mortality and vaccine coverage during the pandemic. For instance, we can hypothesize that cities heavily impacted by COVID-19 will tend to vaccinate more, resulting in higher vaccination coverage, but at the same time would suffer from greater disruption to vital elements such as transport and healthcare that could worsen all-cause mortality.

Conclusion

Berenson incorrectly suggests that COVID-19 vaccination can increase all-cause mortality based on the results from a non-peer-reviewed publication. However, Berenson’s interpretation of the publication is flawed. Furthermore, several limitations in the design of the study itself make it impossible to draw conclusions from the results. By contrast, large clinical trials, as well as the millions vaccinated against COVID-19 in real-world conditions show that COVID-19 vaccines’ benefits largely outweigh their risks.