Current data from clinical trials offer no reliable evidence that ivermectin is effective against COVID-19; better-quality clinical trials are needed to resolve this question

A Sky News segment hosted by Alan Jones and featuring Craig Kelly, a Member of the Australian Parliament, claimed that scientific studies supported the use of ivermectin for treating COVID-19. Published on 15 April 2021, the segment was shared on Kelly’s Facebook page, receiving more than 150,000 views and more than 6,500 interactions, including likes and comments.

This isn’t the first time that Kelly, who has no scientific training, made unsubstantiated claims about the efficacy of ivermectin for treating COVID-19. During the interview, Kelly also cited the findings of physician Tess Lawrie, alleging that 50 out of 51 studies show “ivermectin is not just effective, but highly effective”.

Ivermectin is an antiparasitic drug. Health Feedback addressed the unsupported claim that ivermectin is effective against COVID-19 in several reviews (see here and here). Claims about ivermectin’s efficacy began with a study that reported that ivermectin reduced viral replication of SARS-CoV-2 in cell cultures^[1].

However, other scientists pointed out that this only occurred at “physiologically unattainable concentrations”, as seen in this June 2020 editorial published in the American Journal of Tropical Medicine and Hygiene^[2]. The same editorial acknowledged that this alone wouldn’t rule out the possibility that ivermectin had some beneficial effect in people, hence clinical trials would still be of interest.

Since that editorial’s publication, numerous clinical trials studying the efficacy of ivermectin against COVID-19 have been conducted. Public health authorities have attempted to use currently available data to provide the best possible guidance, at this time, on the use of ivermectin.

For example, the World Health Organization (WHO) evaluated data from 16 randomized controlled trials, which included 2,407 participants. Its living guideline on proposed therapeutics for COVID-19, most recently updated on 31 March 2021, recommended “not to use ivermectin in patients with COVID-19 except in the context of a clinical trial”.

The guideline also explained the decision not to recommend ivermectin:

“The effects of ivermectin on mortality, need for invasive mechanical ventilation, hospital admission, duration of hospitalization and time to viral clearance all remain very uncertain (all very low certainty evidence). The uncertainty results from important concerns related to risk of bias in the included studies, and imprecision from a very low number of events and, in some cases, wide confidence intervals (CIs) in pooled estimates.”

The guideline also stated that “further high-quality clinical trials examining this drug would be essential before any recommendation for use as part of clinical care”.

Likewise, the U.S. National Institutes of Health evaluated multiple clinical trials, some of which are listed here, and concluded:

“There are insufficient data for the Panel to recommend either for or against the use of ivermectin for the treatment of COVID-19. Results from adequately powered, well-designed, and well-conducted clinical trials are needed to provide more specific, evidence-based guidance on the role of ivermectin in the treatment of COVID-19.”

Overall, clinical trials have thus far failed to provide reliable evidence supporting the use of ivermectin against COVID-19. This is because the trials varied in terms of quality and many were at a high risk of bias. These limitations are the result of several factors, such as the trial design and the study population. And as scientists pointed out in this Health Feedback review, more rigorously designed studies found that ivermectin provided no benefit to COVID-19 patients.

Kelly cited physician Tess Lawrie when claiming that 51 studies showed ivermectin worked. While Lawrie and her colleagues published a meta-analysis of ivermectin studies, which itself was rife with problems as detailed in this Health Feedback review, that meta-analysis only examined fewer than 30 studies. Kelly may have instead been referencing the website run by a group named CovidAnalysis, which claimed that there have been 52 trials for ivermectin to date, a figure much more similar to Kelly’s.

CovidAnalysis conducted a meta-analysis of ivermectin studies and claimed that ivermectin works to treat and prevent (prophylaxis) for COVID-19. However, its own analysis indicates that its conclusions aren’t sound.

For example, one important measure to note in a meta-analysis is heterogeneity. This measure tells us whether the results of the studies in a meta-analysis are consistent with each other. Variability in results can arise from different factors, such as the study population and the way outcomes in the study were measured. A common way to measure heterogeneity is I².

The Cochrane Network, a not-for-profit organization which conducts “systematic reviews and other synthesized research evidence”, stated that “Meta-analysis should only be considered when a group of studies is sufficiently homogeneous in terms of participants, interventions and outcomes to provide a meaningful summary”. It also provided a “rough guide to interpretation [of I²] in the context of meta-analyses of randomized trials”:

“0% to 40%: might not be important;
30% to 60%: may represent moderate heterogeneity*;
50% to 90%: may represent substantial heterogeneity*;
75% to 100%: considerable heterogeneity*.

*The importance of the observed value of I² depends on (1) magnitude and direction of effects, and (2) strength of evidence for heterogeneity (e.g. P value from the Chi² test, or a confidence interval for I²: uncertainty in the value of I² is substantial when the number of studies is small).”

In an article published by BMJ^[3], Higgins et al. explained:

“Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis.”

Overall, the lower I² is, the smaller the heterogeneity is, and the greater the confidence researchers can have in the reliability of the results.

Looking at CovidAnalysis’ ivermectin analysis (see figure below), we can see that the I² values reported for “early treatment”, “late treatment”, and “prophylaxis” are 85%, 65.6%, and 83.8%, respectively. These values indicate that the results of the studies in the meta-analysis are very different from one another, or very heterogenous. As explained above, this reduces the reliability of the meta-analysis’ conclusions.

Figure 1. A forest plot showing the meta-analysis of ivermectin’s effect on different outcomes, conducted by CovidAnalysis.

Yet at no point did CovidAnalysis acknowledge the limitations in its analysis or express any uncertainty about its conclusions, misleading readers into believing that there is already reliable evidence for ivermectin’s efficacy. This behavior is consistent with its earlier, misleading “analysis” purporting to show that hydroxychloroquine effectively treats COVID-19.

Indeed, by examining the details of each study provided in Figure 6, we can see details of how the studies differed from each other, which can suggest how the heterogeneity came about.

For example, 12 of the 52 ivermectin studies involved combined treatment of ivermectin with other drugs. This means that the effects observed in these studies cannot be attributed to ivermectin alone.

Another source of variability is that the studies measured different outcomes, ranging from mortality, length of hospital stay, changes in viral load, and recovery time, which were all combined into a single analysis.

In other words, far from providing reliable evidence that “ivermectin works”, as the website claimed, the findings of CovidAnalysis only reinforce the fact that clinical trials so far provide a low level of certainty—as highlighted in the WHO guideline.

Notably, a preprint study by Elgazzar et al., used in CovidAnalysis’ meta-analysis, was withdrawn in July 2021 after numerous issues with the study were detected. Among these issues were plagiarism, potential data fabrication, and data that was inconsistent with the study’s protocol. These issues were uncovered by student Jack Lawrence, who analyzed the preprint for a class assignment.

It is useful to note that among the voices discouraging the use of ivermectin for COVID-19 treatment is Merck, the pharmaceutical company that produces ivermectin. Given that it would stand to profit if ivermectin were used for COVID-19 treatment, it is significant that the company stated that it doesn’t recommend ivermectin for COVID-19 because there is no meaningful clinical evidence supporting it.

Finally, Kelly accused “the people that are pushing the vaccines” of suppressing the use of ivermectin for treating COVID-19. He provided no evidence for this claim nor did he provide any examples of who these people were. The claim is also flawed in reasoning. Even if ivermectin were eventually proven to be an effective drug for COVID-19, it wouldn’t make vaccines useless (or vice versa), just as seat belts don’t render brakes or airbags in a car redundant.

Overall, the claim that ivermectin can effectively treat and prevent COVID-19 remains unsubstantiated by reliable evidence. Researchers have called for high-quality clinical trials to resolve this question. While recommendations may change in the future, should reliable evidence emerge, many health authorities, including the WHO, don’t currently recommend ivermectin as a COVID-19 treatment or preventative due to the lack of evidence supporting it, except as part of clinical trials evaluating its efficacy.

READ MORE

Scientist Hilda Bastian published several blog posts explaining how to interpret a meta-analysis here, here, and here.

UPDATE (16 Jul. 2021):

This review was updated to indicate that a preprint study by Elgazzar et al., which was used in the meta-analysis by CovidAnalysis, has since been withdrawn due to ethical concerns.

REFERENCES

• 1 – Caly et al. (2020) The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro. Antiviral Research.
• 2 – Chaccour et al. (2020) Ivermectin and COVID-19: Keeping Rigor in Times of Urgency. American Journal of Tropical Medicine and Hygiene.
• 3 – Higgins et al. (2003) Measuring inconsistency in meta-analyses. BMJ.