Five negative RCTs and one negative pre-print do not a miracle cure make.

This week, I am going against the advice of my loved ones who feel my pain as I read some pretty rough comments online to try and address the hydroxychloroquine issue.

I know, I know.

Whenever I discuss hydroxychloroquine, people come out of the woodwork to tell me what a bad doctor I am.  Even when I made a video simply demonstrating how searching pubmed.gov works, I got some pretty rough replies.

But I guess I’m a glutton for punishment.  So I have set out today to collect, in one place, the randomized trials that have been conducted that form the centerpiece for why doctors like me and Anthony Fauci don’t think hydroxychloroquine works for COVID-19.

Now, before we get started…

I am not paid by any pharmaceutical company, nor do I hold a patent on any drug or device. My grant funding comes from the NIH and the Department of Defense.  My NIH studies have nothing to do with drugs.  My DOD study, in fact, is looking at repurposing an old, cheap, relatively safe drug – valproic acid – for the treatment of a certain type of kidney disease. I literally study whether old cheap drugs can be repurposed for new benefit.

Which is to say that I am really, really interested in cheap effective ways to fight COVID-19. Because, another thing I have to remind people, I’m not just a clinical researcher. I’m a medical doctor who has spent months caring for COVID-19 patients on the wards of Yale New Haven hospital.  I don’t know what else to say – I’m really just trying to synthesize the data we have.

Now, when you look for HCQ studies in COVID-19, you will find a ton- - roughly 900 published now at last count. The vast majority of these are observational studies.

Why do people like me put so much more weight in randomized trials than observational studies? It’s simple.

In an observational study, the observed effect of the exposure of interest (HCQ) on the outcome of interest is due to BOTH it’s true causal effect AND the characteristics of who was selected for treatment.

In a randomized trial, since the selection is random, the observed effect is due SOLELY to the true effect of the treatment.

That’s why we put so much stock in RCTs. Before RCTs are available, observational data is ok – we can use it to generate hypotheses. But observational should be used to design RCTs, and RCTs should be used to guide therapy. Hence my focus today on the extant RCTs.

But to illustrate I’ll give one observational example since I think 100 people have e-mailed it to me in the past week.

This is one of Didier Raoult’s studies, appearing in Travel Medicine and Infectious Disease.

They report on 3,737 patients with COVID-19. These were mostly outpatients, and the study states that, barring contraindications, they were prescribed 200 mg of HCQ three times a day for 10 days with five days of azithromycin. They then compare the 3,119 people who took that regimen for >= 3 days to 618 who didn’t.

In the observational setting like this, the key question is WHY did they not take the medications. Looking at Table 1 in the paper, we can see that, at baseline, these groups are quite different.

Those who took the standard therapy tended to be younger – 53% were under age 44 compared to 36.4% who got the other treatments. They had less cancer, less diabetes, less chronic heart disease, and lower “NEWS” scores which is a measure of disease severity. In other words, this was a group poised to do well. The treatment wasn’t assigned randomly, it was given to the healthiest.  That’s not unethical or anything by the way – it’s totally reasonable to be careful about who you give drugs to. It just makes it harder to interpret the results.

And the results were better in the group who got the HCQ regimen. 0.5% death rate compared to 3.1% death rate.

Now, you can adjust for baseline differences – here they adjusted for that severity score and comorbidity score, not age or anything else and still found significance but let me highlight two issues with adjustment.

First, adjustment isn’t magical – you have to adjust for all the factors that are different at baseline to get an unbiased estimate of treatment effect.

Second, you don’t know all those factors. You can only adjust for what you measure. Unmeasured differences in the groups will always be present with one exception.

You guessed it.  If you randomized, you will balance not only measured differences but even UNMEASURED differences between the groups. 

That’s why clinical epidemiologists like me get so psyched about randomization.

Hopefully, I’ve convinced you that randomization is where it’s at.  And with that, I will present the five peer-reviewed randomized trials that, to date, have been published on HCQ and COVID-19. I’ll also review one pre-print, since it is by the same group that did the dexamethasone study. We’ll do published in chronological order, then the pre-print.

Caveat – I may have missed a trial here or there, and more are coming. We are never DONE learning – we just let the evidence make our conclusions more or less firm.

Study Number 1: A study appearing in the BMJ.

150 hospitalized patients in three provinces in China were randomized to receive either 1200 mg of HCQ for three days followed by 800 mg daily for 2-3 weeks vs usual care. There was no placebo in this study. The groups were well balanced because this was a randomized trial.

The primary outcome was absence of virus PCR on nasal swab at 28 days. 85.4% cleared the virus in the HCQ group compared to 81.3% in the usual care group (not a statistically significant difference at a p-value of 0.34).

The adverse event rate was 30% in the HCQ group and 9% in the usual care group.

Number 2: A study appearing in the NEJM.

821 asymptomatic patients with a high-risk exposure to someone with Covid-19 were randomized to receive HCQ 800 mg once, followed by 600mg 6-8 hours later, then 600 mg for four days versus placebo. The groups were well balanced because this was a randomized trial. The primary outcome was the incidence of Covid-19 at 14 days. 11.8% of those receiving HCQ versus 14.3% of those receiving placebo hit the outcome, a non-significant difference at a p-value of 0.35.

The adverse event rate was 40% in the HCQ group and 17% in the placebo group.

Number 3: A study appearing in the Annals of Internal Medicine.

491 non-hospitalized adults with COVID-19 or high-risk exposure were randomized to the same regimen from that NEJM trial or placebo. The groups were well balanced because this was a randomized trial. The primary outcome was change in symptom severity at 14 days.

The HCQ group improved by 2.6 points and the placebo group by 2.3 points, a non-significant difference at a p-value of 0.12. There was no difference in effect among the 24% of the study population who reported zinc supplementation by the way. The adverse event rate was 43% in the HCQ group and 22% in the placebo group.

Number 4:  A study appearing in the journal Clinical Infectious Diseases.

293 non-hospitalized individuals with early COVID-19 were randomized to 800mg HCQ on day 1 followed by 400 daily for six days versus usual care. There was no placebo in this study. The groups were well balanced because this was a randomized trial. The primary outcome was reduction of viral RNA load in nasal swabs at 3 and 7 days.

The viral load went down by 3.44 logs in the HCQ group and 3.37 logs in the usual care group, a non-significant difference. The adverse event rate was 72% in the intervention arm, 8.7 % in the usual care arm.

Number 5: Another NEJM Study.

667 patients hospitalized with covid-19 and on 4L of oxygen or less were randomized to receive 400 mg HCQ twice daily, 400mg HCQ twice daily + azithromycin, or usual care for seven days. There was no placebo in this study. The groups were well-balanced because this was a randomized trial. The primary outcome was clinical status at 15 days on an ordinal scale.

There was no significant difference in the outcome rates at day 15 for either of the treatment groups compared to usual care. The adverse event rate was 39.3% in the combined therapy group, 33.7% in the HCQ group, and 22.6% in the usual care group.

And finally the pre-print.

Again I am showing this because this study,  appearing (for now) on medrxiv is from the RECOVERY trial group. They are the ones who published the randomized trial showing that dexamethasone (a cheap, widely-available drug) reduced mortality in severe COVID-19. It’s a talented group. They want to find cures. Anyway, they randomized 4,716 hospitalized patients with COVID-19 to 800 mg of HCQ at entry and 6 hours, followed by 400 every 12 hours for the next 9 days or until discharge versus placebo. The groups were well balanced because this was a randomized trial. The primary outcome was all-cause mortality at 28-days.

At 28-days, 26.8% of the patients in the HCQ arm and 25% of those in the usual care arm had died, a non-significant difference at a p-value of 0.18.

Let’s put this together. I am not going to say that HCQ has no effect on COVID-19. We can never be 100% sure of that. But I am sure that if it has an effect, it is quite small.  Think of a world where HCQ was a miracle cure for COVID-19.  Think how different all these randomized trials would look. It would be immediately obvious. 

Straight talk – HCQ is unlikely to kill you.  It will kill someone – rare cases of torsades de pointes occur – but it is unlikely to be you, or your patients. It really is a relatively well-tolerated drug. But there are adverse effects, as all of these trials show and, given that, our ethical obligation to “first, do no harm” is paramount here. There simply is not good evidence that HCQ has a robust effect and there is evidence of at least moderate harm. Ethically, the choice is clear.

A few final caveats. Yes, only one of these trials reported on the use of zinc with HCQ – no effect by the way - but two things on that particular issue. First, we know that many individuals take zinc supplements, so if, as the argument goes, HCQ IS a miracle cure when given with zinc, you’d still see a benefit in an HCQ trial because a subset of people  - maybe 25% - are taking zinc.  

The zinc issue falls into this no-true-scotsman land of HCQ studies. Any negative study can be dismissed “oh you didn’t give it early enough, or late enough, or with zinc, or with azithromycin, or on Sunday or whatever”.  That’s not how science works. I’m not saying any of these studies are perfect. Just that they are the best evidence we have right now. The burden of proof is to show the drug WORKS. Though I’m sure pharma would be stoked to be able to argue that their latest negative trial can be ignored because their billion dollar drug wasn’t given in concert with vitamin C or whatever. 

Yes, I know another Yale professor is saying that HCQ can save lives.

And to those of you who have pointed out that he is a full Professor while I am a mere Associate Professor – you really know how to hurt a guy. I have no idea why he wrote that article and didn’t mention any of the randomized trials. But I embrace the academic freedom he and I both have to present our best interpretation of the data.

Science is ever-learning, ever-developing. More trials will come out and we need to integrate those results into these results to make decisions. We need to let new, high-quality evidence change our beliefs.  I am willing to do that. I hope you are too. But I want randomized trials. Because, if I didn’t say it enough.  Randomization ensures the treatment and control groups are well-balanced and that makes all the difference.

This commentary first appeared on medscape.com.