What the COVID-19 Pandemic Can Teach Us About Clinical Trials

he search for COVID-19 treatments has been filled with controversy and confusion. It is clear that the randomized clinical trial is more valuable than ever but is not being utilized as effectively as possible. In this commentary I give examples of these problems and discuss some potential improvements.

For example, publication of a small non-randomized French study, claiming effectiveness for hydroxychloroquine as treatment for COVID-19 led to a world-wide frenzy for the drug. The study incorporated data for 20 treated patients of whom six discontinued treatment early. Sixteen patients, who did not receive the drug, were used as a control group. Excluding results for the six patients who discontinued treatment, the investigators found that 70 percent of the hydroxychloroquine group had negative SARS-CoV-2 nasopharyngeal swabs at day 6 compared to 12.5 percent in the control group. The resulting rush on chloroquine led to a shortage of the drug worldwide for patients taking regular chloroquine or hydroxychloroquine for lupus or other systemic diseases.

Under section 564 of the Federal Food, Drug, and Cosmetic Act, the US Food and Drug Administration (FDA) Commissioner may allow unapproved medical products to be used in an emergency to “diagnose, treat, or prevent serious or life-threatening diseases or conditions when there are no adequate, approved, and available alternatives.” Medical products that may be considered for an emergency use authorization (EUA) are those that “may be effective,” i.e., if “the known and potential benefits of the product, when used to diagnose, prevent, or treat the identified disease or condition, outweigh the known and potential risks of the product.” It could be argued that this standard for EUAs could open the door to concerns about non-objective criteria being used in the FDA decision process. Also, approving a treatment for EUA may make it impossible to conduct needed randomized clinical trials.

Chloroquine and hydroxychloroquine have been used for a very long time to treat malaria and autoimmune diseases, such as lupus or rheumatoid arthritis. With the assumption that the known and potential benefits of the unapproved treatment would outweigh its known and potential risks, the FDA granted an EUA for hydroxychloroquine given alone or in combination with azithromycin for the treatment of COVID-19 to slow the progression of the disease in seriously ill patients. This allowed for the distribution of anti-malarial drugs to hospitals across the country

The publication of the original study was followed by the publication of other non-randomized studies of various sizes, some of which supported the use of hydroxychloroquine and some of which did not. An observational study, published in the Lancet, later withdrawn, reported that patients with COVID-19, who were treated with chloroquine or hydroxychloroquine (with or without a macrolide), showed an increased risk of dying in hospital and of de novo ventricular arrhythmia compared to patients who did not receive the drugs. This led to the US Centers for Disease Control and Prevention (CDC) issuing a health advisory on March 28, 2020, warning against the use of non-pharmaceutical chloroquine phosphate and hydroxychloroquine, which in turn caused the FDA to withdraw its EUA for the drug.

A second example of the current confusion is the use of convalescent plasma transfusion for SARS-Cov-2 infected patients. Convalescent plasma is a century-old passive antibody therapy that has been used to treat outbreaks of novel infectious diseases. Given the health emergency, the US FDA established an expanded access program for convalescent plasma transfusions for COVID-19 patients. Between April and June 2020, this program supported the transfusion of a convenience sample (i.e., a sample of participants that was easy to obtain) of 20,000 hospitalized patients with COVID-19 convalescent plasma. The frequency of serious adverse events (SAEs) was low and considered unrelated to the actual plasma transfusion, including for the majority of the thromboembolic or thrombotic events (n=75) and cardiac events (n=597). The seven-day mortality rate was 13.0 percent (confidence interval: 12.5–13.4 percent) and overall higher among critically ill patients. The FDA gave an EUA for the transfusions, but a committee convened by NIH expressed concern that the evidence for benefit was too limited for such authorization. Human convalescent plasma is currently being used world-wide to treat COVID-19. However, there remains a lack of consensus on the efficacy of convalescent plasma as a therapeutic for hospitalized patients with COVID-19.

The development of remdesivir for the treatment of patients with COVID-19 was methodologically better, perhaps because it was sponsored by industry for the purpose of obtaining evidence of effectiveness that would support traditional full regulatory approval. Prior to the release of the trial results, some confusion was caused by the release of results at some medical centers and by the publication of a small randomized clinical trial (RCT) terminated for lack of accrual in which no indication of treatment benefit was evident. Finally, an RCT positive for a time-to-improvement intermediate endpoint led to FDA approval of the drug for hospitalized patients. The study of 1,063 patients included 538 who received remdesivir and 521 who were given a placebo. Mortality in the remdesivir group was 7.1 percent compared to 11.9 percent for the placebo group, but this difference was not statistically significant. There was no evidence of benefit for patients who were intubated at entry into the study. Some controversy about the result was due to the fact that the sponsors changed the primary endpoint shortly before the final analysis of the data. The study was stopped prematurely based on the recommendation of the Data Safety and Monitoring Board (DSMB) using the new endpoint. This change did not allow the evaluation of the effect of drug on survival. There was no second RCT with a standard-of-care control arm.

Conclusions

Despite the concerns and emergencies resulting from the pandemic, it seems evident clinical trials are not used as effectively as possible. Large numbers of patients are being treated with agents that have not been shown to be effective and the resulting data are not being properly analyzed using well-accepted, common methods of analysis.

The single-arm trial has been useful in oncology for screening new single agents when follow-up RCTs are assured. In a pandemic, however, a follow-up RCT cannot be assured in the timeframe needed, and so single-arm trials should be eliminated in favor of a phase II or phase II/III randomized trial design incorporating a standard-of-care control arm. This trial can be conducted in two stages: a 2:1 randomization used for the phase II stage (treatment vs. control) and a 1:1 randomization in the phase III stage, if one is warranted.

It is not correct to view the RCT study of a new drug as more experimental and requiring greater oversight than the single-arm design. The administrative barriers to employing randomization from the start should be reduced; if a randomized trial involving a treatment of unknown benefit requires informed consent or external monitoring, then so should a single arm trial involving the same drug

It is often said that we need more data. Generally, however, we don’t need more data; we need more interpretable data. And that can only be obtained from a properly controlled clinical trial. I have heard statisticians say that they do not want to use randomization in phase II trials, because physicians will accept the results and make it more difficult to conduct phase III trials. However, this is also true of single-arm trials, as the COVID-19 examples illustrate.

RCTs are not being started early enough, and, consequently, by the time large multi-center RCTs are established, a lot of misinformation may have already been generated. An RCT does not have to be large or multi-center to be useful. And delaying the start of a trial until centers from multiple continents are involved can be counterproductive.

Suppose one is interested primarily in finding a treatment with a large treatment effect. If one conducts a single-arm trial and gets a large outcome, one doesn’t know whether the treatment was effective or whether the patients had a better prognosis than expected. On the other hand, if you conduct a small RCT you may very well have adequate power for detecting a large treatment effect. Or the trial can serve to demonstrate that such a large treatment effect is unlikely to exist for that drug.

Instead of conducting a single-arm trial of n patients, one can use a weighted 2:1 randomization in which most of the patients receive the test treatment; instead of a single arm trial of 30 patients one can have a randomized controlled trial of 45 patients.

If the prior probability of the test treatment being effective is small, it is obvious that the trial is not conducted entirely with therapeutic intent. Ethically, therefore, it is important that the trial give interpretable results. It is also more important to avoid giving all patients a regimen that is not proven to be efficacious and is possibly harmful, with no research benefit for future patients for lack of a control group.