he COVID-19 pandemic has posed enormous challenges to the healthcare systems and economies around the world. As a result of tremendous public-private partnerships, several vaccines are being developed at an unprecedented speed. Because of the complexity of study designs and interim analysis strategies, it is important that communication of clinical trial results (by, e.g., press release) be as clear as possible so there is a good public understanding of the efficacy and safety of these vaccines.

For example, two mRNA COVID vaccines, one developed by Moderna and the other co-developed by BioNTech and Pfizer, received emergency use authorization (EUA) by the US Food and Drug Administration (FDA) in December 2020 based on interim results from their phase 3 trials. The vaccine codeveloped by AstraZeneca and Oxford University was approved by the UK for emergency use based on interim results of two phase 2/3 trials conducted in the UK and Brazil. Clear and accurate communication of trial results like these is instrumental in building public confidence in new vaccines and supporting the rollout of mass vaccination programs after approval.

Types of COVID-19 Vaccines and the Regulatory Pathway

There are four main types of vaccines currently in development for the prevention of COVID-19. All four aim to produce immunity to the SARS-Cov-2 virus that causes the disease:

1. The mRNA vaccines, the newest approach employed by Moderna and Pfizer/BioNTech, use genetic material called messenger RNA (mRNA) to instruct human cells to make the spike protein found on the surface of the coronavirus to stimulate the body’s immune response.
2. The vector vaccines developed by AstraZeneca, J&J, and Sputnik use a harmless, inactivated adenovirus (a common cold virus) to carry the genetic code of the spike protein into a human cell to make the protein needed to trigger the immune response.
3. The inactivated whole-cell vaccine developed by Sinovac uses the killed version of the entire coronavirus to elicit an immune response.
4. The protein subunit vaccines developed by Novavax and Sanofi use just little pieces of the coronavirus spike protein, rather than the entire inactivated virus, to elicit an immune response.

In the US, the FDA issued guidelines in October 2020 for EUA for vaccines to prevent COVID-19. One of the critical criteria for EUA requires a vaccine to demonstrate at least 50% efficacy from a placebo-controlled clinical trial, with the confidence interval’s lower bound around the efficacy estimate >30%. Another critical criterion requires safety data from a median follow-up of at least two months after receiving the complete regimen of the vaccine. It is important to highlight that EUA is not the approval, which will require additional data on safety and efficacy. Vaccine efficacy is generally measured by the relative reduction of the incidence of the target disease in individuals receiving the vaccine compared with those receiving the placebo vaccination.

The large-scale, pivotal vaccine efficacy trials initiated by Pfizer/BioNTech, Moderna, AstraZeneca, J&J, and Novavax all aim to enroll tens of thousands of individuals, including high-risk individuals such as older adults or individuals with certain chronic medical conditions. Because of the uncertainty of COVID-19 infection rates, these trials utilize a fixed endpoint design that targets a fixed number of COVID-19 cases to ensure sufficient study power to demonstrate vaccine efficacy. For example, in the efficacy trial conducted by Moderna, more than 30,000 individuals were randomized to receive either mRNA vaccine or placebo in a 1:1 ratio. The study targets to accumulate a total of 151 COVID-19 cases, providing 90% statistical power to demonstrate vaccine efficacy using the EUA criteria if the true vaccine efficacy is 60%. The study also plans for two interim analyses, at 30% and 70% of the total cases, and the interim analyses are reviewed by an external data monitoring board (DMB).

Communication of Trial Results: Two Examples

Example 1: On November 16, 2020, Moderna issued a press release about the results of the first interim analysis from the mRNA vaccine efficacy trial (n=30,000). Because of the increased incidence rate of COVID-19 observed in the trial, 95 COVID-19 cases were reported in the first interim analysis with an estimated vaccine efficacy of 94.5% (5 cases in vaccine recipients versus 90 cases in placebo recipients). Moderna’s press release also included detailed information about the study design and interim results, including the demographic characteristics of the study participants, statistical significance of the interim results, efficacy results for the older age group and for prevention of severe COVID-19 disease, as well as the safety profile from the trial participants.

Overall, this press release provides clear communication about the vaccine’s efficacy and safety profile. However, it would have been beneficial to provide a precision estimate, such as a confidence interval, around the efficacy estimate to account for the uncertainty associated with interim analyses.

Example 2: In a November 23, 2020 press release, AstraZeneca reported interim results from two studies of their COVID vaccine conducted in the UK and Brazil (n=11,636). A total of 131 COVID-19 cases were included in the interim analysis, and the estimated vaccine efficacy ranged from 62% when given two full doses, to 90% when given as a half-dose followed by a full dose at least one month later. The company also reported an average efficacy of 70% based on the aggregate analysis of the two vaccine regimens.

The information provided in the press release lacks clarity for the following reasons:

• The half-dose regimen was a result of error of potency calculation and was not administered by design.
• No numbers of COVID-19 cases were reported for the separate regimens and so it is not clear how precise or reliable these efficacy estimates are, especially for the one-dose regimen with a small number of participants (n=2,741).
• There is no mention about the age of participants in these trials so it is unclear whether the vaccine efficacy will hold up in older adults.

Because of the lack of clarity, the above press release has caused some confusion about the overall vaccine effectiveness. We expect that the ongoing pivotal trial of the AstraZeneca vaccine (n>30,000) will provide more definitive answers.

Recommendations

Although press releases regarding interim analyses usually contain limited information while the trial is ongoing, it is important to provide more detailed information during a pandemic when public concern is high. To help the public understand key results and their potential implications, we recommend that any communication or press release of study results provide clear information regarding the salient features of the study, such as:

• characteristics of study population,
• criteria for evaluating efficacy and safety as appropriate,
• precision of the efficacy estimates,
• efficacy and safety results for key subgroups as appropriate, and
• some overall interpretation of the study results in the context of the overall vaccine development.

Details of the guidance are provided in Table 1. Clear communication of these critical pieces of information will instill greater confidence of the public in new vaccines and ultimately support the rollout of mass vaccination programs in the fight against the pandemic.

Acknowledgements

We thank the DIA Statistics and Data Science Community and DIA Bayesian Working Group (COVID-19 Working Group) for their insights and discussions.