lear, effective communication and close collaborations between statisticians and other functions at all stages of the clinical trial pipeline are key to ensuring that clinical trials are informative (regardless of the outcome). Unfortunately, clinical and statistical colleagues within the industry sometimes find this challenging. Statistical jargon often results in inefficient, confusing discussions between statisticians and the team at large, and study teams, particularly those from smaller organizations, often seek statistical support too late in the process. Both can compromise the study design and the data and their analysis. These challenges can lead to inefficiency and inconsistency in messaging to senior leaders who are keen to see how assets are progressing so that they can determine the next steps in terms of resourcing, prioritization, and regulatory discussions.

FDA regulations on drugs for human use note that (a) the purpose of conducting clinical investigations of a drug is to distinguish the treatment effect from other influences (e.g., the placebo effect or spontaneous change in disease), and (b) reports of adequate and well-controlled investigations provide the primary basis for determining whether there is substantial evidence to support the claims of effectiveness for new drugs.

Formally speaking, statistics is the term used to refer to the collection, analysis, interpretation, and presentation of data. It provides us with the tools needed to understand data in a quick and efficient way while also allowing us to make conceptual sense of the data so that we can communicate accurate information to others. As a result, statisticians have an important role to play when it comes to ensuring that clinical trials use (a) designs and practices that provide data that are robust and aligned with the scientific questions, and (b) the analysis methodologies used are appropriate and the interpretation of any results are clear.

There are two areas of the clinical trial pipeline where statistical input is important and improvements in communication and knowledge sharing may lead to study benefits.

1. Study design phase: Valuable insights require the right data and good quality data

The adequate, well-controlled investigations referred to in CFR – Code of Federal Regulations Title 21 have certain characteristics, all of which will benefit from input from a statistician. One of the key criteria is a “clear statement of the objectives and a summary of the proposed or actual methods of analysis.” Having a clear statement of the study objectives will allow the team to decide on the most appropriate study design, data collection, and analysis methods. The estimand framework can help align the clinical trial objective with the study design, endpoint, and analysis. Published in 2020, the ICH E9 R1 addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials discussed the statistical methods used in clinical trials to improve study planning and the interpretation of analyses. Estimands are essentially a precise description of the treatment effect using five components, each of which will require cross-functional expertise and regulatory input. The multidisciplinary approach needed to implement this fundamental framework highlights the real need to ensure that statisticians and other functions are speaking the same language.

When we proceed to the data analysis phase of the pipeline, we need to ensure that the selected methodologies are appropriate for both the clinical question and the collected data. Statistics can be a powerful tool when analyzing data, but we need to ensure we have a good understanding of the assumptions and conditions of the methods as well as enough data to satisfy those requirements. All too often, stakeholders and/or senior leaders are excited to see answers, and it is statisticians who are expected to deliver. Where this thought process breaks down is when the data density for a particular data type is poor. For example, in the biomarker space, samples that are not handled correctly or not collected at the correct volume can lead to a large proportion of the samples being nonviable when it comes to processing on a specific assay. This can lead to a sparse data set for statistical analyses, which severely limits what can be done with those data.

With strong interest from leadership in study results, particularly for promising assets, the desire to gain maximum insight from all data is only natural. When planned analyses have to be scaled down, or even stopped, due to data density issues, we enter a difficult phase of expectation management. This requires clear communication of the challenges faced by the statistical team, the impact on analyses and insights, and the plan going forward. This messaging needs to be understood by key study personnel so that consistent messaging can be brought to all stakeholder groups. Unfortunately, the challenges faced by the team are not always communicated in a way that facilitates this, mainly because statisticians often use technical jargon when discussing the methodologies they can and cannot use.

Stakeholder management is important. Statisticians need to take a step away from statistical jargon and articulate the challenges in a language that their colleagues can understand and then use it to convey which insights an investigation can contribute to.

2. Results interpretation phase: P-values are not the whole story

P-values are perhaps the most universally recognized statistic. In clinical trials, the assessment of effectiveness of a treatment depends on p-values, and most people are aware that a p-value less than 0.05 suggests a statistically significant difference between the groups being compared. Despite this widespread understanding, there is often a lack of clarity on what p-values really are and what they can tell us.

A p-value is a probability that, by definition, is a number that takes on values between 0 and 1 inclusive. In terms of statistical hypothesis testing, p-values represent the probability of observing the predicted outcome; or in more technical terms, if the null hypothesis of no observed difference between treatments is true. Put another way, suppose the data you have collected shows a difference of 10 units between the two treatment arms. The p-value here would tell us the probability of observing a difference of 10 units or more if there is truly no difference between the arms. The smaller the p-value, the less likely we are to observe our results and thus the stronger the evidence that there is a difference between the arms.

The issue with p-values is that they only tell us that the data are statistically different. They cannot tell us the clinical relevance of the observed difference. It is this important point that is often overlooked when interpreting p-values. It is commonplace to see researchers use p-values to describe the strength of evidence available, with some publications using phrases like “study met its primary endpoint” with a p-value of 0.049.

It is possible to obtain a small or statistically significant p-value for an extremely small difference between treatment arms provided you have a large enough sample size. While the result would be statistically significant, the size of the difference between the arms would probably not be large enough to make a difference to patients. Similarly, observing larger, nonsignificant p-values does not mean the study has found nothing of clinical importance. It may be that the loss to follow-up rate was higher than expected, leading to a small sample size and an underpowered study.

It is therefore important that statistical teams who are responsible for the interpretation and dissemination of data remember that the p-value is, as noted by the American Statistical Association, a “useful statistical measure” and that they ensure that clinical colleagues and/or senior stakeholders are aware of the need to interpret them alongside both the effect size and associated confidence intervals. Only by looking at all these measurements will researchers be able to understand what the data is telling us, and thus be able to provide fully informed guidance on what is best for patients.

Conclusion

Statistics and the interpretation of statistical results are an important part of what researchers do and how healthcare teams make decisions on how best to help their patients. To do so effectively, it is imperative that we ensure we all have clear lines of communication and effective collaboration. Statisticians need to get better at providing unambiguous and nontechnical interpretations of their analyses and ensure that others who are handling results are aware of the caveats around them. At the same time, we need to ensure that our nonstatistical colleagues engage with statisticians at the earliest possible stage and have familiarity with the lexicon used so that they can “translate” the results to senior leaders while maintaining the messaging about the caveats surrounding any results. This is not about training everyone to be a statistician, it is about providing a platform to help bridge the gap in knowledge so that conversations can be more engaging and effective.

References available upon request.