rtificial intelligence/machine learning (AI/ML) has been leveraged in various industries for decades, but generative AI (GenAI) has recently emerged to offer new use cases in life sciences. Ultimately, GenAI—or any other form of AI—is not going to meet today’s lofty expectations if it does not provide a tangible return on investment (ROI).

We are nearing the peak of the AI hype cycle, and skepticism is starting to grow. For example, recent surveys indicate that AI has not won public trust: consumers need to see proof of benefits before AI is widely utilized in healthcare. Even GenAI comes with equal parts excitement and skepticism. Moreover, the industry’s substantial interest in AI (one report says the AI in the life sciences market is expected to reach $7.09 billion by 2028, growing at a CAGR of more than 25% from 2023 to 2028) has not yet resulted in widespread deployment.

Adoption and change-management roadblocks start and end with ROI. AI applications must be tied to use cases that demonstrate specific advantages, such as increased efficiency or commercial lift. Despite early examples of technical validation, broad industry adoption will inevitably cease if AI doesn’t provide value commensurate with the cost for deployment, training, and maintenance. The bar is even higher in clinical research, an evidence-generating industry that relies on concrete proof. Trial sponsors, contract research organizations (CROs), and sites all require proof of added value in everyday workflows.

If we cannot generate evidence that AI can deliver substantive and tangible value, it will not gain mainstream acceptance. To provide evidence of ROI—and ultimately lead to a sizable impact on clinical research—AI technologies must incorporate three fundamental characteristics: comparative baselines, value attribution, and data interoperability.

1. Comparative Baselines Provide Tangible Proof

Historical comparisons are often used to validate accuracy. In healthcare, this is referred to as “concordance,” as in this analysis: Concordance in Breast Cancer Grading by Artificial Intelligence on Whole Slide Images Compares With a Multi-Institutional Cohort of Breast Pathologists.

In a phase 3 hematology-oncology study, a clinical trial intelligence technology provider used a comparative baseline to validate an enrollment forecast algorithm. An AI model analyzed historical data (just as it would for a live study) and initially projected that the timeline set for the study was not adequate for enrolling the required number of participants, citing a zero percent chance of meeting the goal within the planned enrollment period. Additionally, the algorithm forecasted a more realistic timeframe. This forecast showed concordance with actual performance and was proven accurate: the model predicted the trial to complete at month 42, but in reality, it completed at month 43, a 40-day margin from the actual last patient enrolled.

Predictive models that indicate a project is off-track drive teams to take corrective action and change course. Naturally, this alters the outcome, making it difficult to gauge what would have happened without intervention, which is why historical comparisons are vital. Historical comparative baselines offer insight into potential outcomes in the absence of preemptive adjustments and ultimately provide tangible evidence of success. In this way, it is crucial for AI tools to leverage retrospective controls to prove their outcomes.

2. Value Attribution Drives Confidence, Trust

Determining the exact impact of an AI model in complex, interrelated systems presents a notable challenge. In clinical research, how can we confidently and solely attribute improvements to the deployment of an AI model? How do we know, for certain, that some other decision didn’t influence the outcome? For AI to be linked directly to ROI—which is critical for it to be adopted—AI must be traceable and not a black box.

One method is through the establishment of a comparative baseline (as noted above) while maintaining consistency across other variables—and therein lies the rub. Clinical researchers must contend with the inherent uniqueness of each study, compounded by the constant innovation in targeted treatments. For instance, in evaluating what influences patient enrollment and retention in a specific study, several factors—such as recruitment agencies, countries, and healthcare systems, both independently and in combination—impact outcomes. In the real world, there are many more variables to consider, necessitating a more complex experimental design to isolate AI’s contribution to a result.

Fortunately, multifactorial analysis, which considers the individual and combined impacts of input variables on outcomes, can help navigate this complexity. These analyses test the impact of each input variable (i.e., recruitment agency, country, etc.) on the outcome (i.e., patient enrollment), as well as every combination of those factors on the outcome (i.e., the impact of a particular recruitment agency in a specific country on enrollment). In this way, multifactorial analysis can identify the main factors, such as a major increase or decrease in enrollment rate for a region or a country, responsible for performance. Similarly, AI designed for multifactorial analysis can identify the contributions of specific AI models to performance on any given task—and with that hard evidence comes confidence in its value.

3. Data Interoperability and Sharing Fuels Sustainable AI Value

Data is the lifeblood of AI. Without it, AI models cannot learn or make accurate predictions. However, in clinical research, especially for rare and niche diseases, no single entity possesses all the necessary data. Collaborative efforts are thus essential.

The rapid development of COVID-19 vaccines was a testament to the power of collective action and data sharing amongst pharmaceutical entities, medical professionals, technology experts, and regulatory bodies. Such collaboration allows AI to achieve monumental tasks within clinical research.

The MELODY respiratory syncytial virus (RSV) trial illustrates the benefits of shared data. This study utilized federated learning to pool valuable protein data while safeguarding proprietary information. Protein-based drug development is typically a lengthy and costly process. However, with shared resources and the application of AI, the organizations involved in the MELODY trial experienced enhanced efficiencies, a feat much more difficult working in isolation. Federated models, such as those used in the MELODY trial, allow data to physically remain with the data owners, while also enabling compliant, shared access for all stakeholders.

In the long term, AI that draws from accurate and comprehensive sources enables a continuous learning cycle for increasingly more granular and accurate results.

Show Me the Money: ROI is the Key to Unlock AI Potential

AI is not just a passing trend. Rather, it’s a paradigm shift. Yet, a cautious and highly regulated clinical research industry will not accept AI without the same hard evidence that they seek in every trial. Value must be proven to shed any reservations about LLMs, generative AI, and any other form of AI. As with all things, only the best will survive. However, AI solutions that are consistently developed to show proof of value will unlock endless potential in clinical research for us all.