Special Section: Pharmacovigilance

scientist pointing to graphics of scientific icons
Monitoring Safety Signals for Vaccines: Current Strategies and Future Directions
Kausik Maiti
Neeraj Parmar

he role of vaccines in preventing infectious diseases globally has become more important than ever in the COVID-19 pandemic. Safety has always been a key focus in vaccine development, but vaccines (like all medical products) are not without risks of adverse effects. Along with meticulously conducted clinical trials, a robust post-market safety surveillance system is essential to ensure safety throughout the vaccine lifecycle. Efficient and scientifically sound signal detection systems and responsible communication also play a significant role in vaccine safety.

Signal Detection for Vaccines

There is a considerable overlap between vaccines and drugs in signal detection methodologies and approaches. However, signal detection for vaccines presents a number of unique challenges.

Adverse events following immunization (AEFIs) require careful evaluation. AEFIs can often be attributed to various factors such as background incidence of the reported event; error in vaccine preparation, handling or administration; reaction to the injection; and/or reaction to the vaccine antigens, adjuvants, or other constituents (e.g., preservatives or stabilizers). Arriving at a common understanding of AEFIs across regions and settings is critical. Brighton Collaboration case definitions (typically structured with multiple levels of diagnostic certainty) are useful in identifying cases and determining their diagnostic certainty.

Amongst various quantitative and qualitative signal detection methods, observed versus expected analysis is one way of detecting signals related to adverse events of special interest (AESIs). The number of cases observed in temporal relationship to vaccination is evaluated against the expected number of occurrences estimated from background incidence of the event of interest and projected exposure from the number of doses distributed. Particular attention should be given to ensure age-appropriate selection of background incidence rates and equitable time periods between observed and expected. Pitfalls of this evaluation method include underreporting of AEFIs in the Spontaneous Reporting System (SRS) databases and nonavailability of true exposure data.

Currently, statistical data-mining algorithms use a variation of an observed-to-expected calculation based on the assumption that the accumulated case reports in SRS databases can be treated like a cohort for statistical analysis purposes, even though they do not represent a cohort in a traditional epidemiological sense. Disproportionality analysis is performed to estimate whether the number of spontaneous reports received is greater than what might be expected by chance. Disproportionality analysis tools are not meant to be used in isolation but can be very powerful additions to our existing array of traditional methods for safety signal detection.

When an individual vaccine is recommended for specific age groups or populations, a stratification procedure is usually incorporated in the disproportionality measure to avoid confounding by known variables such as age, gender, ethnicity, or regional distribution. However, observed-to-expected ratios are quite susceptible to the effects of overstratification, and stratification must be carefully designed to avoid very small strata. When appropriate, both stratified and unstratified scores should be monitored.

Another useful approach for detecting vaccine signals is time-to-onset analysis. This includes analysis of time-to-event distributions for other events following exposure to the same vaccine, and the time-to-event distributions of AESIs after exposure to other vaccines.

Event clustering algorithm is another methodology employed to monitor the safety of COVID-19 vaccines. This consists of identification of natural groupings of adverse event terms through cluster analysis of the terms themselves based on empirical reporting patterns or semantic similarity, as well as deep neural networks. This can provide insights into which adverse event terms tend to co-occur or be used in a similar clinical context but does not in itself enable unsupervised case series identification.

Choice of comparator data set (combined drugs and vaccines versus all vaccines versus selected age-appropriate vaccines) is another important consideration. Given the uniqueness of vaccines (such as the prophylactic and often universal use), it seems logical to use the “vaccines only” database for primary comparison. However, the challenge of using “vaccines only” as the comparator dataset is that events commonly reported following most vaccinations might be missed. Additionally, small databases can be especially prone to the effect of masking because of the relative lack of diversity of events or products. In addition to comparing vaccine-specific databases, it is recommended that additional screening be performed against large databases containing both vaccines and drugs.

Real-time assessment of real-world data from various linked, large data sources (e.g., hospital, pharmacy, immunization, birth, and death) is especially useful in early detection of safety signals as shown by the Rapid Cycle Analysis of the Vaccine Safety Datalink (VSD) in the US.

Current Challenges

Underreporting, lack of uniformity, and inconsistent data quality received in the SRS pose significant challenges in safety evaluation for vaccines. Uniform, global reporting practices along with increased awareness of the need for quality reporting (especially in low- and middle-income countries, or LMICs) are already in focus. Duplicate case reports within SRS create noise for signal detection and assessment, but removal of duplicate case reports can be challenging within vaccine SRS databases when population-based vaccination programs are ongoing.

Quantification of risk is difficult to estimate without the availability of true incidence. In the context of COVID-19 vaccines, benefit-risk quantification will continue to be challenging until complete information on benefits is available (e.g., the duration of protection provided by the vaccines, and the duration of COVID-19 prevalence). Effective and responsible public communication regarding safety of vaccines is challenging but critical to the success of these vaccination programs.

India as a Case Study

India has an established AEFI surveillance system from the grassroots to the national level. A National AEFI Committee reviews causality assessment of AEFIs regularly, the signal detection system is active under the Pharmacovigilance Programme of India (PvPI), and there is collaboration between various stakeholders and with the WHO. Underreporting is still an enormous challenge. Collaborations between government and professional bodies such as the Indian Medical Association are being created and/or strengthened to sensitize healthcare professionals on the importance of adverse event reporting. The CoWIN web and mobile phone app is a great example of technology-enabled comprehensive data collection of vaccine recipients during India’s massive COVID-19 immunization program. If such robust data-collection practices are applied across all countries, it can provide meaningful baseline information that supports further analysis of signals, and quantification and management of risks.

Future Directions

Further global collaboration is the way forward for a harmonized vaccine safety database containing datasets from all geographies. Artificial intelligence can be used to achieve clean datasets by removing duplicates and reconciling products. Increasing awareness for safety reporting is the key to mitigate underreporting. Mobile-based applications for adverse event reporting will further facilitate reporting.

Efforts to bring together real-world data from many countries hold promise for a Global Vaccine Safety Datalink. Additionally, social media mining can be explored for early indicators of safety signals.

Identifying host factors that predispose to rare serious adverse effects of vaccines could help to clarify whether they are a result of genetic predisposition. This may introduce personalized vaccination approaches to enhance vaccine safety focused on either vaccine modifications for individuals or individual exclusions based on genetic profiles.

Ultimately, these advancements would help maintain public confidence in vaccines, facilitate development of safer vaccines, and further advancement in public health.

References available upon request.