Opportunities: A Regulatory Perspective on the Development of Suitable Clinical Outcome Assessments (COAs) for Rare Diseases

Ebony Dashiell-Aje
Clinical Outcome Assessments (COA) Staff, Office of New Drugs (OND), CDER, FDA

Sarrit Kovacs
Clinical Outcome Assessments (COA) Staff, Office of New Drugs (OND), CDER, FDA

ncorporation of suitable clinical outcome assessments (COAs) in clinical trials is one critical step toward patient-focused drug development in rare diseases (RDs). Opportunities in COA development for RDs, when considered thoughtfully, help ensure the likelihood of successful drug development within an ever-changing and competitive landscape.

There are nearly 7,000 RDs in the world; however, there is still a dearth of treatments for many of these diseases. RD drug development can be complex and it is important to acknowledge unique issues related to RDs when designing clinical trials. There can be a combination of factors, such as the absence of natural history data; limited knowledge of the likelihood, range, or course of clinical manifestations of the disease; and heterogeneity (e.g., phenotypic or genotypic variations, age, clinical manifestations, and rate of disease progression), making assessment of clinical benefit challenging. There are numerous opportunities to use suitable COAs to assess symptoms, signs, and impacts of RDs that are important and relevant to patients and support endpoints to measure clinical benefit. Innovation, judgment, and regulatory flexibility are all critical in facilitating the use of COAs in RD drug development, while at the same time adhering to regulatory requirements and good measurement principles in order to benefit patients.

Paving the Way for Suitable COAs for RD Trials

The United States Food and Drug Administration (FDA) has created two helpful tools to aid in COA selection, modification, development, and evaluation. The first tool is the 2009 patient-reported outcome (PRO) guidance for industry, which defines good measurement principles that FDA considers when evaluating whether PRO instruments (and other types of COAs) intended to support clinical benefit for patients are well-defined and reliable.

The second tool is the Roadmap to Patient-Focused Outcome Measurement in Clinical Trials (henceforth the “Roadmap”), which illustrates a process for COA selection, modification, and development that encourages alignment between drug development program objectives and patient-focused goals.

Topic #1: Understanding the Condition and Defining the Target Patient Population

As outlined in FDA’s Roadmap, prior to selecting and developing a COA, one must understand the disease under study. Drug developers should consider the etiology, natural history, and the age of patients (e.g., pediatric), as well as rate and variability of symptom occurrence in the specific patient population. There are large knowledge gaps related to the etiology and natural history of many RDs that are compounded by variations in disease genotypes and/or phenotypes. In addition, limited availability of patients or caregivers for qualitative studies make it difficult to understand the full spectrum of disease manifestations and patient experiences for assessment in clinical trials. Well-designed natural history studies provide an opportunity to evaluate a variety of COAs that can ultimately be used in clinical trials.

Considerations

• Leverage existing information through literature (e.g., observational studies, clinical case reports).
• Account for potential differences in disease manifestations, cognitive or linguistic developmental differences, and willingness, ability, and motivation to self-report by age subgroups.
• Generate qualitative evidence from clinical experts, patient partners, and other stakeholders on the most important clinical characteristics of the RD and the most appropriate patient population(s) to include in the study.
• Where possible, conduct well-designed natural history studies independently or through partnerships with patient organizations and/or utilize existing patient registry data.

Topic #2: Targeting the Appropriate Treatment and Measurement Concepts

Given the phenotypic variation within many RDs, different treatment goals and outcome assessments may be more appropriate for one subpopulation versus another. There is often uncertainty about aspects of the disease that are both affected by the treatment and meaningful to patients. Likewise, in RD trials, there might be a number of measurement concepts that are differentially important to measure across disease subtypes and at different stages of disease progression. This lends to poorly or broadly defined treatment targets that do not adequately represent what is expected to improve with a given treatment, making it difficult to identify the most appropriate COA measurement concepts that should be assessed in a clinical trial.

Considerations

Determining the most appropriate treatment target and measurement concepts requires a thorough understanding of a drug’s mechanism of action and what aspect of the disease the treatment is expected to improve. Therefore, one should do the following:

• Consider what relevant disease-related symptoms, signs, and impacts will likely be affected by the treatment, be most sensitive in detecting treatment effects, and reflect what patients consider to be meaningful.
• Determine whether measurement of frequency or intensity of concepts is appropriate.
• Leverage knowledge published in the literature, engage clinical experts, and gather qualitative evidence from patient stakeholders (patients, caregivers, and clinicians) to understand the different aspects of a disease that are meaningful to measure and matter most to patients.

Topic #3: Defining the Efficacy Endpoint

For many RDs, well-characterized efficacy endpoints appropriate for the disease are not available in the absence of a well-defined target population or treatment and there is limited knowledge of what COAs might be appropriate to measure relevant aspects of the disease.

Considerations

• Define endpoints based on clinical trial objectives and measurement concepts that are expected to improve or stabilize with treatment, adequately reflect patients’ symptom experiences (e.g., frequency and intensity, and chronic versus episodic), and are clinically meaningful to patients.
• When heterogeneity in disease symptoms, signs, and impacts exists, consider defining the endpoint based on measurement concepts that are most relevant to a broad range of patients, while including less common concepts lower in the endpoint testing hierarchy.
• Note that the reliability, validity, sensitivity, or interpretability of an endpoint may be different across patient subpopulations (e.g., early-stage or slowly-progressing phenotypes vs. severe, late-stage, or rapidly-progressing phenotypes). Therefore, consider including a diverse range of patients (e.g., with a heterogeneity of clinical manifestations), or narrowing your patient population to a specific subset in a clinical trial, whichever is most appropriate.

Topic #4: Determining the Adequacy of the COA

Detailed knowledge of the reliability and validity (including content validity) of a COA helps guide the decision to select, modify, or design a COA intended to assess clinical benefit. Unfortunately, few COAs have been developed or tested in RDs and there is often a lack of or very limited evidence generated to support the reliability and validity of COAs in RD patient populations. Importantly, COAs that are developed for use in clinical practice are not necessarily suitable for regulatory use.

Considerations

• Refer to FDA’s Roadmap and PRO Guidance to help guide COA selection, modification, and development.
• Engage early with therapeutic area and COA experts, who understand the nuances of disease progression and COAs in RDs, as well as FDA to get guidance on COA selection, modification, development, and implementation processes that will generate reliable and valid data.
• Consider the testing and thoughtful application of existing COAs developed in other patient populations that appropriately measure the concept of interest (e.g., physical function) for use in RD trials. Note that using COAs that have been developed in other patient populations “off the shelf” without careful thought and evidence to support their suitability in the RD patient population can lead to unsuccessful trials and/or difficulty with data interpretation.
• Consider patient burden (e.g., factors related to frequency of site visits, travel time for site visits, and potential patient fatigue) when conducting standalone studies to develop COAs and when administering COAs in clinical trials.

Topic #5: Scoring and Interpretation of Meaningful Change or Stability in Patients’ Scores

Given the challenges related to generating the empirical evidence necessary to determine whether a COA is suitable for use in RD trials, there continue to be issues related to (1) whether the COA score reliably and accurately reflects the concept of interest, and (2) the ability to interpret clinically meaningful within-patient change or stability, especially with small sample sizes.

Considerations

• Depending on the proposed concept of interest and conceptual framework of the COA, consider whether individual items, domains, or total scores are more appropriate to define the endpoint.
• Use qualitative interview data from patients or caregivers along with quantitative evidence to determine what constitutes stability, worsening, or a meaningful improvement in COA scores. This is particularly useful when small RD sample sizes present challenges in score interpretation.
• Consider conducting exit surveys or interviews to obtain qualitative data from patients or caregivers on (1) what constitutes a meaningful improvement from baseline in COA scores, (2) whether they believe stability or a meaningful improvement has been achieved from baseline, or (3) if they perceive that a meaningful aspect of the disease has improved or deteriorated but was not measured in the trial. This can help in designing future studies.

Meeting Challenges Head-On: Flexibility and Partnerships

It is important to first understand the disease or condition and then conceptualize clinical benefit before finally focusing on COA selection, modification, or development intended to support clinical trial endpoints. However, many considerations related to RDs must be addressed to maximize the success of a drug development program. We encourage engagement early and throughout the drug development process to optimize a partnership to finalize COA measurement strategies in RD trials. We are open to considering multiple sources of confirmatory evidence to aid in data interpretation (i.e., COAs and clinical trial endpoints). We also encourage drug developers and other stakeholders to collaborate early in instrument selection, modification, or development efforts to advance the science of patient-focused drug development in RDs.

References available upon request.

This report reflects the views of the authors and should not be construed to represent FDA’s views or policies.