Regulatory Perspective: Digital Health Technology Tools

t FDA, there is a focus on evaluating and considering valuable novel approaches and tools that can support our regulatory decision-making. The dramatic growth and incorporation of digital technologies into daily life has afforded new opportunities to use these tools to understand patients’ functioning and how it is affected by different diseases and treatments.

Former FDA Commissioner Scott Gottlieb has made statements regarding advances in the field of digital health innovation, including launching the Agency’s Digital Health Innovation Action Plan. Likewise, the US President’s budget includes a proposal to create a Center of Excellence for Digital Health. In the current landscape, the Agency is receptive to innovative, technology-derived, study endpoints in various therapeutic areas.

Why Digital Health Technology Tools (DHTTs)?

Digital health technology tools (DHTTs) can help capture patient experiences in real world settings. DHTTs can include, but are not limited to, “wearables” (e.g., accelerometers), implantable or ingestible sensors (e.g., blood glucose monitors), and stationary sensors (e.g., home-based motion sensors to detect gait patterns and capture falls) that are designed to generate data not observable in traditional clinic settings. (For the purpose of this article, the term DHTT refers to an electronic technology tool, its score(s), and the interpretation of its scores that are intended for use in clinical investigations.) DHTTs can passively capture data (e.g., through accelerometers, cardiac rhythm monitors) or capture data through patient responses (e.g., an electronic patient-reported outcome [ePRO]).

In addition to providing richer and more comprehensive information on how patients are functioning and feeling, DHTTs can help minimize barriers to obtaining patient experience data during clinical investigations in many ways. For instance, DHTTs can be operated and accessed remotely, reducing both clinical site costs (e.g., decreasing the number of clinical site visits) and patient costs (e.g., travel costs, if the study requires fewer visits). Remote data collection using DHTTs has the potential to help streamline study and data monitoring procedures; help maximize recruitment efforts among hard to reach, geographically dispersed, or rare patient populations; or generally increase the chances of patient enrollment and retention in a study through decreased burden of study participation.

DHTTs to Evaluate Clinical Benefit

DHTTs may be used to assess study endpoint concepts that are meaningful to patients. Data derived from DHTTs can serve as either clinical outcome assessment (COA, including data generated via patient-reported outcome [PRO]; clinician-reported outcome [ClinRO]; observer-reported outcome [ObsRO]; or performance outcome [PerfO] instruments) or biomarker data (see table below). DHTT COA data can help describe how patients feel or function in daily living and whether a treatment provides clinical benefit to patients. DHTT biomarker data can provide information related to patients’ physiologic characteristics and may have the potential to predict clinical benefit to patients (e.g., when used as part of a validated surrogate endpoint). For the purpose of this article, we will discuss DHTTs used to generate COA data. Definitions for many of the terms in this article can be found in the FDA-NIH BEST Glossary.

DHTT data alone may not provide a complete picture of how patients are both feeling and functioning. For instance, while passively collected DHTT data (e.g., walking speed) may help assess aspects of patient functioning in real world settings, it would be most useful to pair these data with traditional COA data. A PRO (Patient-Reported Outcome) would be needed to accurately reflect how patients feel in order to measure pain or shortness of breath while walking or to assess if any of the observed changes in functioning are meaningful to patients.

Regulatory Standards: Evidence to Support Regulatory Decision-Making and Labeling Claims

When evaluating the appropriateness of a DHTT-based study endpoint to inform regulatory decision-making and labeling claims, evidentiary standards that are broadly applicable to other types of outcome assessments are applied. When deciding whether to use a DHTT in a clinical investigation, some factors to consider include: what you want to measure (e.g., concept of interest); the study population and other study design aspects; endpoints being examined; and DHTT design, implementation, and analysis.

Key Concept of Interest Considerations

Input from key stakeholders (e.g., patients and/or their caregivers) through interviews and/or focus groups can inform what concepts are most relevant and important to measure in the proposed study population, including what patients would most like to see improved with treatment (e.g., walking speed versus walking distance), and what would constitute a meaningful improvement in that concept; this input can also help inform whether patients can use the DHTT as intended. This input is most useful when obtained early in the medical product development process, including the precompetitive setting.

Key Endpoint Considerations

It is important to construct an endpoint(s) that is well-defined, reflects an outcome that is clinically meaningful, and can be statistically analyzed with regard to its measurement properties (i.e., descriptive characteristics, reliability, validity, ability to detect change, etc.). DHTTs can generate a large volume of data, therefore, it is important to determine (a priori) how DHTT data will be aggregated to generate a clinically meaningful score that can be used to define an endpoint.

Key Study Population Considerations

When determining whether a given DHTT is appropriate to use in the proposed study population:

• Consider the technical aptitude, ability, and willingness of the study population to use the DHTT in the context of a clinical investigation.
• Consider conducting usability testing in patients (and/or caregivers, if they are helping with collection of the patient DHTT data) who are representative of the study population (where prior evidence has not been generated).
• Note that elderly or disabled patients may require tools with larger text, buttons, or screens, and special tools may be necessary for patients with conditions that limit their dexterity.

Key DHTT Design, Implementation, and Analysis Considerations

Instrumentation and Instrument Validation. Device calibration and evidence to support the DHTT’s ability to yield consistent, reliable data and to generate accurate, valid data (e.g., that 100 steps = 100 steps recorded by the device) is important. Likewise, evidence is necessary to support that the device is measuring the concept of interest in the study population in a reliable and valid way.

Variable Selection and Endpoint Definition. Data from qualitative research and the literature can be used to evaluate whether the concept(s) of interest and variables selected are appropriate for measuring clinical benefit in the study population and to support the proposed study endpoint(s). These variables are most informative when they are specified a priori, clearly defined, and represent the most important concepts to measure that can show change with treatment.

Data Collection Characteristics. Clearly specifying details regarding the data collection location (e.g., at home), the duration of monitoring, and days of the week for monitoring is important. Note that the potential benefit of collecting more hours of usable data can be weighed against the following:

• Participants’ time
• Battery life
• Data processing time
• Risk of missing data or lost DHTTs

Maintaining consistency in DHTT assessment days of the week for each patient throughout the study can improve data comparability across timepoints.

Data Processing, Scoring, and Analysis. Data processing elements include:

• Time intervals for data transfer (e.g., whether data will be automatically uploaded continuously, every evening; how episodic events will be captured).
• Note that frequently scheduled data transfer may reduce risk of missing data compared with one data transfer at the end of a study. Safeguards can help ensure secure and reliable data transfer.

Data scoring and analysis elements include:

• Methods utilized to score and analyze raw DHTT data; how the data will be analyzed according to the characteristics of the data (intensity, frequency, event, etc.).
• Scoring criteria (often provided by manufacturers) are necessary to pre-specify and finalize a priori how data will be aggregated to generate a score; these criteria must be followed exactly.
• The DHTT score should measure the right concept(s), in the right way, and be able to capture improvement in scores due to treatment.
• Missing data must be documented, carefully examined, and explained; plans for handling missing data must be pre-specified. 
• Suitable anchors should be identified to help determine clinically meaningful within-patient change thresholds on the DHTT score(s).
• Note that statistically significant changes in a score alone are not fully interpretable. A very small change in score that is statistically significant does not necessarily connote a meaningful within-patient improvement.

Looking Ahead

Experts, including CTTI’s Mobile Clinical Trials (MCT) program, have developed recommendations and resources regarding use of DHTT data to increase the quality and efficiency of clinical investigations. However, further experience with the use of DHTTs in clinical investigations is necessary. As with any proposed endpoint measure in a clinical investigation, we recommend frequent engagement with the FDA early in the medical development process to discuss inclusion of DHTTs in studies intended to support regulatory decision-making and labeling claims.

Disclaimer: This publication reflects the views of the authors and should not be construed to represent the views, policies, or guidance of the FDA.