eal-world data (RWD) offers the possibility of providing important information to help inform regulatory decision-making. But turning this possibility into reality still depends on the quality and reliability of the data. With appropriate efforts to ensure data quality, RWD can support regulatory decisions in Japan (and other countries/regions as well). Close collaboration between health authorities, study sponsors, and data owners is critical in better understanding and overcoming issues in making this possible.

In September 2022, MHLW also released the administrative notice Questions and Answers (Q&A) on Points to Consider for Ensuring the Reliability in Utilization of Data from Registry or Medical Information Database in Applications for Marketing Approval and Re-examination for Drugs. It stated that if data quality management was not performed sufficiently by the registry/database provider for the data items that the applicant intended to use for a regulatory application, then these data can only be used for regulatory applications if the registry/database provider took additional actions to ensure that quality management of these data items was sufficient.

One example of using RWD in a study is to design a prospective trial for one group receiving treatment with the investigational drug and deriving the data for the treatment-naïve group or a group treated with an existing comparator from their historical (RWD) data.

In Japan, RWD means accumulated medical information obtained from routine medical practice. Medical information including patient records, laboratory test values, image data, prescription information, etc., are collected and stored either electronically or on paper at medical institutions. Since universal health insurance is established as a social insurance system, medical expenses are almost always covered by the government and the social insurance medical fee payment fund in Japan. For the government’s payment, the National Health Insurance Medical Fee Review Committee commissioned in each prefecture confirms the medical treatment and/or medications to determine appropriate reimbursement for them. This Review Committee only determines the appropriateness of the insurance coverage in question. Unless the Review Committee has specific questions about it, they do not ask medical practitioners to verify the data. Therefore, medical information is unlikely to be corrected even if it has obvious inconsistencies. Database providers often prepare their databases by pooling medical information and health insurance claims data from medical institutions that include these inconsistencies. This is a primary reason why the need to ensure data quality and reliability of RWD is so urgent in Japan.

From a different perspective, the US does not have a universal health insurance system. Except for the elderly and low-income patients who are insured by the government (Medicare and Medicaid programs, respectively), individuals receive insurance coverage through their employers or by purchasing insurance plans from private health insurance companies. These insurers require specific medical practice and treatment information when managing, reimbursing, and providing their coverage. Thus, as a result, in the US, RWD collected by private health insurance companies are likely to be more standardized and feature fewer inconsistencies.

Quality of RWD and Clinical Trial Data

Clinical trial data, on the other hand, are collected on standardized Case Report Forms (CRFs). Data quality control is normally performed at the time of data acquisition by multiple personnel. CRFs are carefully completed by the investigator and/or the study coordinator and signed by the principal investigator for confirmation, while the clinical monitor appointed by the sponsor checks CRFs for consistency with source documents. The data manager also reviews the data in the CRFs to make sure that the data provided are consistent with expected results, that the subject met the inclusion criteria, that the protocol was adhered to, that concomitant medications do not contradict any complications, etc. Computerized clinical trial and/or data management systems are generally used to manage these processes.

Conversely, in the case of RWD, data quality control is not always concurrently performed at the time of data acquisition. If a sponsor needs an RWD data set for use in a clinical trial, they may request the provider to prepare the data set by selecting specific data from their database.

That database provider might have performed their own data quality management procedures, but they may not necessarily have followed the sponsor’s standard operating procedure or other data quality or management procedures. To use these RWD for regulatory applications, additional quality-control activities are required. Operating procedures for data handling are fundamental, as stated in the MHLW’s administrative notice. Database creation and update should be performed through pre-defined procedures (Wakabayashi, et al., 2021). Daily and periodic monitoring are also effective measures (Yamaguchi, et al., 2019). Even under well-defined processes, however, some data may be missing or incorrect because of human error, program error, etc. In such cases, the solution would be to reconfirm the program or operation and request the medical institution that initially provided the relevant data to verify it (Yamaguchi, et al., 2019).

Regulator and Sponsor Initiatives

Many patients in Japan, as elsewhere, do not participate in clinical trials. As a result, it is necessary to devise ways to ensure the reliability of RWD that complements clinical trial data. In developing Japan’s regulatory framework, MHLW suggests that trial sponsors consult with PMDA (the Pharmaceuticals and Medical Devices Agency) early in the process of leveraging RWD in a regulatory application. Defining the essential data elements of the trial is critical.

Applicants are encouraged to review the database provider’s quality-control procedures and implementation frequency and to ask the registry/database provider to perform additional quality control if these seem insufficient in generating “fit for purpose” RWD.