ranslational medicine and precision drug development are plagued by the irreproducibility of fundamental results published in the scientific and medical literature. That irreproducibility is often the result of neglect of the scientific method. In one published study, in 89% of cases, the authors of the original publication were unable to reproduce their preclinical results when the repeated experiments were performed in a “blinded” manner.

Upholding research integrity and improving the quality of research that underpins clinical studies is critically important. This article outlines some of the considerations that could help improve the research enterprise and illustrates the crucial role laboratory scientists play in this endeavor.

Within pharma drug discovery R&D, the scientific team closely monitors the scientific literature for new preclinical discoveries that could provide the basis for therapeutic programs that might improve outcomes for cancer patients. As a first step, pharma scientists attempt to reproduce the reported findings and if unable to do so contact the authors of the original publication to seek their assistance.

After a Confidentiality Agreement (CDA) is set in place, the authors of original scientific research published in the top-tier journals often welcome scientists into their laboratory to allow the visitors to observe the experiments being repeated and to demonstrate that the reported research findings could be repeated. In one published study, in 89% of cases, the authors of the original publication were unable to reproduce their preclinical results when the repeated experiments were performed in a blinded manner. This was a staggering result. Because of the CDAs and the understanding of how the visits had been presented, the identities of those laboratories were not made public. However, similar results had previously been reported by scientists from a different pharma company. Multiple groups have subsequently reported a similar experience in fields including psychology, economics, medicine, and machine learning. Taken together, this demonstrates that there is a widespread problem that currently pervades research in multiple fields.

These data are difficult to reconcile with the widespread belief that medical research is generally of high quality reflecting the investigators’ desires to make genuine, reliable discoveries that have a real impact on human health. They raise the question of how data make their way into a peer-reviewed publication and in so doing force a consideration of the unfortunate incentives that force academic researchers to adopt the “publish or perish” mentality. It also compels contemplation of measures that could serve to help address this problem.

What follows are some considerations that could help improve the research enterprise. In so doing, the crucial role of the laboratory scientist in this endeavor is underscored.

Basic Research Scientists Play a Crucial Role in Ensuring Research Integrity

When thinking about translational research as a stepping-stone to providing a therapeutic product, device, or test that will ultimately serve patients, a long list of essential elements are necessary to attract investor support and funding and achieve regulatory approval. But these seem to have little to do with basic research. In fact, one might even assume that there is no role for basic research scientists in translational research efforts. That is far from the truth: the role of the basic research scientist is crucial in providing a solid foundation for any translational and clinical development program. Their input and perspective ensure the quality of the preclinical research that underpins translational efforts.

The issues that lie behind irreproducible, poor-quality research include subjective endpoints analyzed by nonblinded researchers, failure to repeat experiments, failure to include positive and negative controls, use of nonvalidated reagents (e.g., antibodies, inhibitory RNAs, and small molecule inhibitors that can all display nonspecific or off-target effects), failure to show all the relevant data generated, and inappropriate data analysis (DOI: https://doi.org/10.7554/eLife.71601; https://doi.org/10.1038/497433a). These deficiencies are evident even in top-tier peer-reviewed journals that are considered to be of high impact.

“The most selective journals frequently require results to be packaged as excessively tidy stories that claim to lack ambiguity or uncertainty, often promising to transform the field and lead to therapeutic outcomes,” explains Jeffrey Flier, endocrinologist and former dean of Harvard Medical School, in his 2022 article. Perhaps because of the desire to present “tidy stories,” nonreplicable publications are cited more highly than replicable publications. This might be because when results are more “interesting,” lower standards are applied regarding their reproducibility.

Although there are emerging measures that place less value on the total number of publications (e.g., “H-index,” “citation index”), an ongoing challenge for the academic research scientist is that success may still be measured in terms of number of publications in top-tier journals and success in grant applications, with little acknowledgement of their contribution to a major therapeutic advance. This places an understandable stress on the academic research scientist to deliver against the incentives that will be used to judge success.

To be clear, developing a potential therapeutic is not in the job description for a basic researcher. To do so might demand resources that are unavailable to the academic researcher. Even the need to demonstrate that their important, novel finding in one model system can be generalized to other systems as a first step toward a therapeutic may be out of scope of the initial research grant. What the researcher, however, can ensure is that their work is well designed, well executed, and reproducible within the system they have examined and then accurately placed in context.

Although scientific journals and editors clearly bear some responsibility for the quality of publications, it is important for scientists to avoid the temptation to solely blame the editors and scientific journals. The peer-review process was created with the goal that manuscripts would be carefully reviewed. However, publication of papers with obvious flaws indicates that many reviews are superficial and fail to highlight fundamental methodological deficiencies. Reviews themselves are rarely conducted in a blinded fashion, and therefore reputation or perceived influence may impact a reviewer’s mindset. As a result, “famous” researchers and well-known institutions may receive cursory and inadequate reviews. This problem can at least in part be mitigated by capable and thoughtful basic research scientists who have the knowledge, skills, opportunity, and responsibility to directly address concerns about data quality when reading and reviewing publications. The “Open Science” movement is seeking to address and overcome these problems by making scientific research and its results widely available. The movement advocates for greater transparency, data sharing, open access to published results (rather than having results sequestered with access only upon payment to the journal), and processes that make publishing and communication of results more readily available. This important initiative is to be applauded and has already received endorsement and support from some major national funding agencies.

Basic research scientists can take the initiative and decrease the “currency” of poor-quality publications and improve the robustness and integrity of basic research that underpins translational research. By way of example, research scientists can do the following:

• Read papers diligently before citing them, not just the abstracts. This could be instituted by scientists immediately and would be very valuable in directing attention away from poor-quality publications.
• Not cite papers from famous institutions and researchers by default. Understandably, some researchers will feel compelled to cite publications from scientific leaders despite quality issues. This could be because failure to cite such a publication could be viewed as ignorance or even insolence. Some may even fear retribution by those who might review the researcher’s own grant applications or publications in the future.
• Not accept the journal’s reputation as a surrogate for quality. This is particularly important when reviewing grant applications or CVs for promotion or potential appointment of staff. Rather than simply accepting an interesting, provocative, or exciting result, it’s important to focus on reviewing the Materials and Methods, with an emphasis on how experiments were actually performed and results generated (e.g., were experiments repeated, subjective endpoints evaluated blinded, positive and negative controls included, reagents validated? etc.). Conversely, it’s equally important to perform and report experiments properly themselves, i.e., without cherry-picking results that tell the best story, p-hacking by reanalyzing data, or HARKing (“hypothesizing after the results are known”). These corrective actions may only be possible, however, if a researcher’s home institution encourages and introduces processes that recognize research quality, and rewards scientists who undertake robust, rigorous research and perform careful and thoughtful peer-reviews themselves.

Research Institutions Play an Important Role in Driving High-Quality Research

• ensuring that principal investigators and trainees undertake annual training on research ethics and research methodology;
• creating and resourcing an Office of Research Integrity;
• providing a mechanism for students and post-doctoral fellows to seek guidance when concerned about substandard research practices;
• instituting random reviews of laboratories to assess research practices;
• providing genuine consequences for poor-quality research (e.g., loss of privilege to train students, loss of ability to submit grants, loss of laboratory space);
• requiring compliance with guidelines (e.g., ARRIVE Guidelines) and data sharing;
• establishing formal, rigorous review of publications prior to submission;
• limiting expectations about Impact Factor and the number of publications a researcher can publish (e.g., two or three publications maximum per year) to help drive a focus on quality rather than quantity;
• rewarding and recognizing staff who demonstrate high-quality research; and
• assuring realistic public statements that do not exaggerate the significance of a discovery or related findings nor the timeline estimate from discovery to clinical implementation.

In conclusion, there is a role for all participants in the research enterprise to make a contribution to ensure that research ultimately brings value to patients. However, at the point of conception it is the laboratory researcher who is central in seeking to fulfil this objective. It is the rigorous, careful, self-critical laboratory scientist who provides the foundation upon which the translational process is built.