Biomedical research can yield awe-inspiring results. Incredibly precise gene editing tools. Sophisticated prosthetic devices. 3D printing of human tissues. Targeted immunotherapies for cancer.
Research in life sciences has created new industries with millions of jobs and saved countless people from pain, suffering and untimely death. We point with pride to the “gee whiz” scientific discoveries of recent decades, but in other areas, there has been a flat lining of research results without significant progress.
The reason? Out-of-control waste in both our public and private research institutions.
Research, including life sciences, is known to be chronically underfunded. Experts point out that shrinking public funding for science means that a very large number of meritorious research projects cannot get funding. At the same time, far too many of the research dollars we do spend are wasted on throw away experiments that can’t be repeated or confirmed.
Well-intentioned but ultimately useless research shows a variety of quality deficiencies that make many seemingly promising results non-repeatable. These quality deficiencies include, for example, insufficient sample size, misidentified cell lines, contaminated chemicals, inattentive statistical analysis and biased reporting.
As much as 50 percent of medical research is non-reproducible due to flaws in research quality.
Even prestigious academic and scientific journals such as Nature — intended to be the gatekeepers of quality — have fallen prey to poor science. In 2012, for example, a prominent team of researchers from the U.S. and China retracted their paper from Nature after both the authors and other researchers could not duplicate the results. The original paper stated that a novel approach, inhibiting the body’s production of a particular enzyme, can protect against cell injuries in stroke and heart attack. In their retraction, the authors said they continued to believe their approach to be effective but admitted that their initial finding “appears to be irreproducible.”
Even comedian John Oliver recently took a jab at junk science on his HBO program, “Last Week Tonight.” With his usual dry delivery, he described how a new study suggests a glass of red wine is just as good as spending an hour in the gym, that coffee will kill you and/or extend your life, and on and on.
Unfortunately, the situation is far from amusing. The most direct consequences of wasteful, non-repeatable research include useless spending of research grant funds, pointless use of animals in experiments, human participation in falsely started clinical trials, and futile attempts to commercialize non-performing results.
Eye-opening studies from Amgen and Bayer pharmaceutical companies point out the tremendous amount of money spent on trying to repeat non-repeatable studies for product development. These studies suggested that an astounding more than 50 percent of published research falls into this category. According to some estimates, the U.S. spends $28 billion per year on non-reproducible preclinical research; the global spending is probably twice as much per year.
According to Cleveland Clinic scientist Dr. Jeffrey Cummings, 99.6 percent of compounds found promising for the treatment of Alzheimer’s disease in preclinical experiments ultimately fail in clinical trials due to issues of efficacy. We can’t clearly decipher what they do because the initial experiments on animal models or cell cultures were probably either misguided by factors of non-repeatability or the experimental model was non-representative of the human disease. Trial and error are fine but people should be subjected to clinical trials when the likelihood of success is sufficiently high.
Another consequence is that today’s pharmaceutical industry is running on the fumes of decades-old work. My colleague Arturo Casadevall, a researcher and professor at Johns Hopkins University who also says he sees too many journal publications with results that can’t be repeated or confirmed, has a 93-year-old mother with heart problems whose medications are based on research that was already old when he was trained in the 1980s. Among the “blockbuster” drugs born of this old research are such long timers as the high-blood pressure medications Lasix, Cardizem and Propranolol; Warfarin, used to prevent strokes; and even good old aspirin.
The downstream consequences of non-repeatable research also deserve attention: misleading further studies (more wasted research), false hope and disappointment to patients and families, undeserved recognition and promotion for publishing irreproducible results, potential retraction and harm to reputation and erosion of societal trust in research.
Concerned discussions about research waste are no longer confined to the hallways of science. In fact, the National Library of Medicine, an arm of NIH, recently convened a two-day conference on the issue, discussing best practices of biomedical research, including ways to improve the reproducibility and transparency of preclinical research. Several findings emerged.
First, research is inherently innovative and often applies never used before methodologies. The overwhelming majority of quality defects are produced by well-intended researchers; intentional fraud is rare. Any attempt to solve the problems must involve collaboration with researchers, not blaming or shaming them.
Nonetheless, academic and research institutions need to change. One effective initial step would be to end the current “bean counting” of the number of publications and journal articles produced by researchers when considering them for promotion or tenure. Rather, schools should place more emphasis on evaluation of a researcher’s contributions to scientific innovation and societal impact. Future generations of researchers should be educated about quality improvement in science and the production of meaningful innovation.
The leadership of academic and research institutions should also take responsibility for managing quality improvement in the research enterprise. While many institutions pick up millions of federal dollars for research facilities and administration, too often their leaders pass the buck to scientific journals and unpaid reviewers to catch quality defects in the submitted scientific manuscripts. Obviously, it is already too late when an erroneous manuscript is submitted for peer-review – the bad research has already been done; the research dollars have already been spent. If we use the analogy of building automobiles, defects in new cars should not be caught in the dealer’s showroom but prevented from occurring in the first place at the manufacturing plant.
In fact, even a push to more rigorous standards of journal article acceptance won’t solve the problem. The most competitive scientific journals have also proven to be very vulnerable to accepting non-repeatable research reports (e.g., journals Nature, Cell, and Science).
The most crucial step, however, is for the academic and research establishment to come clean with itself on the crisis in research waste. The time for hushed tones is over. Acknowledging and confronting the problem of research waste head-on will not only mean that future research dollars are better allocated; it could lead to more future breakthroughs. Every defect is an opportunity for improvement.
In the 1950s, for example, Japanese cars were dismissed as junk — and rightly so. But after Japanese automakers embraced the need for change, they transformed themselves into some of the largest, most productive and most impressive automakers of the world.
Similarly, in the 1990s, the Institute of Medicine released a landmark report, To Err is Human, finding that medical errors are among the leading causes of death among Americans. Despite some initial resistance, the medical establishment has since made the reduction of medical errors a priority, saving thousands of lives every year.
No one-time effort will solve the problem for all times. New methodologies continually generate new sources of vulnerability, all the time. Continuous vigilance, dissection of errors and readiness to develop new protections will be needed as long as science is pursued.
The current quality crisis of research should give hope that the recognition of waste will not only lead to its elimination but also to unprecedented growth in scientific productivity and more amazing discoveries to promote health and wellness. With the emergence of continuous quality improvement in research, we can surely multiply chances for discovery and heighten optimism about the future of science and the difference it will make in people’s lives.
