A piece by Keith Humphreys showed us meta-analysis that reveals Omega-3 supplements that were previously thought to lower the risk of having heart attack or stroke actually do not have that therapeutic effect. As sample size of the study participants increased, beneficial effect of the fish-oil decreased to nothing. So it seems like we spent a massive amount of resources for a pill that does not do anything for us.
As Aaron Carroll has mentioned, it costs millions of dollars for a research involving number of participants just under hundred people. To put into better perspective, let’s calculate how much money we spent on Omega-3 research. A very rough estimate of the cost of a 100-participant study is $1 million (as detailed in Humphreys’ piece). Wherever the funding may have come from, private or federal, would then sum to about $4 billion over 15 years—$250 million per year—even leaving out all the other research that had not made it to publication, on what amounted to a wild goose chase.
Now, that is a fairly crude estimate, and of course retraction of a wrong scientific discovery is important for the progress of science, but one hopes we could find some way of finding null results that did not have such a colossal price tag. Yet, one thing is clear: the supply of medical research money is not bottomless. Money spent developing ineffective supplements is money that could have been spent on discovering new antibiotics or designing new biomedical equipment.
However, there is a reason for this wasteful spending in research. Because the cost of conducting quality research with large pool of participants, which is crucial to produce a conclusive result, is so enormous, researchers generally have to stick with small number of subjects at the initial stage of finding something new and interesting. As Mr. Humphreys has stated, studies that produce original findings have much better chance to gain traction (such as healthy effects of Omega-3) in the academia while identical topics with identical number of small pool of participants (just under 100) with conflicting results (such as null effects of Omega-3) are too dull to be published.
Thus, any research that somehow has resulted in compelling findings will gain the spotlight even though the number of participants does not sufficiently legitimize the true effect of newly discovered intervention. Only after the idea is published and blown up to catch funders’ attention are researchers able to even think of applying for a grant that could support more rigorous research. Therefore, to break this cycle of new medicines that are quickly discredited, scientists would have to be able to garner lots of funding for the research before it even produces eye-catching, breakthrough results.
But what funders would give money to some far-fetching, risky idea that might turn out to be non-marketable and nothing more than a bust? Is this cycle of waste that is inevitable and just a part of damage our society has to bear for the exchange of hope of finding new treatments? Well, we shouldn’t give up so easily, and Mr. Carroll has some encouraging suggestions. He refers to Dr. John Ioannidis who suggests that much of the money spent on huge trials could be saved through efficiency measures:
A data-intensive mega-trial of 20 000 randomized participants with 4 years of follow-up costs an estimated $420 million, but streamlining the design, monitoring, data collection, and outcomes could save 90% of that cost. For a blockbuster with $2 billion annual sales, 1-month sales ($167 million) would suffice to conduct a mega-trial of 80 000 participants.
I should note that this solution is essentially skipping intermediate researches that involve moderate-sized pools of participants, but since those generally do not produce conclusive results, we might as well skip to the endgame.
This could be a scary idea for pharmaceutical companies which fund and conduct most of drug development researches. If the drug turns out to be pointless, they just have burned their time and money for nothing. They must be sorely tempted to push probably-bogus drugs to quickly generate a profit before follow-up researches prove its ineffectiveness.
Yet, at the same time, companies have vested interest in developing drugs that actually do work and that could become backbone of their long-term growth. Same goes for the public—they would prefer to see the drug in the market that actually works rather than the one that comes and goes. After all, drugs are not like fads in trousers. They are products that deal with people’s health and lives.
If any drug company is just exploiting the public’s willingness to pay for products that could potentially improve their health in exchange for its short-term profit, there is something seriously wrong with rules that protect consumers against such an action. Drug companies will not prosper in the long run if they always resort to foisting a rapidly changing set of treatments that turn out to not work.
