Studies that Show Drugs are Ineffective are Often Deep-Sixed

The production and distribution of scientific information has, of late, frequently become a broken process. Richard Horton, editor of the pre-eminent medical journal The Lancet, recently observed, “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

This is a problem in all of science. When it comes to industry-sponsered science, industry employs well thought out and highly researched processes for inducing scientists, doctors, and the general public to buy into highly biased ideas that are good for the industry's bottom line. This of course is true for the pharmaceutical industry just as it is with oil companies and the like. 

Not to mention the worst and by far the most dangerous-to-your-health offender of all, the managed care health insurance industry. George Dawson does an superb job tearing them a new one on his blog. But this post is about the pharmaceutical industry, other scientists who do drug studies, and the politics of acceptance of studies by medical journals. 

A well-designed study - manipulation of study design is a 'hole 'nother issue - that tries to measure whether a drug is effective for treating certain symptoms or a certain condition can turn out, broadly, in two possible ways. The drug is either shown to be effective for the symptom or condition, or it is shown to not be, or to be not very. 

Any single study's result may be invalid due to a problem with the sample of subjects picked, which may be unrepresentative of the whole population of subjects exhibiting the particular condition or symptom under study. It can also be just a coincidence, what with the standard 5% chance of that being so. So in general positive results need to be replicated in several different studies before the US Food and Drug Adminstration (FDA) will approve the drug for public consumption.

But what happens if a drug company or academic scientists who are fighting to get their studies published so they can attain tenure submit for publication only the studies that seem to show that a drug works, and then does not submit the other studies that had a negative result? Well, of course then it looks to everyone like the evidence for the drug's efficacy is much stronger than it actually is.

It is also true that, even when a negative study is submitted for publication, the editors of journals often reject the manuscript. Editors seem to think that, when a study did not show a positive result, there is no reason to publish it, supposedly because it does not tell us anything. That's ridiculous, of course - negative studies can tell us what is not true, and are essential to good science. 

This bias against so-called "negative" studies happens all over science, by the way, whether studies are industry sponsored or not. I remember renowned biologist Stephen Jay Gould decrying this in a book from decades ago.

In order to deal with the problem of negative studies not seeing the light of day as it applies to drug studies, a 2007 U.S. Federal law required study authors to report the results of all of their clinical trials to a public website. The website is clinicaltrials.gov, which draws 57,000 visitors a day, including people who are confronting serious diseases and looking for experimental treatments.

The law was enacted also because of public concern that a failure to report negative results could harm participants in similar studies by failing to warn them of possible risks.

The Food and Drug Administration Amendments Act requires sponsors of most clinical trials to register and report their basic summary results within 1 year of either completing data collection for the primary outcome or of terminating. Failure to report study findings is supposedly punishable by sanctions including civil penalties of up to $10,000 per day and loss of funding.

So how are we doing? Not so good.

According to a widely reported story, a study from Duke University finds that five years after the reporting law took effect, only 13 percent of scientists running clinical trials had reported their results! The article about this study was published online in the New England Journal of Medicine.

Only 13.4% of investigators reported their results within 1 year, and only 38.3% reported their results at any time during the study period (N. Engl. J. Med. 2015;372:1031-9). Moreover, “despite ethical mandates, statutory obligations, and considerable societal pressure, most trials that were funded by the National Institutes of Health (NIH) or other government or academic institutions ... have yet to report results at ClinicalTrials.gov, whereas the medical-products industry has been more responsive to the legal mandate,” the researchers explained.

Interesting that the pharmaceutical industry is doing somewhat (although not a whole lot) better than the NIH-funded scientists on this score.

At 1 year, the rate of reporting was 17.0% for industry-sponsored trials, 8.1% for NIH-funded trials, and 5.7% for other government- or academically funded trials. The corresponding rates of reporting at 5 years were only slightly better, at 41.5%, 38.9%, and 27.7%, respectively.

According to Clinical Psychiatry News, despite the regulation’s threat of penalties, no enforcement has yet occurred, the researchers noted, in part because this portion of the FDA Administration and Amendments Act is still under public discussion and hasn’t been finalized. 

Anyone who wants to contribute towards changing this situation can do so at Alltrials.net.

The lack of publication of negative studies is not the only strategy employed by big Pharma to bias everyone's impression of their drug's effectiveness study data. Some other tricks they employ include:

• Publishing positive studies more than once by using journal "supplements."
• Conducting a study at multiple locations and then publishing the results of the individual locations as if they were separate trials - and doing so selectively if that makes the drug look better.
• Publishing different measures of drug efficacy at differnt times to give the impression that the results published later are from a new or different study.
• Following study patients for longer and longer time periods and then publishing the results from each time period separately, again making it look like there was more than one study.
• Publishing positive results in major or more prestigious journals and negative or neutral studies in more obscure journals.
• Combining the results of multiple trials in ways that are more favorable than any individual study in its own right.

Let the buyer beware!