Cognitve Behavioral Therapy "Evidence-Base" Grossly Exaggerated

In my post on my Psychology Today blog on November 21, 2011, I discussed how the purveyors of today’s most predominant psychotherapy methodology, cognitive behavioral therapy, grossly exaggerate the strength of their research evidence base in the psychotherapy outcome literature.

My opinion was recently confirmed in a review of meta-analyses of the CBT literature in the Journal of the American Medical Association, published online September 21, 2017 (“Cognitive Behavioral Therapy the Gold Standard for Psychotherapy:  The Need for Plurality in Treatment and Research” by Falk Leichsenring and Christiane Steinert).
 
They reported that a recent meta-analysis using criteria of the Cochrane risk of bias tool reported that only 17% (24 of 144) of randomized clinical trials of CBT for anxiety and depressive disorders were of high quality. The “allegiance factor”—study authors were CBT therapists themselves and often designed the studies to make their treatment look better than it was, and opposing treatments look worse that they were—was rarely controlled for.

Compared with "treatment as usual" —letting subjects get whatever other treatments outside of the study treatment that they chose to have, allowing good therapists and bad therapists, and good therapies and bad therapies, to essentially cancel each other out—the sizes of treatment effects were only small to moderate and might eventually even be found to be due to the allegiance effects.

In panic disorder, CBT was not more effective than treatment as usual but only to being on a waiting list.

Even with these amazing biases, for depressive disorders, response rates of about 50% were reported. This was true for anxiety disorders as well. “Response” just meant there was some significant improvement in symptoms, not that the symptoms of the disorders actually went away. Rates for actual remission from the disorders were even smaller. Conclusion: a considerable proportion of patients do not sufficiently benefit from CBT.

Last but certainly not least, there was no clear evidence that CBT was more effective than other psychotherapies, either for depressive disorders, anxiety disorders, personality disorders or specific eating disorders.

Personally, my biggest beef with CBT and other psychotherapy outcome studies has less to do with symptom relief than with actually changing maladaptive interpersonal behavior. The latter is almost never even looked at, let alone measured in these studies.

CBT’ers seem to think anxiety, depression, and self-destructive behavior are all due to screwed up thinking by individuals rather than being normal reactions to stress-inducing environments. In experimental psychology circles, this is known as the fundamental attribution error. Telling people with these particular symptoms that their problems are basically “all in their heads” in this manner is very invalidating for them.  Ironically, an ‘invalidating environment” is one of the two primary factors these very same therapists cite as the main causes for borderline personality disorder.

Randomized Controlled Studies vs. Widespread Clinical Experience: the Case of Lamictal

There has been an increasing debate about whether doctors are using "evidence-based medicine" or instead are merely going by conventional wisdom or listening to pharmaceutical company sales pitches. Often, as I have discussed several times, the so-called "evidence base" consists of randomized controlled studies (RCT's), while clinical experience is written off as "anecdotal." 

An anecdote is a report of a single, or at most a few, specific incidents, such as a patients' seeming to get better after taking a medication, along with the interpretation of the event by an allegedly biased observer. Besides the fact that one or two cases may not be representative of a psychiatric condition, and that other causes for the observed effect have not been ruled out, the description provided by the source of the anecdote may be incomplete or incorrect. And his or her conclusion may be logical, or it may be fallacious in any of a variety of different ways.

Widespread clinical experience — or the clinical experiences of a wide variety of practitioners who are known to do careful diagnostic work-ups and to follow patients closely — is somehow also written off as "anecdotal," but this is just nonsense. 

First of all, as I discussed in a previous post, in psychiatry there are almost no objective diagnostic blood tests or direct measurements of brain function that can tease out the difference between neuropathology and normal neural plasticity in response to environmental factors. Therefore, conclusions drawn in RCT's are based entirely on the self-report data from subjects or by the potentially biased observations of the experimenters. They are just as much anecdotal as widespread clinical experience in that sense. 

Actually, widespread clinical experience is better than most RCT's in determining the efficacy of drugs. It employs a sample size far larger than all the RCT's on a treatment put together. Also, there are several important  limitations with RCT's.

In a paper by G. Parker and S. McCraw (Acta Psychiatrica Scandinavia, 2015: 1-10) about the drug Lamictal (lamotragine), they discuss the disconnect between the reported efficacy of a new psychotropic medication as quantified in RCTs and its actual effectiveness as observed in the 'real world' of psychiatric practice:

"Most commonly any such disconnect is one of degree, with the medication being progressively clinically judged as less or more effective. Less commonly, the medication may be judged to have a different therapeutic 'signal' than for its formal indication. Two examples of the latter are the selective serotonin reuptake inhibitors (SSRI's) which seemingly modulate emotional dysregulation as much as they have antidepressant propensities, and some atypical antipsychotic drugs having utility in augmenting antidepressant medications and in having mood stabilizing propensities rather than being confined to the management of psychotic conditions.

Any such disconnect can reflect multiple factors, as considered now in relation to antidepressant medications. An antidepressant's 'efficacy' is evaluated from RCTs with the 'control' treatment being either a placebo or a comparator drug. In such trials, the duration is often limited to several weeks, the sample constrained by inclusion and exclusion criteria (e.g. non-suicidal, no substantive co-morbid conditions) that do not hold in clinical practice, and which may, depending on recruitment strategies, be weighted to those with potentially spontaneously remitting and/or less severe conditions. 

RCT-evaluated efficacy may he overestimated using a lower-than-usual dose of any comparator medication, or underestimated if the antidepressant is prescribed at what might be later determined to be a suboptimal dose. Biases may emerge if the sponsoring developers provide data only on trials generating superior findings. "

Richard Horton, editor of the medical journal The Lancet, recently observed, “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

In studies used to evaluate medications for treatment of various mood disorders, there is another glaring problem: the use of inadequate and incomplete diagnostic evaluations of the subjects. I have written several posts about the nonsensical expansion of the diagnosis of bipolar disorder, in which researchers lump together Bipolar I with the bogus disorder Bipolar II. 

Readers of this blog know my attitude about the latter: In my practice since I started training, I have seen four patients who actually met the DSM criteria for this supposed disorder when they were interviewed carefully and followed closely, and all four responded to lithium, suggesting that they were just milder cases of Bipolar I. And most patients who were given that diagnosis by previous psychiatrists did not have bipolar disorder at all but had the mood instability characteristic of certain personality disorders. Some studies have shown this to be true.

Worse yet, some studies include patients diagnosed with unspecified bipolar disorder and "bipolar spectrum disorder."  Neither is defined clearly nor is there any agreement on exactly with what criteria these diagnoses are to be made. So the RCTS's are basically comparing apples to oranges.

This particular problem is not discussed clearly in the Parker and McCraw article. In fact, they conclude that the anticonvulsant drug Lamictal (lamotrigine) is probably not effective in bipolar I disorder, but probably is effective in bipolar II disorder - which probably means it may help the affective instability of patients with personality disorders. This puts this drug as a possible alternative to SSRI's, which the authors themselves note "... seemingly modulate emotional dysregulation."  Emotional dysregulation and affect or mood instability are basically the same thing.

(Combining SSRI's and a long acting benzodiazepine such as clonazepam is even more effective in "raising the bar" on the strength of environmental stimuli required to set off an episode of affect dysregulation, as well as in decreasing self-injurious behaviors like cutting. There are no RCT's on this combination for these symptoms; the drug companies won't do them because most of these drugs are generic and the drug companies probably know that it is effective but do not want docs to know this. However, widespread clinical experience by those doctors who know the difference between bipolar disorder and borderline personality disorder backs me up on this).

The history of the FDA approval for the drug Lamictal for bipolar disorder is bizarre. Its manufacturer first touted it as a treatment for the depressive episodes in bipolar disorder, although later studies showed it to be ineffective in the acute phase of bipolar depression. Then, when it got the FDA approval for psychiatric use, rather than being given the indication for prophylaxis for (prevention of) episodes of bipolar depression, it was given a general indication for bipolar disorder as a whole. This, even though there was zero evidence that it prevented episodes of mania, or that it was useful in acute mania.

In fact, most of the studies in bipolar disorder with the drug were based on a rather flawed outcome measure. The drug was found to delay, but not to prevent, the emergence of another bipolar episode. Whether the episode was a depressive episode or a manic episode was not specified in the majority of these studies!  

Furthermore, lithium — the standard prophylactic treatment—when used at the dosages which lead to the correct blood levels of the drug in patients who respond to and can tolerate it (about 80% of bipolar I patients) completely prevents the re-emergence of manic episodes. In those cases, the measurement "time until the next manic episode" would essentially be the patient's entire lifetime. 

Delaying episodes of bipolar disorder is better than having them more frequently, but why would you use a drug to do that when there is another drug that can prevent them from happening completely?

Conventional Wisdom that Seems Obvious Once Again Found to be Actually True

As I did on my posts of November 30, 2011,  October 2, 2012, September 17, 2013, and June 3, 2014, it’s time once again to look over the highlights of the latest issue of one of my two favorite psychiatry journals, Duh! and No Sh*t, Sherlock. We'll take a look at the unsurprising findings published in the latest issue of No Sh*t Sherlock. My comments in bronze.

As I pointed out in those earlier posts, research dollars are very limited and therefore precious. Why waste good money trying to study new, cutting edge or controversial ideas that might turn out to be wrong, when we can study things that that are already known to be true but have yet to be "proven"? Such an approach increases the success rate of studies almost astronomically. And studies with positive results are far more likely to be published than those that come up negative.

 

5/28/14.  Physical activity program may reduce mobility disability in seniors.

USA Today (5/28, Painter) reports that for seniors, “losing the ability to walk a short distance often means losing independence.” Now, “researchers say they have found a treatment that, for some, can prevent that loss of mobility,” and that is “a moderate exercise program.” The Washington Post (5/28, Bahrampour) reports that the study, “called the Lifestyle Interventions and Independence for Elders and funded by the National Institute on Aging and the National Heart, Lung, and Blood Institute, was the first of its kind to test a specific regimen of regular physical activity for sedentary older people.” The Boston Globe (5/28, Kotz) “Daily Dose” blog reports that the study, published online May 27 in the Journal of the American Medical Association, “found that elderly people who walked and did basic strengthening exercises on a daily basis were less likely to become physically disabled compared to those who did not exercise regularly.” The study control group consisted of people who were instructed to take health education classes. 

I guess it's still OK for seniors to sit very, very still while posing as nude models for art students.

6/17/14. Study Shows Association Between Mental Illness Severity and Employment and Income.

More severe mental illness appears to be associated with lower employment rates in recent years, and people with serious mental illness are less likely than people with no, mild, or moderate mental illness to be employed after age 49, according to the report, “Employment Status of People With Mental Illness: National Survey Data From 2009 and 2010,” published in Psychiatric Services in Advance.

We now know for sure that employers are not always hot to hire people who are too mentally impaired to perform the work.

6/20/14. Brain Injuries Linked To Higher Risk For Headaches.

HealthDay (6/20) reports that research scheduled to be presented at the American Headache Society meeting suggests that “U.S. veterans of the Iraq and Afghanistan wars who suffered brain injuries are at a much higher risk for headaches, especially migraines.” This “study included 53 veterans who had suffered a traumatic brain injury during deployment and...53 veterans without brain injuries.” Investigators found “that all of the veterans in the brain injury group said they experienced headaches, compared with about 76 percent of those in the control group.” Eighty-nine percent of the headaches in those with brain injuries were migraines, while just 40 percent of the headaches in the control group were migraines.

Now just a minute. Bodily injuries produce pain?? Since when?

9/1/14. The relationship between premorbid body weight and weight at referral, at discharge and at 1-year follow-up in anorexia nervosa. 

European Child and Adolescent Psychiatry, 09/03/2014: Focker M, et al.  In this study, the relationship between pre-morbid body mass index (BMI) percentile and BMI at admission was solidly confirmed. In addition to pre-morbid BMI percentile, BMI at admission and age were significant predictors of BMI percentile at discharge. BMI percentile at discharge significantly predicted BMI percentile at 1–year follow–up. An additional analysis that merely included variables available upon referral revealed that premorbid BMI percentile predicts the 1–year follow–up BMI percentile.

Oh, I did not see it before, but I get it now. More severe disorders have a worse prognosis.

11/25/14. Talk Therapy May Prevent Suicide in High-Risk Patients

Talk therapy may decrease risk for future suicide attempts and completions in patients who have already made a previous attempt, new research suggests. 

God, I should hope so, or I'm in the wrong business!!

1/13/15. Self-injurers experience greater negative emotionality, particularly self-dissatisfaction, compared to individuals with no NSSI history.

Self-injurers also reported less positive emotion, but these effects were smaller. The pattern of results was similar when controlling for Axis I psychopathology and borderline personality disorder.

And here I thought cutters and burners did so because their joy was just soooo unbearable.

1/30/15. Repeated Blows To Head In Boxing, Martial Arts May Damage Brain.

HealthDay (1/30, Preidt) reports that research published in the British Journal of Sports Medicine “supports the notion that repeated blows to the head in boxing or the martial arts can damage the brain.” Investigators studied “93 boxers and 131 mixed martial arts experts,” as well as 22 individuals who had never suffered a head injury. “MRI brain scans and tests of memory, reaction time and other intellectual abilities showed that the fighters who had suffered repeated blows to the head had smaller brain volume and slower processing speeds, compared to non-fighters.”

So I guess I should quit beating my head against the wall trying to get researchers to actually look into things we actually do NOT already know.

1/30/15.  The US Food and Drug Administration (FDA) has approved lisdexamfetamine dimesylate (Vyvanse, Shire) to treat binge eating disorder (BED) in adults.

The drug is the first FDA-approved medication to treat this condition. "Binge eating can cause serious health problems and difficulties with work, home, and social life," said Mitchell Mathis, MD, director of the Division of Psychiatry Products in the FDA's Center for Drug Evaluation and Research. "The approval of Vyvanse provides physicians and patients with an effective option to help curb episodes of binge eating." The efficacy of Vyvanse in treating BED was shown in two clinical studies that included 724 adults with moderate to severe BED, as reported by Medscape Medical News. In the studies, participants taking Vyvanse experienced a decrease in the number of binge eating days per week and had fewer obsessive-compulsive binge eating behaviors compared with patients in a placebo group.

Shocking new finding: appetite suppressants reduce eating.

January 2015.  Alcohol, Depression potent risk factors for suicide.

BERLIN– Alcohol dependence and major depressive disorder are similarly potent yet independent risk factors for suicidal behavior, according to Dr. Philip Gorwood. Although alcohol use disorder and major depression are extremely common and often comorbid, the mechanisms by which they boost the risk for suicidal behavior are very different, he said at the annual congress of the European College of Neuropsychopharmacology.

Insert your own joke here. No prize will be awarded for best gag, but let's see what you got!

And yes, it is OK to joke even about suicide. Black humor often helps us all to squarely face up to very serious issues, and is therefore to be encouraged. 

Increasing Placebo Responses in Psychiatric Drug Studies

In 2013, authors Rutherford and Roose  [American Journal of Psychiatry, 170  (7),  723-733] wrote a paper that discussed the results of a previous study that had found that the placebo (inactive "sugar" pill) response rates in random clinical trials (RCT's) of antidepressant medication had risen at a rate of 7% per decade over the past 30 years. Consequently, the average difference between active medication and placebo observed in published antidepressant trials decreased from an average of 6 points on the Hamilton Rating Scale for Depression (HAM-D) in 1982 to only 3 points in 2008.

Now the lead author of that paper and his colleagues have found something similar going on in RCT's between 1960 and 2013 of anti-psychotic medications for schizophrenia (the findings were published on line in October in JAMA Psychiatry). Most interestingly, in the 1960's, patients who received the placebo in such studies actually got worse on them.  By the 2000's, however, they were getting better on placebo.

Even more striking, the average RCT participant receiving an effective dose of medication in the 1960s improved by 13.8 points on the Brief Psychiatric Rating Scale (BPRS), whereas this difference diminished to 9.7 BPRS points by the 2000's.

What the heck is going on here? Are the medicines somehow becoming less effective than they used to be?

In their article from 2013, Rutherford et. al. try to explain this by looking at such things as expectancy (what do subjects think is going to happen with their symptoms), a statistical phenomenon known as  regression to the mean (see this post for a definition), the amount of contact subjects have with the study doctors, the social desirability of certain responses, and the “Hawthorne Effect” (subjects in an experiment improve or modify the aspect of their behavior under study simply by virtue of knowing that the behavior is being measured).

While the expectations of the average John Q. Citizen that antidepressants will work may have increased somewhat over the decades because of such things as celebrities describing their experiences with depression or commercials for Cymbalta and Abilify, there has also been a lot of negative information on that same score on television shows like Sixty Minutes and from the anti-psychiatry rants of Scientologists and others.

Whether these two influences completely cancel each other out is debatable, but I think it is safe to say that many of these possible reasons for a change in placebo response rate advanced by authors have in fact not changed significantly since the 1960's. In fact, if people in the 60's didn't think antidepressants would work, expectancies would have been lower, not only in the placebo group, but in the active treatment group as well.

If certain factors that affect placebo response rates have not changed a lot, then those factors can not explain the rise in the placebo response rates

The authors also mention a couple of factors that I believe to be more on the mark: first, that assessments of eligibility for a clinical trial may be biased toward inflated symptom reporting at the beginning of the study - when investigators have a financial incentive to recruit patients - and second, that most research participants in the 1960s and 1970s were recruited from inpatient psychiatric units, whereas current participants are symptomatic volunteers responding to advertisements.

The biggest change in research with RCT's over the period in question is that many studies are no longer done in medical schools, but by private entities called Contract Research Organizations (CRO's). The doctors who run the studies are paid for each subject they recruit, and subjects only get paid if they are recruited. This means that there are not just one set of financial incentives for everyone to exaggerate their symptoms at the beginning of a study, but two! This tendency will lead to a higher placebo response rate because after they are recruited, subjects no longer have an incentive to exaggerate their symptoms. So they seem to get better.

It is very easy to bias a research diagnostic interview. I'll get to that in a minute, but first a digression.

I was fortunate to train at a time when patients could be kept in the hospital for several months if necessary, so we got to see the patients in depth over a considerable time period, and could watch medication responses. People who have trained more recently do not see this any more.  Antidepressant responses clearly took a minimum of 2 weeks  - and then only if the patient responded to the very first drug given at the first dose given.

Because most patients do not understand this, the doctor can usually discriminate a placebo response from a true response by observing when the patient starts to get better combined with the rate at which they improve. Since subjects don't know what to expect, being on this timeline could not be due to the expectancy factor, which in turn is necessary for a having a good placebo response.

I can tell you that a severe, properly diagnosed melancholic depression almost never showed a significant placebo response.  The placebo response rate was probably about the same as the placebo response rate to a general anesthetic.

Another thing we observed was that patients with an acute schizophrenic reaction did not seem to get any better at all with such things as additional contact with doctors, which might be expected if a placebo response were taking place. In fact, the more you spoke with them, the more likely it would be that you would hear evidence that patients had a significant thought disorder than if you just had a briefer, casual conversation with them.

A thought disorder is at least as important as delusions and hallucinations in showing that someone is, in fact, someone with schizophrenia. People with a thought disorder see relationships between things that are completely illogical (loose associations). For example, the first patient I ever saw with schizophrenia in medical school believed that everyone who wore oxblood-colored shoes was a descendant of George Washington.  

Huh?

Anyway, back to the question of biasing diagnostic exams. This is particularly easy when diagnosing a clinical depression. It is important to distinguish them from those people who are merely chronically unhappy.  People with a clinical major depression, especially with so-called melancholic features, are a very different breed of cat.  

The  symptoms of both disorders do overlap a bit, so there are some cases in which it is really hard to tell one from the other. However, in the majority of cases it is a fairly easy call. It is, provided you do a complete psychiatric assessment, over several days, to see if a symptom of depression meets the requirement known as the Three P's: 

The symptoms need to be pervasive (they do not go away depending on what the patient is doing at a particular time), persistent (lasting almost all day every day for at least two weeks), and pathological (the patients symptoms and functioning differ to a highly significant degree from the patient's usual state). In addition to the three p's, all of the patient's symptoms have to always occur simultaneously.  

These types of characteristics do not usually show up on the type of symptom checklists used to assess patients in clinical trials, because the checklists are mostly based on a patient's self report.  Unfortunately, the majority of people do not know the difference between a clinically significant symptom and one that is not. The rate of false positive responses on checklists is staggering.

Many studies instead use something that is called a semi-structured diagnostic interview (such as one called a SCID) to make a diagnosis. It is called semi-structured because it tells the examiner to ask certain questions exactly as they are posed verbatim. However, the examiner is then free to ask any follow-up questions needed to clarify the clinical significance of any symptom the patient reports.

If you want to diagnose major depression regardless of whether or not the patient actually has it, all you have to do is accept every "yes" answer a patient gives to a question about a symptom without any follow-up questions to see if the symptom is characterized by the three P's. If the patient answers "No" to a question, however, you keep pumping the patient for additional clarification until you can find something the patient says that will justify changing the "no" answer to a "yes."  Voila.

Book Review: How Not to Be Wrong by Jordan Ellenberg

A while back (11/2/11) I reviewed the book Stats.con by James Penston. That book discussed how the statistics used in randomized clinical trials can be highly deceptive. How Not to be Wrong also covers some aspects of statistical misuse, in more detail, and certainly in a much more entertaining way. Some of his comments are funny as hell. 

Jordan Ellenberg

Consider the widespread use of a statistic call the p value, which estimates the probability that the result of a study could have just been a chance coincidence rather than an actual meaningful finding. A study is generally considered positive if the p value is 5% or less.

5% is of course not 0%. There is a one in twenty probability that the study results that are deemed positive were in fact negative. But what happens if journals only publish the positive studies and not the negative ones, when there might be a large number of negative studies, and when the positive study results are not reproduced (replicated) in a second study? Well, people start believing things that are not true, that's what.

The reason for this is because, as the author points out, improbable things actually happen quite frequently. Especially if you do lots and lots of things – like experiments. 

Another issue he mentions is that, if your sample size is too small, the chances increase dramatically that one of your subjects will be an outlier that dramatically but artificially changes the average for whatever characteristic you are measuring. With a small sample, you are more likely to get a few extra prodigies or slackers in a study of people's ability to perform certain tasks. A famous example: if Bill Gates walks into a bar with a few other people, the average guy in the room is a billionaire.  

Here’s how the author starts out a discussion of the p value problem (pages 145-46) :

"Imagine yourself a haruspex; that is, your profession is to make predictions about future events by sacrificing sheep and then examining the features of their entrails...You do not, of course, consider your predictions to be reliable merely because you follow the practices commanded by the Etruscan deities. That would be ridiculous. You require evidence. And so you and your colleagues submit all your work to the peer-reviewed International Journal of Haruspicy, which demands without exception that all published results clear the bar of statistical significance.    

      

Haruspicy, especially rigorous evidence-based haruspicy, is not an easy gig. For one thing, you spend a lot of your time spattered with blood and bile. For another, a lot of your experiments don't work. You try to use sheep guts to predict the price of Apple stock, and you fail; you try to model Democratic vote share among Hispanics, and you fail…The gods are very picky and it's not always clear precisely which arrangement of the internal organs and which precise incantations will reliably unlock the future. Sometimes different haruspices run the same experiment and it works for one but not the other — who knows why? It's frustrating…

      

But it's all worth it for those moments of discovery, where everything works, and you find that the texture and protrusions of the liver really do predict the severity of the following year's flu season, and, with a silent thank-you to the gods, you publish

      

You might find this happens about one time in twenty.

      

That's what I'd expect, anyway. Because I, unlike you, don't believe in haruspicy. I think the sheep's guts don't know anything about the flu data, and when they match up it's just luck. In other words, in every matter concerning divination from entrails, I'm a proponent of the null hypothesis [that there is no connection between the sheep entrails and the future]. So in my world, it's pretty unlikely that any given haruspectic experiment will succeed.

      

How unlikely? The standard threshold for statistical significance, and thus for publication in IJoH, is fixed by convention to be a p-value of .05, or 1 in 20... If the null hypothesis is always true — that is, if haruspicy is undiluted hocus-pocus —then only one in twenty experiments will be publishable.

      

And yet there are hundreds of haruspices, and thousands of ripped-open sheep, and even one in twenty divinations provides plenty of material to fill each issue of the journal with novel results, demonstrating the efficacy of the methods and the wisdom of the gods. A protocol that worked in one case and gets published usually fails when another harupex tries it, but experiments without statistically significant results do not get published, so no one ever finds out about the failure to replicate. And even if word starts getting around, there are always small differences the experts can point to that explain why the follow-up study didn't succeed."

The book covers many subjects about which the non-mathematically-inclined can learn to think in a mathematical way in order to avoid coming to certain wrong conclusions and to zero in on correct ones. Many of these, however, are irrelevant to this blog – the chapters on lotteries come to mind. I of course found those parts a bit less interesting. But the chapters relevant to medical studies are so right on.

Another important topic the author covers is known mathematically as regression to the mean. This phenomenon can lead, as examples, to overestimates about the genetic component of human traits and explains why fad diets always seem to work at first but then later on everyone seems to forget about them. As mentioned, when you average any measurement applied to human beings, the averages can be deceptive.  

In addition to the sample size considerations described above, you can get into trouble if you start with a sample that contains people who are higher or larger on average on the relevant variable that the average person in the general population.

If two tall people marry, their progeny will usually be, on average, tall compared to others in the general population. However, they are not all that likely to be taller than their parents. As Ellenberg states, “…the children of a great composer, or scientist, or political leader, often excel in the same field, but seldom so much as their illustrious parents” (p. 301). Their heredity mingles with chance environmental considerations, and pushes them back toward the population average. That is the meaning of regression to the mean.

To understand this, think about those who embark on weight loss diets. One needs to consider the fact that most people’s weight tends to fluctuate a few pounds either way depending on a lot of chance factors, such as their happening by an ice cream truck. And when are people most likely to start a diet? When their weight is at the top of their range! So by the law of averages, they are probably in many instances going to lose weight whether they diet or not. But when they do diet, guess what happens? They attribute the loss to the fantastic new diet!

I can not say for certain, but I wonder if studies on borderline personality disorders (BPD) yield misleading results because of regression to the mean. Long term follow-up studies on patients with the disorder seem to indicate that it seems to go away after a few years in a significant percentage of subjects. This finding is misleading, however, when you look closer. 

To make the BPD diagnosis, the subject needs to exhibit 5 of the 9 possible criteria. Many of the "improved" subjects merely went from 5 criteria down to 4 of them, and were therefore not diagnosed with BPD any longer. Actually, they became just what we call "subthreshold" for the disorder. Their problematic relationships, however, were still pretty much the same.

These results could mean that subjects with BPD may naturally vacillate between meeting criteria for the disorder and being subthreshold, or between exhibiting a high number of the criteria and a lower one. Which would mean that if they qualified for the diagnosis at the beginning of the long term follow-up study, a significant proportion of the long-term study subjects were at their worst. If so, the study results may indicate regression to the mean, and therefore say nothing else significant about the long term prognosis for the disorder.

Other important statistical issues the author discusses clearly and brilliantly include assumptions that two variables are related in a linear fashion when the are not (non-linearity - cause and effect relationships that are not based purely on an increase in one variable always leading to either an increase or decrease in another); torturing the data until it confesses (running multiple tests on your study data, controlling for different things, until something significant seems to pop up); and the following problem inherent in studies designed to see if two things like being married and smoking are correlated: 

"Surely the chance is very small that the proportion of married people is exactly the same as the proportion of smokers in the whole population. So, absent a crazy coincidence, marriage and smoking will be correlated, either positively or negatively."

Any one who is serious about critically evaluating the medical literature owes it to themselves to read this book.

Researchers Aren't Wasting Time Looking for Cures for Alzheimer's Disease or Schizophrenia

As I did on my posts of November 30, 2011,  October 2, 2012, and September 17, 2013, it’s once again time to look over the highlights of the latest issue of one of my two favorite medical journals, Duh! and No Sh*t, Sherlock. Let’s take a look at the unsurprising findings published in the latest issue of Duh! My comments in bronze.

As I pointed out in those earlier posts, research dollars are very limited and therefore precious. Why waste good money trying to study new, cutting edge or controversial ideas that might turn out to be wrong, when we can study things that that are already known to be true but have yet to be "proven"? Such an approach increases the success rate of studies almost astronomically. And studies with positive results are far more likely to be published than those that come up negative.

9/13/13. Effects of Child Abuse Can Carry Over, Study Finds.

Researchers with the National Academy of Sciences reported Thursday that the damaging consequences of abuse can not only reshape a child’s brain, but can last a lifetime. Untreated, the effects of child abuse and neglect, the researchers found, can profoundly influence a child’s physical and mental health, their ability to control emotions and impulses, their achievement in school, and the relationships they form as children and as adults.

Cognitive behavioral therapists are all up in arms in reaction to this, thoroughly annoyed that the psychoanalysts were right about some things.

9/16/13.  Teens Who Text About Fighting, Drug Use More Likely To Engage In Those Behaviors.

HealthDay (9/14, Preidt) reported that research published in the Journal of Abnormal Child Psychology suggests that “teens who text about bad behaviors such as drug use or fighting are more likely to actually engage in those behaviors.” Researchers examined the text messages of more than 170 ninth-graders. Their behaviors were rated by their teachers, parents, and fellow students. The investigators “found a strong link between antisocial text messages and higher ratings of antisocial and aggressive behavior at the end of the school year.”

If they were real sociopaths, they wouldn’t have to brag about it.

 

9/27/13. Common Pain Relievers May Reduce Depression In Individuals With Osteoarthritis.

Reuters (9/27, Doyle) reports that research published in the American Journal of Medicine suggests that common pain relievers may reduce both pain and depression among individuals with osteoarthritis. Investigators came to this conclusion after looking at data from five trials that included approximately 1,500 patients.

Pain causes people unhappiness??  I always thought pain was something that causes unremitting happiness and celebration.

10/21/13. Stalking May Cause Psychological Distress.

HealthDay (10/19, Dallas) reported that, according to a study published online in the journal Social Science Quarterly, “women who are the victims of stalkers are up to three times more likely than their peers to experience psychological distress.” Researchers arrived at this conclusion after examining data “compiled on over 8,100 women from three major surveys.”

And here I thought stalkers were spreading joy wherever they went.

2/27/14.  Suicide Attempts Early in Life Signal Long-Term Social, Health Problems, Study Finds

Young people who attempt suicide are not only more likely to have persistent psychiatric problems as they approach midlife than non-attempters, but they are also more likely to have physical health problems, engage in violence, and need more social supports as they age. These are key findings from a study by led by Sidra Goldman-Mellor, Ph.D., and colleagues at Duke University and several other institutions and reported in JAMA Psychiatry.

The best predictor of future behavior is past behavior?  Who knew?

2/27/14. Study: Stigma Associated With Mental Illness May Prevent Many People From Seeking Care.

HealthDay (2/27) reports that research published in Psychological Medicine suggests that “the stigma often associated with mental illness prevents many people from getting the care they need.” Investigators looked at data from 144 studies that included a total of approximately 90,000 people. The researchers found that “stigma ranked as the fourth highest of 10 barriers to care.” The investigators also found that, “aside from the stigma of using mental health services or being treated for mental illness, the participants also reported feelings of shame and embarrassment as reasons for not seeking care.”

Caring about what other people think?  Worrying about your reputation?  Who does THAT?

3/17/14. Stress May Impact Kid's Health, Well- Being  

HealthDay (3/15, Preidt) reported that according to research presented at the American Psychosomatic Society’s annual meeting, “stressful events can have an almost immediate impact on children’s health and well-being.” After analyzing data on some 96,000 US children, researchers also found that youngsters “who experienced three or more stressful events were six times more likely to have physical or mental health problems or a learning disorder than those who had no stressful experiences.”

Nonsense.  Learning how to react to constant threats to your well being builds character!

4/8/14. Study: Physician appointment availability greater with private insurance than Medicaid.

Reuters (4/8, Seaman) reports on a new study, published in the current edition of JAMA Internal Medicine, which shows the availability of physicians varies depending on a patient’s insurance coverage. Researchers, from the Perelman School of Medicine at the University of Pennsylvania in Philadelphia, found they were able to book appointments 85% of the time when claiming private insurance, compared to just 58% when they claimed to be covered by Medicaid.

Oh come on. Doctors absolutely hate to make money.

4/14/14. Paternal Alcoholism Tied To Family Conflict.

Reuters (4/11, Bond) reported that according to a study published online March 15 in the journal Addictive Behaviors, families in which the father had a problem with alcohol appeared to experience increased levels of conflict. However, treating men for alcoholism may result in an improved home life for their children.

Gee, and I thought drug addiction was a symptom of family harmony.

4/22/14.  False-Positive Mammograms Linked To Increased, But Temporary, Anxiety.

The Los Angeles Times (4/22, Kaplan) “Science Now” blog reports that in a study published in JAMA Internal Medicine, investigators “examined data from a large clinical trial of digital mammography and concluded that false-positives produced a ‘significant increase in anxiety,’ though it was only temporary.”

       

People get nervous if they think they might die. Really?

5/1/14. Effects of Recurrent Violence on Post-traumatic Stress Disorder and Severe Distress in Conflict-affected Timor-Leste: a 6-year longitudinal study

Silove D, et al. – Recurrent violence resulted in a major increase in post–traumatic stress disorder and severe distress in a community previously exposed to mass conflict. Poverty, ongoing community tensions, and persisting feelings of injustice contributed to mental disorders. The findings underscore the importance of preventing recurrent violence, alleviating poverty, and addressing injustices in countries emerging from conflict.

So what does trauma have to do with PTSD anyway?