Were They Awarded Tenure for These Studies?

It’s once again time to discuss some brand new psychiatric studies that would be just perfect for my two favorite journals, “Duh!” and “No Sh*t, Sherlock.”

As we all know, our collective experiences are nothing more than anecdotal evidence for anything, and what appears obvious to almost everyone cannot be considered true unless subjected to a randomized controlled or epidemiological study. 

Research dollars are very limited and therefore precious.  Why waste good money trying to study new, cutting edge or controversial ideas that might turn out to be wrong, when we can study things that that are already thought to be true but have yet to be "proven"?  Such an approach increases the success rate of studies almost astronomically.

And studies with positive results are always far more likely to get published than negative ones, so why should an academic take that risk?

Here are some of my favorite recent headlines reported by psychiatric news gathering organizations:

“Body dissatisfaction appears to be the major factor propelling young people on the road to eating disorders.”  Really? I though most people do not complain about being too fat or too thin.  I mean, especially women.

“Sleep disorders are prevalent with mental illnesses.”  And here I was under the impression that depressed, anxious, and paranoid patients slept more soundly than anyone.

“Youngsters with depressed fathers are more likely than other kids to have emotional and behavioral problems, according to a new study of more than 20,000 U.S. families.”  This is so good to know.  I had no idea that having unhappy, miserable adult family members might affect a child’s mood.

"A new analysis released by the Kaiser Family Foundation shows that tough economic times have led to a downturn in doctor visits."  That can’t be true in the United States, where as we all know, everyone gets free healthcare.

"Receiving a diagnosis of dementia increases a person's risk for suicide, particularly if symptoms of depression and anxiety are present," according to a studyresearch published in the November issue of the journal Alzheimer's & Dementia.”  Now come on!  The prospect of becoming senile and a financial and emotional burden on one’s family might cause an already depressed or anxious person to despair?  No way!

“Long-Term US Unemployment Taking Psychological Toll." Now this is really surprising, since we all know money cannot buy happiness.  I was just positive that not having enough food to eat and a roof over your head would hardly matter.

“A history of maltreatment during childhood increases the risk for depression in adulthood and poor treatment outcomes, new research suggests.” Another amazing discovery.  This had never been noticed by either psychiatrists or psychotherapists before now.

And finally, "Violence against women is significantly associated with mood, anxiety, and substance use disorders throughout the victim's lifetime."  Since we already know these psychiatric disorders all have purely genetic causes, we now know that these very same genes also cause women to get beaten up.

Ve Have Vays of Making You Talk, Part V: Non Sequiturs

In Part I of this post, I discussed why family members hate to discuss their chronic repetitive ongoing interpersonal difficulties with each other (metacommunication), and the problems that usually ensue whenever they try.

I discussed the most common avoidance strategy - merely changing the subject (#1) - and suggested effective countermoves to keep a constructive conversation on track. In Part II, I discussed strategies #2 and #3, nitpicking and accusations of overgeneralizing respectively. In Part III, I discussed strategy #4, blame shifting. In Part IV, strategy #5, fatalism.

This post is the first in a series about strategy #6, the use of irrational arguments.  It will be subdivided into several posts because in order to counter irrational arguments, one first has to recognize them.  I will hold off describing strategies to counter irrational arguments until after I have describe some of the most common types.

To review once again, the goal of metacommunication is effective and empathic problem solving. In this post, I will discuss an avoidance strategy called fatalism, and describe appropriate counter-strategies to get past it.

As with all counter-strategies, maintaining empathy for the Other and persistence are key.

Irrational arguments are used in metacommunication to confuse other people so that they either become confused about, or unsure of the validity of, any point they are trying to make.  They are also frequently used to avoid divulging an individual's real motives for taking or haven taken certain actions. 

In my first book, A Family Systems Approach to Individual Psychotherapy, I attempted to educate therapists about how their patients will use irrational arguments to confuse them.  I referred to irrational arguments as mental gymnastics. They are particularly effective in emotionally-charged exchanges. 

One can learn to recognize irrational arguments meant to throw someone off the track by learning about the logical fallacies that are taught in debate clubs and philosophy classes.  Today's subject: the non sequitur.  This is a Latin phrase meaning "it does not follow."

A non sequitur occurs when a conclusion is drawn deduc­tively that does not follow logically from the preceding proposi­tions. Someone will take a fact or make a generalization or a categorization, assert than some other fact or generalization is an example of this, and then draw a conclusion.   This process is called deductive reasoning.  

Correct deductive reasoning can best be demonstrated using syllogisms. Let us look at perhaps the most famous of all syllogisms:

                        All men are mortal.

                        Socrates is a man.

                        Therefore, Socrates is mortal.

The first thing one should understand, if one is to correctly evaluate deductive reasoning, is that this is a valid deduction whether or not the initial statements "All men are mortal" and "Socrates is a man" are correct. A deduction - the last statement in the syllogism - can be judged to be valid if the structure of the syllogism is correct. This means that a conclusion can be com­pletely wrong but the deduction can still be valid logically if the conclusion follows correctly from the initial propositions.

The truth of a proposition, as opposed to its logical validity, de­pends upon the truth of the information from which the propo­sition is deduced. In a valid syllogism, if the first two statements are true, the conclusion must be true. The presence in the syllogism of the word all is extremely important. If some men are mor­tal and some are not, Socrates might fall outside the set of "things that are mortal," and the syllogism would become invalid. This is precisely why the equally famous fallacious syllogism

                        The Virgin Mary was a virgin.

                        My name is Mary.

                        Therefore, I am a virgin.

is invalid. The set of "people named Mary" falls both inside and outside of the set of "virgins." Therefore, Marys may or may not be virgins, and the conclusion is thus invalid. In this case it is not true that if the first two statements are true, the conclusion must be true, as would be the case with a valid syllogism.

A word here regarding inductive reasoning is in order. Inductive, as opposed to deductive, reasoning attempts to go in the reverse order. One attempts to make a generalization by examining several phenomena that seem to have something in common. One then makes the leap of faith that because all observed instances of the phenomena have this characteristic in common, therefore all instances of the phenomena, now or in the future, observed and not observed, share the characteristic.

For example, every ­time an object of whatever size or shape is dropped on earth, it falls down. The inductive conclusion is that the set of "things that fall down" entirely subsumes the set of "things that can he dropped" and that anything droppable will fall down if dropped.. One makes the prediction that any new object that can be dropped will head earthward if one picks it up and lets it go.

Now the deduction "since all observed instances of a cer­tain phenomenon behave a certain way or have certain things in common, therefore all future instances of the same phenomenon will continue to behave in the same way and have the same things in common" is in all instances a non sequitur. One might come upon an exception to the rule at any time. In other words, all inductive conclusions are invalid!

Nonetheless, induc­tive conclusions are not necessarily unreasonable and are fre­quently correct. I have in my hand a pencil, which I plan to hold up and then let go. Will it fall? I predict, on the basis of induc­tive reasoning, that it will. Let’s see. Well, I'll be. It did it again!

The reasonableness of an inductive conclusion is evalu­ated not by logic but by whether enough instances of the phe­nomenon have been observed to make a generalization possible and by whether there are any instances that contradict the gen­eralization. Deductive reasoning, or reasoning based on proof, would not be possible without inductive reasoning. It would be impossible to conclude that Socrates was mortal if one could not make the generalization "All men are mortal."

The deter­mination of how many instances are required to decide whether an inductive conclusion is reasonable is a very subjective matter, because no matter how many instances there are, the next one could always be the exception. For this reason, anyone look­ing for mental gymnastics when someone else makes an inductive conclusion best asks the questions: Are there significant excep­tions to the generalization that the first person is is making? If so, are they obvious, if only the person would look for them?

I will now describe a case in which a patient presented a therapist with a goodly number of non sequiturs. She was a single woman who came in complaining about being subjected to severely and significant repetitive sexual harassment by a co-worker.  The question the therapist posed was why she had made an appointment to see a psychiatrist in the first place, since she did not seem to have any evident psychiatric problem.  

As it later turned out, she did not really want to give the real answer because she was  protecting her mother, from both the judgment of the therapist and from her own rage.  So she gave spurious and very sublty non-rational reasons in order to throw the therapist off the track.

Her chief complaint was that she was upset - but only because someone was doing something to her about which almost anyone would be upset. People usually see a therapist because they believe there is something wrong with them or with their reactions to things.  That did not seem to be the case with this woman. Why wasn’t she talking to her boss or a lawyer, the therapist wondered?  Actually, she was in the process of doing both! 

The therapist could not seem to get a satisfying answer from her to the central question, and knew for certain that something else was going on with her when the non sequiturs began.

She first stated that she must have done something to make the co-worker behave in this extreme fashion, because people do not hate you unless you've done something bad to them. This was a non sequitur because she had no proof that her tormenter’s behavior was based on hatred of her, and even if it were, she was quite aware that the causes of hate in the world include a great many other things. Hate can be based on prejudice, jealousy, a chip on the shoulder, or any number of things other than what someone has done to the per­son who hates.

The next bit of curious logic occurred when the patient told the therapist that she just could not seem to make him understand that his gossip was disturbing her. How she could possibly have thought that he was not aware of that was simply beyond comprehension. The therapist told her he thought it kind of her to wish to give him the benefit of the doubt after all he had done to her, but it seemed that she had too much evidence to the contrary to support this thesis.

The first clue about the real reason she thought she needed a psychiatrist came when the patient is­sued yet another non sequitur. She began to get upset with herself for being disturbed by some of the slurs verbalized by her nemesis. She stated that he was just calling her names, after all. Why should just names bother her? Sticks and stones, and all that. She should be able to ignore it, like water off a duck's back. The therapist told her she would be quite an unusual person if she had not found the barrage of insults disturbing.

When faced with a recommendation that we explore why she was so upset that the whole situation bothered her, she balked. She said that if she were to get to the reasons be­hind this seemingly self-defeating behavior, she might find something terrible. Well, she might, but how terrible could it be? How did she know that she would not discover something wonderful? The odds were, of course, that she might find something uncomfortable, but as Albert Ellis (the founder of cognitive psychotherapy) says, feeling that one would not be able to tolerate the discomfort is irrational, espe­cially when the level of discomfort is already so high, and when bearing some additional discomfort could reduce it over the long run.

She correctly guessed that certain information she had re­lated to the therapist might incline him to think that her parents' divorce when she was a pre-teen had something to do with her current reactions. She then added a non sequitur that later turned out to be the essential clue to what was really going on. She said that she was puzzled by why everyone seemed to think that a parental divorce had so traumatized her and added that she had taken the divorce in stride. It was done, and there was no reason to get upset about it. The event just did not bother her.

When she made this statement, she was trying to take what may sound like a rational position. After all, the belief that one cannot stand an unpleasant occurrence makes one suffer more than necessary. The patient was saying more than this, however. She was saying that one should not be un­happy about a traumatic experience. No disappointment. No regret. No anything. Ellis states that regret and disappointment are emotional re­sponses that make sense. She was stating that she was unaffected entirely. Furthermore, the available evidence strongly suggested that she was more than just disappointed about what happened. She could hard­ly discuss the matter without breaking into tears.

Much later, the therapist discovered that what the non sequitur really alluded to was the patient's very rational concern about seeming to be bothered about anything. As it turned out, the patient's mother had been subjecting her for years to frequent guilt-ridden harangues about how bad the mother felt about the divorce. The mother would literally badger her with questions about whether she had been upset and traumatized by it. If the patient appeared distressed about anything, her mother would begin drinking herself into a stupor.

No wonder the patient tried to project an image of not being bothered by things! If she were to admit to being the least bit upset, the mother would feel even guiltier. The mother was already self-destruc­tive; perhaps she would become actively suicidal.

Now that seemed like a valid deduction.

The Last Time Biological Psychiatry Over-Reached

There once was another time in recent history when purely genetic explanations for complex human behavior were in vogue just as they are today. You’ve no doubt heard about how two identical twins raised apart were both alcoholics, preferred the color red, and were married to women named Flo.

In reality, most human behavior is learned. For God’s sake, we don’t even instinctually know how to have sex - unless someone tells us or we figure it out by trial and error. (Just the urge is instinctual). Luckily, most of us eventually figure it out.

The following is an excerpt from a chapter called The Brainlessness-Mindlessness Pedulum from my book, How Dysfunctional Families Spur Mental Disorders:

Eugenics

The biological underpinnings of many mental phenomena clearly have their origin in genetics.  Although they are hardly the only determinants of brain functioning, our genes set the parameters by which the structure and abilities of the human brain develop and change over the lifespan.  The subtleties of how the brain functions and what behavioral attributes have genetic components are only now beginning to become clear, but despite the lack of knowledge in earlier times, an interest in the inheritance of mental characteristics was certainly understandable.  

In the 1880’s, a cousin of Charles Darwin named Francis Galton began to think about the relationship between Mendelian genetics and the theory of natural selection in evolution.  The idea that the forces of nature seem to favor the strongest and most adaptive creatures led him to formulate a social philosophy that he called eugenics.  He believed that the human race could be improved through the selection by society of which individuals would be allowed or not be allowed to have children, based on what he believed to be their biologically inherited characteristics. 

The list of presumed inherited characteristics was, even by the loose standards of some of today’s “biological” psychiatrists, absurdly broad. Characteristics thought by many of the followers of eugenics to be genetically transmitted included such traits as sexual promiscuity and even poverty.

Eugenics quickly found many prominent believers, particularly in Germany and in the United States.  Among them were Luther Burbank, Alexander Graham Bell, feminist icon Margaret Sanger, the Carnegie Institute, and the Ford and Rockefeller Foundations.  The philosophy gradually expanded from an emphasis on selective breeding or positive eugenics to the idea that “inferior” members of our species should be forcibly sterilized so that they would never be able to pass down their supposedly bad characteristics.  This was termed negative eugenics.  Some people who believed in the idea that forced sterilization was a moral endeavor eventually jumped to the idea that inferior peoples should be exterminated.

In the United States, the influx of large numbers of European immigrants led to fears that such people might be of inferior stock, and might therefore “pollute” or “contaminate” the gene pool.  Eugenics gave voice and legitimacy to these fears, so it was appealing to a large segment of the American population. In 1910, a man named Harry H. Laughlin established an organization called the Eugenics Record Office (ERO), through which he lobbied politicians to help protect the purity of the human race through restrictions on immigration of peoples from Southern and Eastern Europe.  The peoples from these regions were thought to have “excessive insanity.”   The efforts of the organization led to the passage of the 1924 Johnson-Reed immigration bill which successfully limited the immigration of people from these areas, and completely excluded Asians from entering the States.

Harry Laughlin

The ERO also advocated forced sterilizations of certain segments of society.  It was supported financially by the Carnegie Institute, among others. The idea of forced sterilization of the mentally retarded had already gained acceptance by the time of the founding of the ERO, with the first state law requiring it having been passed in Indiana in 1907.   Eventually, thirty states passed similar laws, resulting in the forced sterilization of over 60,000 Americans.  The practice did not completely stop until approximately 1963.

Laughton was unhappy with the earliest versions of state laws mandating this practice and with their lax enforcement.  He also felt that forced sterilizations should be expanded from just for the “feebleminded” to include the insane, criminals, epileptics, alcoholics, and even the deaf and blind.  He apparently believed all of these characteristics were inherited through genetic mechanisms and that any chance of their being passed on to children had to be eliminated.  He drafted a model law in 1922 that became a template for some later state laws.

He was also influential in a case that came to the United States Supreme Court in which the constitutionality of the forced sterilization of the mentally retarded was upheld: the case of Buck versus Buck in 1927.  Carrie Buck was a woman who was branded as being mentally retarded after she became pregnant following a rape by the nephew of her foster parents.  She was very likely of normal intelligence, as was her daughter Vivian.  Nonetheless, no less a figure than Justice Oliver Wendell Holmes led the way in ruling in favor of the State of Virginia in the case, writing, “Three generations of imbeciles are enough.”

Carrie and Emma Buck

Adolph Hitler and his henchman found this ruling by an American court inspiring.  They loosely used Laughlin’s model law in drafting Germany’s own “Law for the Prevention of Genetically Diseased Offspring,” which went into effect in 1934.  In 1936, Laughlin was granted an honorary degree from the University of Heidelberg in Germany for his work on behalf of “racial cleansing.” 

In a sublime irony, Laughlin himself developed epilepsy in his later years.  Sufferers of this disorder were one of the groups of people he thought should be eliminated from the planet.

The mentally retarded, followed in quick succession by the mentally ill, were among the first victims of the Nazi death machine.  Forced sterilizations began in 1935, followed by the T-4 program for “euthanasia” of the mentally ill in 1939.  One of the architects of this death program was a psychiatrist, Ernst Rudin, as were several of the doctors directly involved in it.  The methods he helped devise for killing individuals with mental problems were later adapted for use in the large scale attempted extermination of those ethnic groups that the Nazis considered genetically inferior, such as the Jews and the Gypsies, as well as of certain individuals within their own ethnic group such as homosexuals.

Ernst Rudin

In the early days of the T-4 program, even small children were not spared.  At one point some families of children with mental problems, who were being told that their offspring had died peacefully of natural causes, became suspicious because they learned that so many of their children seemed to have all died on the same days.  In order to keep the program secret, the Nazis stopped killing the children directly in favor of just letting them starve to death so they would all die on different days.

Meanwhile, back in the United States, support for eugenics waned by the end of the 1930’s because of its association with the Nazis and also because the so-called science behind it was proving to be quite poor.  The Carnegie Institute withdrew its funding of the ERO in 1935 and it soon folded.  Some psychiatrists in the United States, however, apparently did not get the message. 

A psychiatrist named Foster Kennedy gave an address to the American Psychiatric Association’s annual meeting in 1941.  In it, he strongly advocated not only for the forcible sterilization of the mentally retarded, but for killing them, especially if they fell below a certain functional level.  Because he assumed that such individuals were in constant suffering and would be better off dead, he referred to this killing as euthanasia or mercy killing.  His address was published in the Journal of the American Psychiatric Association in July of 1942.  In the same issue an opposing viewpoint by another psychiatrist, Leo Kanner, was also published, along with an editorial.

Leo Kanner

While Kanner had no objection to sterilization, he did object to euthanasia.  He also questioned the validity of assuming that people of low IQ would necessarily beget children who were also mentally deficient, but did not spend any time exploring the ramifications that would ensue for his philosophy if this were indeed the case.  He believed that sterilization should be reserved only for those who could not perform useful work.  He feared that stopping more functional people of low intelligence from reproducing might lead to a labor shortage in unskilled occupations which would adversely affect the functioning of society. 

Of note is the fact that by July of 1942, psychiatrists were already aware of what was going on in Germany.  Kanner noted, “If [journalist and historian] William Shirer’s report is true – and there are reasons to believe that it is true – in Nazi Germany the Gestapo is now systematically bumping off the mentally deficient people of the Reich…” (p.21).

To Tell the Truth??

A new internet-based survey of 2,020 patients who had received treatment for depression, conducted by a Dr. Sawada and presented at the Annual Congress of the European College of Neuropsychopharmacology, revealed something that I have known about for a very long time.  A lot of patients lie to their doctors. 

About 70% admitted lying at least once!  66% stretched the truth about daily activities such as work while almost 53% were untruthful about their symptoms - on purpose.

That patients can be less than honest should come as not surprise to anyone familiar with the literature on patient compliance with doctors' orders.

In the United States, according to some estimates 20-30% of prescriptions are never even filled at the pharmacy.  According to the World Health Organization, only 50% of people complete long-term therapy for chronic illnesses as they were prescribed. According to the US government's Office of the Inspector General, research indicates that 55 percent of the elderly do not follow the medication regimens prescribed by their physicians.

As described in my post of August 10, 2010, Don't Ask, Don't Tell, I  had an exchange with another psychiatrist that included the following:

Me: Family members will not usually volunteer the whole truth during a superficial visit with a doctor. Anyone who thinks that family members act the same way at home as they do in front of an authority figure, or that they will be totally honest about things like family violence or abuse, needs to get out more.

Other Psychiatrist : This broad statement essentially implies and equates pediatric mental disorders with the family abuse and neglect... The reasoning goes along the line "if child has problems, someone in the family caused them." And if the family does not offer any evidence of maltreatment,they are lying. A fundamental fallacy, in my opinion, that for decades prevented psychiatrists from understanding the nature of mental disorders... Suspecting family members of hiding "the whole truth" is a regrettable statement from a professional. 

It sounded to me like it is the writer who is saying that poor discipline or even child abuse is never an issue in any behavior or psychiatric problem at all! And apparently he believes patients never fib or hide information! And the statement about offering evidence of maltreatment?  First, that implies that this psychiatrist probably never even asks about it in the first place but waits for the parents to "offer" evidence. And since we were talking about children, if the parents had been abusive or neglectful, does anyone think that they would admit this to someone who was legally required to turn them in to the authorities?  REALLY?!?

I pointed out in that last post: Obviously this psychiatrist has never done any serious psychotherapy, or he would know that a patient may not reveal absolutely essential information about their situation until they have been seeing a therapist for months.



I've gotten into similar argument with folks who are against psychiatric meds and who demonize antidepressants and mild sedative/hypnotics by telling tales of people they talk to who claim to have had horrible and never-ending side effects from these drugs.  While some of the people such folks talk to undoubtedly did have very bad reactions to the drugs in question, many are also taking other illegal drugs and/or a concoction of several different prescribed psychiatric and/or somatic medications.

More importantly for purposes of this discussion, they may be scapegoating the drugs for personality and family problems that are the real cause of their "side effects." 

When I make the latter point, the usual response I get is something to the effect that "I've met plenty of people who have none of the additional issues you are talking about."  My answer: And you know this how? Do you think people just advertise all the things they are ashamed of and brag about family dysfunction?

Going back to survey that is the subject of this post, the reasons that the subjects gave for not being truthful to their doctors were very instructive.  Although males often withheld or altered the facts due to a fear that the doctor would recommend that they take sick leave or quit their job, the two most common reasons given for lying to the doctor were:

1. The patients found it difficult to talk to their doctor, particularly about things they were ashamed of or embarrassed about (49%).
2. They thought that the doctor would not take it seriously, even if they they told him or her (36%).  In particular, females said they could not trust their doctor or that "he looked busy."

Both of these reasons concern relationship issues between doctor and patient.  For want of a better term, they concern a doctor's bedside manner.  In therapy circles, these issues are referred to as the therapeutic relationship (or as transference and counter-transference, respectively, if the patient and doctor are reacting to one another subconsciously).



In my experience, A doctor - particularly a psychiatrist who must deal with a lot of skeletons in closets - who is not in a rush, who appears empathic and non-judgmental, and who does not seem uncomfortable with highly emotionally-charged or typically squirm-inducing subjects, is far more likely to get the truth out of a patient than one who lacks these qualities.

If the psychiatrist sees the patient frequently in psychotherapy and maintains these qualities, more of the truth will emerge from the patient.

Many of today's psychiatrists only see patients for a few minutes (thereby appearing to be "busy"), and are only interested in the patient's symptoms and not in the environmental and psychological context in which those symptoms take place).  One can safely assume that in many cases such doctors are getting a highly-distorted picture of the patient's clinical condition.

These problems are magnified in randomized controlled drug studies (RCT's).  In such studies, diagnostic interviews and self-report instruments focus only on symptoms and researchers do not even bother to take a psychosocial-relationship history from research subjects. 

Furthermore, researchers do very little to establish a trusting therapeutic relationship with their subjects.  Many do not know how to establish this type of relationship, but even if they did know how and really wanted to try, doing so would increase the placebo response rate and should therefore not be done in a so-called empirical study!

Since they are not based on lab tests, the data from these studies is almost all dependent on the truthfulness of the subjects, which as I have been arguing is highly questionable in many cases. When lab values are available, such as blood levels of such drugs as lithium, depakote, and tricyclic antidepressants, they are seldom checked frequently if at all. 

And even if they were, they are still subject to manipulation by the research subjects.  Most drug levels should be so-called trough levels - the lowest level that a medicine is present in the body over a 24 hour period.  Levels should generally be drawn 10-12 hours after the last dose of the medication taken and before the next dose is taken.  All a patient has to do to appear to be taking the drug more often than he or she actually is is to take a dose just prior to the blood draw.

This is another big reason why randomized controlled studies should NOT, by themselves, be the gold standard of so-called "evidence-based" psychiatry.

Stats.con



Buy this book!

The current craze known as evidenced-based medicine denigrates widespread clinical experience as a synonym for anecdotal and therefore not really scientific.  Anecdotal evidence in this sense is an individual practitioner's experience using a certain drug or procedure on a certain patient. 

If the drug seems to work or not work in an individual instance, that does not mean that the drug led to the observed results.  The results could be a placebo effect, due to the tincture of time, a fantasy of the observer, or caused by some other factor besides the drug.  A single clinical anecdote may suggest experiments, but by itself cannot be considered good evidence of the efficacy of the drug or procedure.

Of course, if a lot of different practitioners tend to get the same sort of results in a wide variety of different clinical populations, then the differences between that phenomenon and a single anecdote are markedly obvious.

Still, in the evidence-based medicine craze, the gold standard is a randomized, double blind, placebo controlled clinical trial, or RCT.  That is supposed to be the mark of true science.  But not so fast.

A book called Stats.con by James Penston, which I highly recommend reading, makes the case that, while RCT's are important, they are hardly the end all and be all of science.  They have a lot of issues.  And sometimes widespread clinical experience is far better evidence for the efficacy of a drug or procedure than an RCT. 

Lies, damn lies, and statistics.

Penston points out that surgeons have for decades operated on various organs in the body with the assistance of effective anesthetics and muscle relaxants - "all without a statistician in sight."

In this post I would like to cite several lines from the book that I think are important in evaluating medical "evidence."

"An individual instance may not logically refute a statistical study, but it cannot be dismissed as being irrelevant to the matter."  (p. 4).

"No wonder politicians are rarely fazed by statistical data.  Presented with a study that challenges their position, they simply bring into question the authors or the data." (p. 7).  Because it is very easy to introduce bias into an RCT, and because any study will have weaknesses, they can always be challenged.  And studies often contradict one another.  I notice some of my critics apply their criteria for judging a study far differently for studies they agree with than for those they do not.  You can always cherry pick studies in this manner to prove whatever argument you are making.

"The simplicity of [randomizing patients to active treatment and placebo groups in an RCT] hides the practical difficulties of selecting two groups of patients equally matched in terms of all relevant factors." (p. 13).

"Numerous errors [in technical aspects of a study] may occur that threaten the integrity of the findings yet these are far from easy to detect."  (p. 19). In other words, practitioners reading the study might easily be oblivious to major weaknesses of the study.

The size of a study sample is "inversely proportional to knowledge of the subject matter, the size of the treatment effects, the value of the results to individual patients, and the overall importance of the study." (p. 23).  This has to do with the difference between statistical significance and clinical significance. 

Say that a given drug caused a reduction in the incidence of gallstones by only one percent or in only one percent of patients.  A study may show that this difference is statistically significant, but in terms of its relative value to you when balanced against the cost and/or the side effects, it may be next to worthless.  In order to find such a small statistically significant differences between one drug and another, or between a drug and a placebo, one needs a very large sample.  Significant results in studies with smaller samples tend to be much more dramatic from a clinical standpoint.

"Confounding is present when both the supposed cause and the supposed effect [of a clinical result] are associated with a third factor which is responsible - in whole or in part - for the difference in outcome detected between the groups...no methods are available for correcting for the presence of unknown confounding variables." (p. 30).

"Yet, surprisingly, trials that are being reported as being randomized yield groups with equal numbers of patients more often than would be expected by chance." (p. 63).  This is strong evidence that the randomization process in a lot of studies is being manipulated to influence the results.

In re subjective symptoms - like almost all symptoms seen in psychiatry: "Such changes, even with the assistance of validated scales of symptoms, depend entirely on the account given by each individual patient." (p.69).  There is strong evidence that patients frequently lie to doctors for a variety of reasons.  I will address this issue in more detail in a future post.

On subgroup analysis:  Critics of the literature refer to a process that pharmaceutical companies use called data mining.  If the main result of the study is not to their liking, they will divide the subjects into several different subgroups based on some criteria like gender‎ or age.  The more of these subgroup analyses they do, the more likely they will find a significant result.  Unfortunately, the statistical odds in this situation indicate that the result is most likely just a coincidence:

"[The pitfalls or running analyses on subgroups of subjects in a randomized controlled study was illustrated by one study in which the] ...overall results showed a reduction in mortality from myocardial infarction with aspirin yet subgroup analysis suggested that the drug was of no benefit to those born under the sign of Gemini or Libra." (p. 75).

"Strictly speaking, statistical data apply to groups, not to individual patients.  But clinicians treat individual patients....In every case, we can legitimately ask whether the findings [of an RCT] are applicable to that particular individual." (p. 89).

"Estimates suggest that less than 1% of patients will be recruited to trials. Thus, from this measure alone, it's highly unlikely that participants will be representative of the broader population of patients with the disease." (p.91).

"Patients excluded from [RCT's] tend to have a worse prognosis than those who are recruited." (p. 95) and "Most clinical research is carried out in teaching hospitals by medical staff with both a particular interest in the disease and considerable expertise, supported by nurses with specialist skills and well qualified junior doctors.  Under these circumstances, the standard of care is expected to be ...superior to the average general hospital." (p.96).   [This second point may in many cases not be true in the United States due to the proliferation of Contract Research Organizations or CRO's].  The chances study patients will get better can be much better than for your average Joe, making the results of the study less generalizable to the population of all patients with a disease.

Another big point made by Penston concerns what he calls the relative risk deception.  Let us say that a cancer drug reduces the percentage of recurrences from 5% to 4%.  The authors may then claim they have reduced the chances of recurrences by 1 in 5, or 20%.  That is highly misleading.  This cited rate of risk reduction, as old Albert Einstein might say, is relative.  The absolute risk reduction is only 1% - which may even be within the margin of error of the study - since 95% of the sample would not have had a recurrence regardless of whether or not they took the drug!

And finally:

‎"We may say, for example, that the probability of the clutch [of a certain car] surviving 50,000 miles based on the analysis of a large sample is 0.6, but this hardly applies to the one owned by a driving instructor who, day in and day out, witnesses assaults on the clutch of his car." (p. 120).