Professor Irving Kirsch who touted studies suggesting antidepressants are not effective
Many reviews have pointed out that there is a significant “publication bias” that affects studies of antidepressants: because studies that don’t find an effect tend not to be published a review of the published literature would suggest that these medications are more effective than they really are. Indeed this is been the basis of a great debate in the popular and professional literature about whether antidepressants “really work.”
Now a review of the studies looking at psychotherapy for depression finds that the same publication bias exists. Psychotherapy studies that did not find a significant benefit of treatment (compared with “placebo”) in patients with depression also tend not to be published and so the published literature on psychotherapy for depression overstates the treatment effect.
Here is a summary of the study and the results –
“We identified US National Institutes of Health grants awarded to fund randomized clinical trials comparing psychological treatment to control conditions or other treatments in patients diagnosed with major depressive disorder for the period 1972–2008, and we determined whether those grants led to publications. For studies that were not published, data were requested from investigators and included in the meta-analyses. Thirteen (23.6%) of the 55 funded grants that began trials did not result in publications, and two others never started. Among comparisons to control conditions, adding unpublished studies (Hedges’ g = 0.20; CI95% -0.11~0.51; k = 6) to published studies (g = 0.52; 0.37~0.68; k = 20) reduced the psychotherapy effect size point estimate (g = 0.39; 0.08~0.70) by 25%. Moreover, these findings may overestimate the “true” effect of psychological treatment for depression as outcome reporting bias could not be examined quantitatively.”
The magnitude of the publication bias in psychotherapy studies was actually larger than was found in previous reviews of studies of antidepressant medications, which was surprising.
Reviewer Peter Roy-Byrne, writing in Journal Watch Psychiatry, summarized the finding this way:
“The reduction in effect size (0.13) was almost twice the reduction (0.07) noted previously in a similar analysis of antidepressants. Moreover, the current authors note that they could not examine outcome reporting bias, which suggests that even their calculations may have overestimated the effects. Although this analysis should not discourage clinicians from using psychotherapy — still underutilized as a depression treatment compared with medication — it should temper the belief of some academic psychologists that psychotherapy is superior to medication for patients with depression.”
All of this led us back to the huge debate in the literature about whether antidepressants “work,” a debate that ended up focusing on the writings and theories of Irving Kirsch, a lecturer at Harvard and author of the book, The Emperor’s New Medications, which argued that for most patients antidepressants were no more effective than placebo.
The debate and its aftermath are neatly summarized in a review article (which is available free online) by Konstantinos N Fountoulakis published in World Psychiatry.
As Konstantinos Fountoulakis notes, this question has received an unprecedented amount of attention, and the question of the effectiveness of antidepressants is perhaps the most thoroughly investigated and debated question in the medical literature these days. All of which argues that the idea that there is a conspiracy of doctors and drug companies to push antidepressants on the unwary public is somewhat overstated, to say the least.
The article finding a publication bias in studies of psychotherapy, which had overlapping authors and a similar thoughtful design to the article finding a publication bias in studies of antidepressants, was published in the Open Access journal, PLOS ONE, whereas the study finding a publication bias favoring antidepressants was published in the New England Journal of Medicine, the most prestigious journal in all of medicine.
The Guardian, ordinarily a newspaper that I value highly, reported the findings of publication bias in antidepressant studies in an article that led with the statement, “Prozac, the bestselling antidepressant taken by 40 million people worldwide, does not work and nor do similar drugs in the same class, according to a major review released today.”
The Guardian did not even report the finding that there is a similar publication bias in studies of psychotherapy…
My conclusion is the same one that most writers of the few articles that did report on the psychotherapy study ended with: treatment of depression, particularly more severe and more chronic depression, is hard, and we often need to use all the treatments that might work to see good results.
And it is still true that there is not enough access to good quality psychotherapy.
References
Driessen E et al. Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of Health-funded trials. PLoS One 2015 Sep 30; 10:e0137864. (http://dx.doi.org/10.1371/journal.pone.0137864) – See more.
Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. PLoS Med. 2008 Feb; 5(2):e45.
Kirsch I. The Emperor’s New Drugs: Exploding the Antidepressant Myth. London: The Bodley Head; 2009.
Turner EH, Matthews AA, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. NEJM. 2008;358: 252–260. doi: 10.1056/NEJMsa065779. pmid:18199864
Fountoulakis KN, Möller HJ. Are antidepressants clinically useful? Conclusion of a decade of debate. World Psychiatry. 2014 Jun;13(2):201-2. doi: 10.1002/wps.20112. PubMed PMID: 24890076; PubMed Central PMCID: PMC4102296.
“Prozac, used by 40m people, does not work say scientists,” by Sarah Boseley, health editor. The Guardian.