Scott Alexander, curated
← Back to curation

The Efficacy Of Everything In Psychiatry In One Graph Plus Several Pages Of Dense But Necessary Explanation

Quality
71
Strong
Claude Shift
40
Moderate
RWI
2
of 10

Summary

Discussion of Huhn et al (2014)'s graph of effect sizes for drugs vs therapy across psychiatric disorders. Conveys useful empirical content (effects mostly ~0.5, comparable to general medicine -- debunking 'psychiatry is less effective,' per Leucht) with characteristic honesty: he admits he doesn't fully grok standardized mean differences on non-normal distributions, flags three surprises (drugs > therapy for borderline and insomnia -- but tiny samples; drugs better at preventing relapse than acute episodes -- with a sharp selection-bias self-correction). The killer methodological point: low-quality psychotherapy trials showed effect size 0.74 vs 0.22 for high-quality ones ('proof it's way too easy to inflate positive findings'), and psychotherapy research 'isn't even good enough to have publication bias.'

Why this score

Quality 71 · Strong. A solid, informative journal-club-style post with genuinely sharp methodological observations (trial-quality inflation; the publication-bias quip) and honest statistical humility -- but fundamentally a discussion of someone else's graph. Strong.

Claude’s paradigm shift 40 · Moderate. Moderate. Analysis plus caveats on others' meta-analytic data; the trial-quality point is sharp but not novel.

Real-world impact 2 · Minor. A solid journal-club post on Huhn et al's effect-size graph conveying useful empirical content (psychiatric effects mostly ~0.5, comparable to general medicine) with sharp methodological observations and honest statistical humility — but a discussion of someone else's graph. Conceptual influence within psychiatry discourse, no material change — low RWI.