Scott Alexander, curated
← Back to curation

Against That Poverty And Infant EEGs Study

Quality
70
Strong
Claude Shift
42
Moderate
RWI
2
of 10

Summary

A statistical debunk of a viral paper (Baby's First Years) claiming a poverty-reduction cash intervention ($300/month) changed infant EEG brain activity, hyped by Vox/NYT. Scott first lays out his prior suspicion (shared-environmental cognition effects are notoriously hard to find; social-science neuroimaging has too many researcher degrees of freedom converting squiggly lines to single numbers; poverty/cognition findings keep failing to replicate -- a recent project found 16/20 priming replications indistinguishable from zero). Then the takedown: every EEG difference LOST significance after the appropriate multiple-comparisons (Westfall-Young) adjustment -- which the authors performed and then effectively ignored, still claiming a positive result; the strongest result (beta waves) wasn't pre-registered; the paper got the easier PNAS-NAS-member publication track; and -- the killer -- Andrew Gelman's randomization shows the eyeballable graph difference is an artifact (coin-flip Red/Blue groups look just as different). Concludes the study is a typical null-result that should not have been reported as an unqualified positive, with a careful heuristic-vs-bias caveat.

Why this score

Quality 70 · Strong. Strong. A sharp, rigorous statistical debunk -- the multiple-comparisons point and the Gelman-randomization artifact demonstration are clean and convincing, and it's careful not to overclaim (cash grants MIGHT affect EEG; this study just doesn't show it). Mid-Strong: it marshals others' critiques cleanly rather than building a framework.

Claude’s paradigm shift 42 · Moderate. Moderate. Assembles existing critiques (Gelman, Ritchie, etc.) into a clear summary; little new of its own.

Real-world impact 2 · Minor. Minor. Within the replication/meta-science discourse; conceptual. 2.