Beware The Man Of One Study
Read the original on Slate Star Codex →
Summary
Introduces "beware the man of one study": because any contested question yields a whole bell curve of studies (from variation, design, and subtopic alone), anyone can cherry-pick a result - and crucially, even lists of dozens of studies, duelling meta-analyses, economist letters, and expert surveys can all be marshalled for either side (shown vividly on the minimum wage). The partial remedies: funnel plots where available; otherwise, lower your confidence, distrust biased "state of the evidence" claims, and always check what the other side has.
Why this score
Quality 80 · Excellent. An important, broadly-applicable epistemic lesson and a durable coinage - the point that a long list of studies can still be bull is sharp and frequently cited. Excellent.
Claude’s paradigm shift 60 · Notable shift. The "bell curve of studies / even meta-analyses can mislead" framing was a fresh, widely-adopted epistemic tool in 2014, building on publication-bias awareness. Notable shift.
Real-world impact 4 · Moderate. Coins 'beware the man of one study' — a durable, frequently-cited epistemic lesson (even long study-lists and duelling meta-analyses can be cherry-picked). Broad conceptual uptake within intellectual discourse, no material change — mid RWI.