Desperately Trying To Fathom The Coffeepocalypse Argument
Read the original on Astral Codex Ten →
Summary
Dissects a recurring bad argument against AI safety: 'people once worried about X [coffee/overpopulation/global cooling], it was fine, therefore AI will be fine.' Scott genuinely can't reconstruct why smart people find it persuasive -- it reduces to the 'halibut is a kind of fish, therefore AI is a kind of fish' non-sequitur. He steelmans it three ways: as an existence proof (weak -- one case of a false impossibility-claim, like Rutherford-then-Szilard, only refutes absolute certainty nobody holds); as a heuristic-trigger (requires establishing that moral panics outnumber head-in-the-sand failures -- but tobacco, lead, OxyContin, pre-WWI tensions cut the other way); and via 'what is evidence anyway' (the Russia-good-tanks analogy: any single fact proves nothing, yet arguments are built from facts -- the resolution is selective sampling, an unbiased random draw updates you a little, a cherry-picked one near-zero). Concludes he still doesn't understand the mindset, with the Douglas Adams 'understanding the universe would make it disappear' joke -- plus the footnote zinger that the coffee fears weren't even wrong (coffeehouses did foster the Glorious and French Revolutions).
Why this score
Quality 72 · Strong. Strong. The selective-evidence / random-sample analysis ('how does any single fact prove an argument?') is a genuinely useful epistemics handle, and the piece is sharp and funny. Held to mid-Strong because it is partly an exasperated complaint that doesn't fully resolve its own question.
Claude’s paradigm shift 50 · Moderate. Moderate. The Safe-Uncertainty-Fallacy and dueling-heuristics points are Scott's existing material; the fresh handle is the selective-sampling framing of how anecdotal counter-examples should (barely) update you.
Real-world impact 3 · Moderate. Moderate. Within the AI-safety-discourse plus a transferable lesson on weighing anecdotal-counterexample arguments; conceptual. 3.