Two Kinds Of Caution
Read the original on Slate Star Codex →
Summary
Responds to an FT piece ('boring techs like barbed wire / shipping containers matter more than flashy ones,' using AI as a foil -- 'perhaps we should fear warehouse-management software more than superintelligent AI'). Scott: the article never argues flashy techs AREN'T disruptive (counter-example: the stirrup), and 'perhaps' is weaselly and commutative. His key contribution is the local-vs-global caution distinction: the NASA engineer with a bad feeling is LOCALLY cautious if he defers to the hundred experts, but GLOBALLY cautious if he runs to stop the launch -- and AI risk gets the wrong kind ('let's not jump to conclusions') when we spend ~$9M/yr on AI safety, ~1% of what's spent on online farming games. The right worry is under-caution, not over-caution; cf. a general who won't post sentries because nothing is certain.
Why this score
Quality 74 · Strong. Top Strong (74). A substantive AI-risk essay whose local-vs-global caution distinction is a genuinely useful, generalizable contribution (epistemic caution vs outcome caution), with sharp supporting moves (the stirrup; 'perhaps' is commutative; the under-supply of meta-contrarianism). Held just below Excellent as a piece framed around rebutting one article.
Claude’s paradigm shift 56 · Moderate. Notable-edge (56). The local/global (epistemic vs outcome) caution distinction is a fresh, non-obvious, portable frame.
Real-world impact 2 · Minor. 2 -- minor/within-blog. Part of the AI-safety-advocacy discourse; the caution distinction is referenced, but no direct material footprint.