Trapped Priors As A Basic Problem Of Rationality
Read the original on Astral Codex Ten →
Summary
The foundational essay developing the 'trapped prior' — one of his most influential and most-applied rationality frameworks. Builds from the perception model (raw experience + context → perception, illustrated by the chessboard illusion, McGurk effect, red-dyed-wine illusion, and placebo as context-modulation), through normal Bayesian reasoning sliding into confirmation bias, to the trapped prior itself: phobias as a failure of habituation (flooding fails because being locked in with the Rottweiler is itself terrifying, so each exposure etches the fear deeper — the prior becomes trapped because every update now favors 'dogs are terrifying'; the bitch-eating-crackers / abusive-relationship version). The emotional version (van der Bergh: unbearable experience narrows the raw-experience channel to protect you, so context dominates). Then phobia-to-bias: scientifically-literate partisans get wronger with more facts; the 1979 capital-punishment study where both sides grew more certain from identical evidence; dog-whistle reasoning as a trapped-prior amplifier turning every neutral outgroup act into proof of evil. Carefully distinguishes the (fundamentally epistemic) trapped prior from emotion and from self-serving bias, then surveys escape routes (gradual exposure, Sloman-Fernbach mechanistic explanation, psychedelics loosening priors, meditation) and a research program — closing with the self-aware 'ironically, my prior on this theory is trapped.'
Why this score
Quality 84 · Excellent. Excellent band, high. A foundational, elegant, and generative framework that unifies perception, phobia, confirmation bias, political polarization, and trauma under one Bayesian mechanism, and which he (and the wider sphere) apply constantly afterward — the reference for the concept. Co-tier with the Rootclaim standout (84); held just below the 85+ curated greats and the Ivermectin standout (88) because it synthesizes van der Bergh and predictive coding (and candidly flags an unresolved gap in why the weighting function fails to update).
Claude’s paradigm shift 68 · Notable shift. Major shift, low. The components (predictive coding, the Bayesian brain, van der Bergh's precision work) pre-exist, but the trapped-prior synthesis — unifying phobia, bias, and polarization into a single named, exported frame — was genuinely novel and influential at publication.
Real-world impact 4 · Moderate. 'Trapped priors' became one of his most-applied rationality frameworks, constantly reused across the rationalist sphere to unify perception, phobia, confirmation bias, polarization, and trauma. Significant, broad conceptual influence (and clinically-adjacent framing), but elite/intellectual reach with no material change — mid RWI.