Do Neural Nets Dream Of Electric Hobbits?
Read the original on Slate Star Codex →
Follow-up reading
↳ GPT-2 As Step Toward General Intelligence — next-day GPT-2 continuation · Feb 2019
Summary
On GPT-2's release (Feb 2019). Showing its outputs (the meandering Civil War essay, the Legolas/Gimli LOTR passage), Scott's central insight: GPT-2's text reads exactly like a DREAM -- high-level narrative coherence but details that appear/disappear, context-switches, things that are also other things -- and that's no coincidence, because GPT-2 works like the dreaming brain. Both are prediction machines (brain predicts sense-data; GPT predicts next token); a predictive net doubles as generative. Drawing on Hobson/Hong/Friston, during wakefulness prediction is chained to sensory error-correction, but dreaming (like GPT running unchained from a prompt) lets it free-run. Three accounts of dream-logic (Friston's model-complexity reduction; wake-sleep/Boltzmann 'dreaming' nets; or simply 'this is what a prediction machine at 5% capacity sounds like -- about as good as 2019's best AI'). Closes with GPT-2 describing itself.
Why this score
Quality 78 · Excellent. A prescient, original, lucid essay connecting LLMs, predictive processing, and dream neuroscience -- the GPT-as-unchained-dreaming-brain frame is a genuine exported insight, remarkable for early 2019. High-Excellent.
Claude’s paradigm shift 58 · Moderate. Low-Major-shift. The GPT-as-dreaming / shared-prediction-machinery framing was genuinely novel and forward-looking when published.
Real-world impact 3 · Moderate. A prescient, original 2019 essay connecting LLMs, predictive processing, and dream neuroscience (the GPT-as-unchained-dreaming-brain frame) — a genuine exported insight. Conceptual influence within AI/ML discourse, no material change — modest RWI.