Scott Alexander, curated
← Back to curation

Who Predicted 2023?

Quality
70
Strong
Claude Shift
42
Moderate
RWI
2
of 10

Summary

Results and analysis of Scott's annual forecasting contest (~3300 Blind-Mode + 460 Full-Mode entrants scored by Eric Neyman). Key findings: Metaculus (99.5th pct) beat prediction markets, superforecasters, wisdom-of-crowds, and Samotsvety; Manifold (89th pct) surprisingly underperformed a random group median ('the market mechanism seems to be subtracting value'); the superforecaster aggregate beat the participant aggregate; and most individual winners are ~90th-percentile skill plus luck (per how 2022 winners regressed).

Why this score

Quality 70 · Strong. Strong (upper): a data-rich, informative contest writeup with genuinely interesting findings (Metaculus-beats-everything; Manifold-subtracts-value), useful to the forecasting community; kept below Excellent as a structured results report rather than a thesis essay.

Claude’s paradigm shift 42 · Moderate. Moderate-low: the aggregation-comparison findings are empirical and topical, not a novel framework.

Real-world impact 2 · Minor. Minor/within-discourse: the Metaculus/aggregation results inform forecasting practice within a niche community.