The Black Swan: The Impact of the Highly Improbable · chapter 5 · id the-black-swan-c5-06-prediction-markets-and-expert-
“Prediction markets and expert forecasts consistently fail at predicting specific Black Swan events. The Good Judgment Project later showed that 'superforecasters' (top 2%) could outperform intelligence analysts with classified access by 30%, but only on incremental geopolitical questions, not true Black Swans.”
TRUE, BUTconfidence: high⚠ extracted by pipeline, re-audit pending
Receipts
Wikipedia: The Good Judgment Projectsource alive
IARPA ran a forecasting tournament from 2011 to 2015 where GJP won. The top forecasters were reportedly 30% better than intelligence officers with access to classified information.
Good Judgment Inc: Superforecaster Accuracy (2022)source alive
Superforecasters -- the top 2% of individual performers -- achieved standardized Brier scores averaging -0.34 across tournament years, substantially outperforming other participants.
EA Forum: Evidence from the Good Judgment Projectsource alive
GJP questions were typically about near-term geopolitical events with well-defined resolution criteria. These are fundamentally different from Black Swan prediction, which concerns events that are by definition outside the forecaster's model.
This claim is a stable, citable object. If you can falsify a verdict, tell us. Corrections are loud here.