The Black Swan: The Impact of the Highly Improbable · chapter 5 · id the-black-swan-c5-04-philip-tetlocks-20-year-study-
“Philip Tetlock's 20-year study of 284 experts making 82,361 predictions found that expert forecasters were only marginally better than random chance -- described as 'dart-throwing chimps' -- with the least accurate experts being the most confident and most quoted in media.”
TRUE, BUTconfidence: high⚠ extracted by pipeline, re-audit pending
Receipts
Tetlock (2005), 'Expert Political Judgment', Princeton University Press, Chapter 1source alive
284 experts whose professions included commenting or offering advice on political and economic trends generated 82,361 forecasts over nearly 20 years. Experts were only marginally superior to random chance and generally outperformed by straightforward extrapolation algorithms.
Tschoegl & Armstrong (2008), SSRN Review of Tetlockpaywalled · manual check queued
The American Political Science Association awarded Expert Political Judgment the Woodrow Wilson Award for best book. Tetlock's methodology of tracking predictions with calibration and discrimination metrics is considered the gold standard.
Karma Colonialism: Experts Versus the Dart-Throwing Chimpssource alive
Tetlock himself has gone to great pains to squelch the soundbite about experts being equivalent to dart-throwing chimps, as it oversimplifies his findings. The key finding was that 'hedgehog' thinkers (one big idea) performed worse than 'fox' thinkers (many small ideas).
This claim is a stable, citable object. If you can falsify a verdict, tell us. Corrections are loud here.