First posted to SSRN in July 2026 under the working title Grading the Graders. The revised version, retitled The Certainty Business, adds an independent second-coder validation of the full grading ledger. A follow-up rescore of the 2026 cycle is pre-registered for December 31, 2026.
The question
Every January, political risk houses, global banks, asset managers, and multilaterals publish year-ahead outlooks that corporations and investors buy to guide capital allocation. This paper asks a simple question of that product: when the calls are graded against what actually happened, how do they score?
The design
Every forecaster is graded at the maximum depth its public record permits, in three tiers. Tier one is a full cross-section: fifteen institutions, spanning political risk and macro research houses, global banks and asset managers, one multilateral, and two prominent individual forecasters, graded across the 2024, 2025, and 2026 cycles. Tier two extends the banks on their flagship annual call, the published S&P 500 year-end target, across six cycles from 2021 through 2026, using dated press documentation. Tier three extends the two institutions whose complete ranked forecasts are continuously public, Eurasia Group’s Top Risks and the World Economic Forum’s Global Risks Report, across a full decade, 2016 through 2026.
The depth distribution is itself a finding. The industry that sells prediction mostly does not preserve its predictions in checkable form.
The rubric
A call earns credit only if it is specific enough to guide a real decision: it must name an actor, a mechanism, a direction, and a time window. Calls are scored against an answer key restricted to events passing a five-part materiality test, so forecasters are graded only on the events that actually moved markets.
What the record shows
- No call in any cycle of the sample earned full credit under the rubric.
- The street consensus on the S&P 500 missed direction in consecutive opposite years: a median target of 4,825 ahead of the 19 percent decline of 2022, and a median of 4,000 ahead of the 24 percent rally of 2023, with survey evidence that targets are revised intra-year to follow realized prices.
- Each January’s consensus is well approximated by a persistence projection of the prior year’s realized environment.
- The instrument transmitting each cycle’s largest market event never appears on any list.
- The genuinely strong calls in the record share one feature: each described a visible, already-formed present condition. None predicted a discontinuity.
Why the product fails
The pattern reflects structure, and competence cannot fix it. The process being forecast is nonstationary. The events are single-shot, so forecasters never receive calibration feedback. Incentives reward vagueness. And the payoff matrices are specified at the level of the state, while the decisions that move markets are made by leaders optimizing personal survival, a game that offers no base rates to extrapolate and no stable logic to solve.
What works instead
Credit accrues where a product describes conditions already visible in the present, and it evaporates at discontinuities. The practical conclusion for corporations and investors is to stop paying for point predictions and buy the things that hold up: scenario preparation, monitoring of visible conditions, and short forecast horizons. Conditions are readable even when events are not.
All grading tables are published with the paper for replication.