Superforecasting

Superforecasting argues that better prediction is possible when uncertain judgments are stated as specific probabilities, tested against outcomes, and revised through disciplined feedback. Philip E. Tetlock and Dan Gardner do not claim that the future is generally knowable, or that a special class of people can see around corners. Their narrower and more practical claim is that some bounded questions can be forecast better than others when the question is clear, the time horizon is defined, the answer can be scored, and forecasters learn from the record of their own errors.

That matters because most public and organizational forecasting falls into two traps. One trap is fatalism: politics, markets, technology, and crises feel so complex that prediction seems like a fool’s errand. The other is theatrical certainty: experts and commentators speak in vivid, confident language, but their claims are rarely precise enough to be tested. Superforecasting rejects both. It treats forecasting as a craft of judgment under uncertainty, not as prophecy and not as pundit performance.

A forecast becomes useful when it can be wrong in public

The book’s most important move is to make prediction accountable. A claim such as “the government is under pressure” or “the market will probably weaken” may sound informative, but it hides the real judgment. How likely is the event? By what date? What exactly would count as the event happening? Without those details, people can preserve the feeling of insight while avoiding the discipline of being checked.

The Good Judgment Project supplies the book’s central research setting. It was part of IARPA’s ACE forecasting tournament, an effort to improve intelligence forecasting by eliciting, combining, and empirically testing probabilistic judgments. Participants answered real geopolitical questions over several years. One research example asked whether Silvio Berlusconi would leave office before a specified date. The point of the example is not the Italian case itself. It shows how a messy political situation can be converted into a bounded, resolvable probability question.

Scoring changes the behavior around judgment. Superforecasting emphasizes Brier scores, which measure the squared distance between a forecast probability and the eventual outcome. Lower scores are better. A 60% forecast for an event that happens is better than a 51% forecast for the same event, because it gave more information and was directionally right. A 99% forecast for an event that does not happen is punished sharply, because confident error is costly.

This is not a worship of metrics. It is a way to make confidence answerable. A culture without scoring often rewards fluency, status, and memorable explanations. A scoring culture asks whether the probability was any good. That distinction is why the book is hard on untested punditry without being hard on expertise itself. Knowledge matters, but only when it improves the forecast rather than decorating certainty.

Calibration is not the same as staying vague

A good forecaster is not simply cautious. Calibration means that, over many cases, events assigned a 70% probability happen about 70% of the time. That is valuable because it connects confidence to reality. But calibration alone is not enough. A forecaster who keeps every answer near 50% may avoid spectacular embarrassment while offering little help to decision-makers.

Tetlock and Gardner therefore pair calibration with resolution: the ability to distinguish stronger signals from weaker ones and assign meaningfully different probabilities when the evidence warrants it. Good forecasting requires both humility and discrimination. A forecaster should be reluctant to claim certainty in a noisy world, but also willing to move away from the middle when the evidence is strong.

This distinction prevents a common misreading of the book. Superforecasting is not an argument for timid forecasts. It is an argument for well-sized forecasts. Overconfidence is dangerous because it understates uncertainty. Permanent vagueness is also dangerous because it refuses to translate evidence into decision-relevant probabilities. The best forecast is neither bold for the sake of boldness nor safe for the sake of avoiding blame. It is as sharp as the evidence justifies.

The outside view restrains the story you want to tell

Superforecasters begin with base rates before immersing themselves in the vivid details of the case at hand. This outside view asks what usually happens in similar situations. It matters because current events feel unique. Names, headlines, personalities, and anxieties make the present case seem unlike anything else. A base rate interrupts that spell by forcing comparison.

The outside view is not a command to ignore local evidence. It is an anchor. After starting with a reference class, the forecaster moves to the inside view: What is special here? Which case-specific facts justify adjusting the starting probability? How large should the adjustment be? The method is disciplined movement from comparison to detail, not blind obedience to historical averages.

Fermi estimation serves a related function. The book discusses the classic problem of estimating the number of piano tuners in Chicago by breaking it into smaller pieces: how many pianos there are, how often they are tuned, how long a tuning takes, and how many working hours a tuner has. The value is not that every input is known exactly. The value is that the estimate becomes inspectable. Once a judgment is decomposed, people can debate the assumptions that actually drive it.

That habit applies beyond numerical puzzles. A political or business forecast often depends on several subclaims. Which actors must behave a certain way? Which constraints matter most? Which piece of evidence would change the probability? Decomposition slows the jump from impression to conclusion. It also makes disagreement more productive, because people can locate the fragile assumption instead of arguing over the whole story at once.

A compact routine follows from the book’s logic:

1. State the question so a later observer can tell how it resolved.
2. Set the deadline and resolution criteria before debating the answer.
3. Start with a base rate or reference class.
4. Break the forecast into assumptions that can be checked or challenged.
5. Assign a numerical probability rather than a verbal hedge.
6. Update when new evidence arrives, then compare the forecast with the outcome.

The routine is modest by design. It does not remove uncertainty. It makes uncertainty visible enough to learn from.

Foxlike thinkers revise without surrendering judgment

Tetlock’s fox and hedgehog distinction, carried forward from his earlier work on expert political judgment, is one of the book’s central cognitive contrasts. Hedgehog-like thinkers organize the world around one big theory and tend to fit ambiguous evidence into it. Foxlike thinkers use many smaller models, draw from multiple sources, and revise when the evidence changes.

The point is methodological, not decorative. The claim is not that generalists always beat specialists, or that expertise is useless. A domain expert with good evidence, base-rate awareness, and willingness to update can be a strong forecaster. A nonexpert who clings to one pet explanation can be a poor one. The fox is not a personality brand. It is a way of handling uncertainty.

Forecasting punishes identity-protective thinking. If someone’s public role depends on being the permanent optimist, the permanent skeptic, or the defender of a grand theory, updating becomes psychologically expensive. Every new probability threatens the self. Foxlike thinking lowers that cost because no single theory has to explain everything. A fox can use a model, then put it down when it stops helping.

The Good Judgment Project also makes an anti-credentialist point without becoming anti-intellectual. The book describes strong forecasters who did not match the usual image of elite experts, including ordinary volunteers from varied backgrounds. Their performance mattered because it was tracked. Status did not vanish, but it stopped being the only proxy for judgment. In a scored environment, the question becomes not who sounds authoritative, but who improves the accuracy of the probability.

Updating is learning, not twitchiness

Superforecasters update their beliefs as new evidence arrives. Research connected to the Good Judgment Project found frequent belief updating to be a strong behavioral predictor of accuracy. But the book’s claim is not that changing your mind constantly is virtuous. Updating helps only when the size of the update matches the weight of the evidence.

This is where probabilistic thinking becomes practical. Weak evidence should move a forecast a little. Evidence that directly changes a key assumption should move it more. A forecaster who jumps from 30% to 80% on thin information is not being open-minded; the forecaster is overreacting. A forecaster who refuses to move from 30% after strong contrary evidence is not being steady; the forecaster is defending a prior belief.

Scored feedback changes the meaning of being wrong. In ordinary argument, error often feels like humiliation. In a forecasting tournament, error becomes diagnostic. If someone is repeatedly wrong at high confidence, the lesson is to recalibrate. If someone never moves far from 50%, the lesson may be to improve resolution. The score does not explain everything, but it points to a specific weakness that can be examined.

Teams can strengthen this process when they protect independence and revision. The Good Judgment Project used teams, selected top performers, training, and aggregation. Collaboration can bring in more evidence, catch errors, and expose hidden assumptions. It can also amplify overconfidence if people converge too quickly or defer to status. The useful team is not a chorus. It is a setting where people can ask what would change their minds and where the forecast is most fragile.

Aggregation matters for the same reason. Individual judgment is noisy. Combining forecasts can improve accuracy when the inputs contain partly independent information and are integrated intelligently. But a crowd repeating the same rumor is not many signals. The book’s broader lesson is that group judgment improves when disagreement is structured around accuracy rather than politics, rank, or rhetorical force.

The promise is powerful because it is bounded

The strongest version of Superforecasting applies to questions that can be stated clearly, resolved within a useful time frame, and scored against reality. Many geopolitical, economic, intelligence, and organizational questions fit that pattern. Many other questions do not. Long-range transformations, moral judgments, preferences, and events with unclear resolution criteria cannot be made rigorous merely by attaching a number to them.

Even where the method applies, probabilities do not make decisions by themselves. A 35% chance of a severe downside may be acceptable in one context and intolerable in another. Forecasting improves the input to judgment; it does not replace values, strategy, or risk tolerance. The book is most useful when it separates two questions that organizations often blur: what do we think will happen, and what should we do if that probability is right?

The cultural demand is the hardest part. Many institutions reward confidence, loyalty, and persuasive narrative more visibly than calibrated accuracy. Superforecasting asks for a quieter discipline: define the question, expose the assumptions, assign a probability, update with evidence, and keep score. That will not make the future transparent. It will make uncertainty less theatrical and more usable.