Quick answer
Calibration: A forecaster (or market) is well calibrated when events given an X% probability happen about X% of the time over many forecasts.
Also called: forecast calibration, reliability
Explained, with an example
Calibration is checked by grouping forecasts into bins (e.g. 60–70%) and comparing each bin's average forecast with the share of events that occurred. Proper scoring rules such as the log score and the Brier score reward honest probabilities and penalise systematic over- or under-confidence [1] [2]. To check a market's calibration, convert prices to implied probabilities (removing any Overround) before binning.
Try it: tools and tutorials
Frequently asked questions
What is calibration?
A forecaster (or market) is well calibrated when events given an X% probability happen about X% of the time over many forecasts.
About this guide
Pred-Data publishes independent, source-linked explainers about prediction markets and the AI tools used around them. Every factual claim on this page links to a primary or reputable secondary source, checked on the date shown above. We do not run a trading venue, take bets, or give personal advice.
Author: Ken Lin · Corrections: if you spot an error, we review it and date-stamp the fix.
Affiliate disclosure
Some links on this site (marked “Partner link”) may be affiliate links: if you sign up through them we may earn a commission at no extra cost to you. Partner links use rel="sponsored nofollow" and never change our assessment. Platform links in our comparison tables are plain, non-affiliate links to official sites.
Sources
- Scores FAQ (proper scoring rules, log score) — Metaculus, accessed 2 Nov 2026
- G. W. Brier, "Verification of Forecasts Expressed in Terms of Probability", Monthly Weather Review 78(1), 1950 — American Meteorological Society (DOI), accessed 2 Nov 2026