The independent record
Calibration.
How accurate are the markets, really? Every number here is computed from resolved flags: the mechanical record of when a market price and the Scopes Fair Value diverged, tracked to outcome. No opinions, no projections; the statistics decide what can and can't yet be said.
Is the difference real yet?
248/612 resolved to the Scopes Fair Value side. If both sides were equally good, a split this uneven or more would occur by chance < 1% of the time (p = < 0.01).
The difference is statistically significant at the 95% level.
At the current flag rate, a true 60/40 skill difference becomes detectable around n ≈ 194 resolved flags; a 55/45 difference around n ≈ 783.
Brier comparison
Mean Brier difference (market minus fair value): -0.047 [95% CI -0.063, -0.032]
The interval excludes zero: the market has shown the more accurate probabilities so far.
The Brier score is the mean squared error between a probability and the 0/1 outcome, lower is more accurate. Probabilities are taken as recorded at flag time, not revised after the fact. The interval is a cluster bootstrap: correlated outcomes, e.g. multiple flags on the same election cycle, are resampled together so they cannot manufacture false precision. This is a paired forecast-comparison in the spirit of Diebold–Mariano. What the scorecard’s split does and doesn’t mean →
By desk
| Desk | n | Right (fair / market) | Binomial p | Brier: market | Brier: fair |
|---|---|---|---|---|---|
| Rate Scope | 2 | — · n < 10 | |||
| stocks | 45 | 22 / 23 | 1.00 | 0.365 | 0.447 |
| energy | 404 | 154 / 250 | < 0.01 | 0.318 | 0.304 |
| weather | 161 | 70 / 91 | 0.11 | 0.276 | 0.469 |
Explore the record
Filter the resolved record by desk, event family, minimum Scopes Divergence, and date range, every slice recomputes the same binomial test and paired Brier comparison. Slices below 10 flags report counts only, no verdict.
Loading the record…
Flags are deduplicated (one per binary event) and grouped into event families for testing; families, not individual flags, are the unit of statistical resampling. Sports flags are the closest to independent observations; election flags within one cycle are treated as correlated.