The independent record
Scorecard statistics.
How accurate are the markets, really? Every number here is computed from resolved flags: the mechanical record of when a market price and the Scopes Fair Value diverged, tracked to outcome. No opinions, no projections; the statistics decide what can and can't yet be said.
Is the difference real yet?
366/786 resolved to the Scopes Fair Value side. If both sides were equally good, a split this uneven or more would occur by chance 6% of the time (p = 0.06).
No statistically significant difference yet (binomial test, n = 786).
At the current flag rate, a true 60/40 skill difference becomes detectable around n ≈ 194 resolved flags; a 55/45 difference around n ≈ 783.
Brier comparison
Mean Brier difference (market minus fair value): 0.001 [95% CI -0.007, 0.009]
The interval includes zero: no side is reliably more accurate yet.
The Brier score is the mean squared error between a probability and the 0/1 outcome, lower is more accurate. Probabilities are taken as recorded at flag time, not revised after the fact. The interval is a cluster bootstrap: correlated outcomes, e.g. multiple flags on the same election cycle, are resampled together so they cannot manufacture false precision. This is a paired forecast-comparison in the spirit of Diebold–Mariano. What the scorecard’s split does and doesn’t mean →
By desk
| Desk | n | Right (fair / market) | Binomial p | Brier: market | Brier: fair |
|---|---|---|---|---|---|
| Rates | 4 | — · n < 10 | |||
| Polls | 12 | 3 / 9 | 0.15 | 0.196 | 0.242 |
| Sports | 1 | — · n < 10 | |||
| stocks | 141 | 73 / 68 | 0.74 | 0.122 | 0.147 |
| energy | 628 | 287 / 341 | 0.03 | 0.197 | 0.189 |
Explore the record
Filter the resolved record by desk, event family, minimum Scopes Divergence, and date range, every slice recomputes the same binomial test and paired Brier comparison. Slices below 10 flags report counts only, no verdict.
Loading the record…
Flags are deduplicated (one per binary event) and grouped into event families for testing; families, not individual flags, are the unit of statistical resampling. Sports flags are the closest to independent observations; election flags within one cycle are treated as correlated.