Research · pre-registered study
Markets vs polls on the Senate toss-ups: pre-registered
The Scopes Desk
Some 2026 Senate races tightened into coin-flips only in the final weeks. This plan pre-registers one mechanical question about them: on the most competitive races, is the Kalshi market or the Scopes Poll Composite the better estimate of the result? The universe, the selection rule, the reference point, the metric and the verdict are all fixed here, before Election Day, and the result is computed from the record afterward.
It is deliberately separate from the frozen eight-seat Senate control forecasting study. That field was fixed in September and is not changed here. This study covers the late-breaking competitive races that roster excludes; a race may appear in both, scored independently in each.
The specification
Universe (selection rule, fixed at registration)
Every 2026 U.S. Senate race whose Scopes Poll Composite estimate of P(Republican wins) is within 3 points of a coin flip, that is in [0.47, 0.53], as of the registration date. The criterion is knowable now; the outcome is not, so selecting on it carries no look-ahead. The composite as of the registration date lives in the append-only, timestamped record, so the qualifying set is fixed and mechanically reproducible from that as-of value. Races onboarded at registration specifically because they entered the band (for example newly competitive states) are included on the same footing once their composite is recorded.
What is compared
For each race in the universe, two published estimates of P(Republican wins): the Kalshi market price and the Scopes Poll Composite. Both are read from the last snapshot strictly before that race's poll-close time, one observation per race. Nothing is recomputed or interpolated; the value the record holds is used.
Metric
The mean Brier of each estimate against the 0/1 result (Republican win), and each estimate's hit-rate (the share of races where its favorite, above 0.5, won). The headline is the paired difference in mean Brier (composite minus market) with a bootstrap band. Lower Brier and higher hit-rate are better.
Power (stated in advance)
This is a small number of races by construction, so no reliability curve is claimed and the comparison is low-powered. The value is a pre-committed, honest record of how the market and the polls did on the hardest races, not a strong verdict.
Resolution
The result of each race comes from the polls desk's curated resolution (Republican win recorded as the outcome), the same source the Senate control study uses. Computed after the races are called.
Verdict rule (applied mechanically after resolution)
Chosen only by where the paired-Brier confidence band falls. A band entirely below zero reads the market was the better estimate; a band entirely above zero reads the composite was the better estimate; a band that straddles zero reads inconclusive (low power). No discretionary text is added.
The selection rule
Both the estimates and the outcome come from the append-only record. The market side is price_snapshots for each race's Kalshi contract; the composite is the Scopes Poll Composite recorded for the race; the outcome is set by the curated poll resolver. For each race the scored values are the last rows strictly before poll close. The scoring code and its methodology version are published with the results.
What this can't test
- It is small. A handful of coin-flip races cannot support a calibration curve or a high-confidence verdict, so a straddling band is the expected, honest outcome.
- Brier and hit-rate measure whether the estimates were right, not why a price or a poll average was where it was.
- It is a comparison of two published estimates, not a forecast by Scopes. The Scopes Poll Composite is a rules-based average, not an opinion, and the market price is the venue's, not ours.
Results here are computed after the races resolve. Until then this page is the plan, timestamped before any of these outcomes were known.