Research · pre-registered study

Are prediction markets calibrated on bitcoin? Pre-registered

The Scopes Desk

Crypto event markets are high-volume, retail-heavy and resolve cleanly against a price at a timestamp, which makes them a strong test of one question: do the daily bitcoin threshold prices match how often those outcomes actually occur? This plan pre-registers that test. It is a reliability study only, the market probability scored against the realized settlement, with the universe, the reference time, the bins, the sample gate and the exclusions fixed here before the markets resolve.

This is deliberately narrow. It reads one venue (Kalshi), it carries no Scopes Fair Value and no second forecaster, and it makes no cross-venue divergence claim. It asks whether the market, on its own, is well-calibrated, nothing more.

The specification

Universe
The Kalshi daily bitcoin above/below threshold series (KXBTC). The end-of-year range ladders (KXBTCY, KXETHY) are collected but excluded from this scored curve: they resolve about once a year, so they contribute a negligible, long-horizon sample that would muddy a short-horizon reliability curve. Settlement for each market is the Kalshi settlement for that contract.

Venue
Kalshi only. Bitcoin is not collected from Polymarket in the current ingest, and Gemini market data is excluded from any benchmark or index use under its data agreement. A single venue means this is a calibration study, not a cross-venue comparison.

Probability used (pre-registered rule)
For each market, the as-published market probability from the last snapshot strictly before 60 minutes to settlement. At that lead the outcome is not yet revealed, so the reading is comparison-valid (pregame-equivalent); nothing is recomputed or interpolated, and the value the record holds is used. Readings taken in play or after settlement are excluded and archived, never scored.

Score
The mean Brier of the market probability against the 0/1 settlement, and a reliability curve over 10 equal-width bins (observed YES-frequency vs mean forecast per bin). Because multiple strikes can settle off the same daily close, the unit of independence is the settlement day: confidence bands are a cluster bootstrap by day, and the independent sample is reported in days alongside the market count.

Sample gate
The desk headline reads "insufficient power, no verdict" until 100 resolved markets. A per-bin observed frequency renders only once that bin holds 20 pooled observations; below that it reads "accumulating". The daily markets are expected to clear these gates quickly.

Output
A reliability table (forecast probability vs observed frequency per bin, each with a Wilson 95% band) and a mean Brier with its cluster-bootstrap band, rendered server-side with the numbers in the HTML. The scoring code and its methodology version are published with the results.

Verdict rule (applied mechanically after the gate)

Chosen only by where the calibration-error confidence band falls relative to the pre-registered band. Expected calibration error at or below 0.05 reads well calibrated; a band entirely above it reads miscalibrated; a band straddling it reads inconclusive. No discretionary text is added.

The selection rule

Both the market series and the outcome come from the append-only record. The market side is price_snapshots (per crypto contract); the outcome is set from the Kalshi settlement by the resolver. For each market the scored value is the last row strictly before the reference lead.

select implied_prob, captured_at
from price_snapshots
where contract_id = :crypto_contract_id
  and captured_at < :settles_at - interval '60 minutes'
order by captured_at desc
limit 1;

Results here are computed after the markets settle. Until the sample gate is reached the desk reads "insufficient power, no verdict" by rule.