Research · pre-registered study
Frontier models vs the market: 2026 Senate control
A pre-registered contest. Before Election Day, every frontier model in our roster forecasts the winner of each contested 2026 U.S. Senate race, plus the one question that decides everything: who controls the Senate. Each model gets our own Scopes Poll Composite as context. The forecasts are locked before the results are in, then scored by a proper scoring rule against the outcomes and against the market.
Scopes is the scorekeeper, not a forecaster. The models forecast; the market forecasts; we run the contest, lock the predictions, and keep the append-only record. The prompt and the exact context shown to each model are published below so any score here can be reproduced.
What we registered in advance
- Across the contested races, does any model's mean Brier score beat the market's (Kalshi per-race implied)?
- Pick record: of the races, how many did each model call correctly vs the market?
- On the single question that matters most, Senate control, does any model beat the Kalshi control market?
- Does our poll composite let a model match a market that already prices the polls?
Registered expectation: with roughly eight contested seats plus one control question, this is underpowered, so we treat any separation as directional, not conclusive. Control is a single outcome the whole field forecasts. The market prices the polls efficiently; a model failing to beat it is a legitimate, reportable result.
The contest
Standing (55 of 62 resolved)
| Competitor | Mean Brier | Log loss | Picks |
|---|---|---|---|
| market (polymarket)benchmark | 0.090 | 0.315 | 7/8 |
| market (kalshi)benchmark | 0.092 | 0.325 | 7/8 |
| deepseek deepseek-v4-pro-0813 | 0.146 | 0.444 | 5/7 |
| google gemini-3.8-flash | 0.155 | 0.479 | 6/8 |
| xai grok-4.6 | 0.166 | 0.497 | 5/8 |
| openai gpt-6-astra | 0.176 | 0.511 | 5/8 |
| anthropic claude-opus-5 | 0.268 | 0.734 | 4/8 |
Lower Brier and log loss are better. Underpowered at this sample; read it as directional.
| market (kalshi)market | P(Republican) 29.2% | Brier 0.085 |
| market (polymarket)market | P(Republican) 28.2% | Brier 0.080 |
| openai gpt-6-astra | P(Republican) 56.0% | Brier 0.314 |
| anthropic claude-opus-5 | P(Republican) 76.0% | Brier 0.578 |
| google gemini-3.8-flash | P(Republican) 44.6% | Brier 0.199 |
| xai grok-4.6 | P(Republican) 50.0% | Brier 0.250 |
| deepseek deepseek-v4-pro-0813 | No forecast: no valid answer at lock | |
| market (kalshi)market | P(Republican) 29.5% | |
| market (polymarket)market | P(Republican) 27.2% | |
| openai gpt-6-astra | P(Republican) 68.4% | |
| xai grok-4.6 | P(Republican) 58.0% | |
| google gemini-3.8-flash | P(Republican) 58.8% | |
| anthropic claude-opus-5 | P(Republican) 69.0% | |
| deepseek deepseek-v4-pro-0813 | P(Republican) 60.4% |
| market (kalshi)market | P(Republican) 3.4% | Brier 0.001 |
| market (polymarket)market | P(Republican) 2.8% | Brier 0.001 |
| openai gpt-6-astra | P(Republican) 12.2% | Brier 0.015 |
| anthropic claude-opus-5 | P(Republican) 17.5% | Brier 0.031 |
| google gemini-3.8-flash | P(Republican) 19.6% | Brier 0.038 |
| xai grok-4.6 | P(Republican) 15.3% | Brier 0.024 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 16.0% | Brier 0.026 |
| market (kalshi)market | P(Republican) 56.1% | Brier 0.314 |
| market (polymarket)market | P(Republican) 58.5% | Brier 0.342 |
| openai gpt-6-astra | P(Republican) 58.4% | Brier 0.341 |
| anthropic claude-opus-5 | P(Republican) 64.2% | Brier 0.412 |
| google gemini-3.8-flash | P(Republican) 56.2% | Brier 0.316 |
| xai grok-4.6 | P(Republican) 56.0% | Brier 0.314 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 60.0% | Brier 0.360 |
| market (kalshi)market | P(Republican) 31.5% | Brier 0.099 |
| market (polymarket)market | P(Republican) 32.5% | Brier 0.106 |
| openai gpt-6-astra | P(Republican) 24.0% | Brier 0.058 |
| xai grok-4.6 | P(Republican) 24.7% | Brier 0.061 |
| anthropic claude-opus-5 | P(Republican) 40.2% | Brier 0.162 |
| google gemini-3.8-flash | P(Republican) 20.4% | Brier 0.042 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 20.9% | Brier 0.044 |
| market (kalshi)market | P(Republican) 23.5% | Brier 0.055 |
| market (polymarket)market | P(Republican) 18.5% | Brier 0.034 |
| openai gpt-6-astra | P(Republican) 18.6% | Brier 0.035 |
| xai grok-4.6 | P(Republican) 17.7% | Brier 0.031 |
| anthropic claude-opus-5 | P(Republican) 23.4% | Brier 0.055 |
| google gemini-3.8-flash | P(Republican) 18.2% | Brier 0.033 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 15.6% | Brier 0.024 |
| market (kalshi)market | P(Republican) 3.2% | Brier 0.001 |
| market (polymarket)market | P(Republican) 2.3% | Brier 0.001 |
| anthropic claude-opus-5 | P(Republican) 17.7% | Brier 0.031 |
| openai gpt-6-astra | P(Republican) 13.0% | Brier 0.017 |
| google gemini-3.8-flash | P(Republican) 25.6% | Brier 0.066 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 10.6% | Brier 0.011 |
| xai grok-4.6 | P(Republican) 16.0% | Brier 0.026 |
| market (kalshi)market | P(Republican) 29.2% | Brier 0.085 |
| market (polymarket)market | P(Republican) 26.5% | Brier 0.070 |
| openai gpt-6-astra | P(Republican) 48.6% | Brier 0.236 |
| xai grok-4.6 | P(Republican) 44.6% | Brier 0.199 |
| google gemini-3.8-flash | P(Republican) 48.4% | Brier 0.234 |
| anthropic claude-opus-5 | P(Republican) 51.4% | Brier 0.264 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 43.0% | Brier 0.185 |
| market (kalshi)market | P(Republican) 31.2% | Brier 0.097 |
| market (polymarket)market | P(Republican) 29.5% | Brier 0.087 |
| openai gpt-6-astra | P(Republican) 62.4% | Brier 0.389 |
| xai grok-4.6 | P(Republican) 65.0% | Brier 0.422 |
| google gemini-3.8-flash | P(Republican) 55.6% | Brier 0.309 |
| anthropic claude-opus-5 | P(Republican) 78.4% | Brier 0.615 |
| deepseek deepseek-v4-pro-0813 | P(Republican) 60.8% | Brier 0.370 |
How the field was locked
The field locked on Sunday, November 1, 2026, in several passes. The questions, the prompt and the context did not change between passes; the generation settings did, as set out below. Every locked row stores its own lock time, the composite value it was shown, and its raw samples.
- Market benchmark: 7:42 AM ET. The first lock that morning captured the market but returned no model forecasts, because our model gateway was unavailable. Market rows are insert-only, so they keep the 7:42 AM capture and were not taken again.
- Model forecasts: 10:30 to 11:26 AM ET. The models do not browse, and their only live input, the Poll Composite, was fixed at 7:34 AM (no poll ingest ran again until 1 PM). The later model lock therefore gave them no information the market lacked.
- Settings changed between passes. Before 11:05 AM, replies were capped at 300 output tokens and 45 seconds per call, and a reply had to be bare JSON. From 11:05 AM the cap was 2,000 tokens and the parser accepted the JSON object with text around it. From 11:15 AM it was 8,000 tokens and 120 seconds per call, later 260. The changes were made so replies could finish, but a larger budget also lets a reasoning model think longer. Sixteen forecasts locked before 11:05 AM (all nine of OpenAI's, five of Grok's, two of Claude's), seventeen between 11:05 and 11:08 AM (all nine of Gemini's, seven of Claude's, one of Grok's), and eleven after 11:15 AM (eight of DeepSeek's, three of Grok's). No forecast was re-sampled once locked.
- Samples. Each model was asked five times per question; the forecast is the mean of the valid answers, and at least three were required. Some forecasts rest on three or four. The count is stored with each row.
- One empty cell: DeepSeek V4 Pro on Senate control. Across three passes every sample either ran past the time limit or spent its whole 8,000-token budget reasoning without answering. We did not cap its reasoning to force an answer, because that would have run one model under settings no other model had. Control is scored over the other four models and the market.
The prompt, put to every model
The identical prompt, with each race's context filled in. The model is sampled five times; the mean of its valid answers is the locked forecast. It never sees the market price: the market is the benchmark, not an input. The seats use the per-race prompt; the control question uses the control prompt.
You are forecasting a 2026 U.S. Senate race that has not yet been decided. Give your best estimate of the probability that the REPUBLICAN candidate wins the seat.
Race: {race}
Scopes Poll Composite (our poll-based estimate of P(Republican wins)): {composite}
{fundamentals}
Use what you know about this race together with the poll signal above. Answer with a single JSON object and nothing else:
{"p_republican_wins": 0.0-1.0, "confidence": 0.0-1.0, "reason": "<=240 chars"}You are forecasting which party will control the U.S. Senate after the 2026 elections. Give your best estimate of the probability that the REPUBLICANS control the Senate.
Scopes Poll Composite (our poll-based estimate of P(Republicans control)): {composite}
{fundamentals}
Use what you know about the 2026 Senate map together with the poll signal above. Answer with a single JSON object and nothing else:
{"p_republican_wins": 0.0-1.0, "confidence": 0.0-1.0, "reason": "<=240 chars"}Context is our own Scopes Poll Composite (a scopes-derived value), not a third-party aggregator. The market benchmark is the Kalshi (and, where shown, Polymarket) implied probability from our own poll data, captured at lock.
Cite this study
Free to cite with attribution to Scopes. For the citation format, boilerplate, and media contact, see Press & citation.