Scopes

Research · the read-off · standing record

Which model reads the event best

For every open event on the scorecard there are two independent estimates of the same probability: the price implied by a prediction market, and an independent fair value. We ask each frontier model the identical question, before the event resolves: which of the two is better calibrated? Once the event settles, the answer is known, and we record which estimate each model chose and whether it matched the one that proved closer. This page is the running tally.

The models are not forecasting the outcome and are not advising a wager. They are grading two estimators. The prompt, the inputs, and the raw outputs are published below so any score here can be reproduced.

The standing record

The standing record scores only the numbers-only reads: the identical prompt every model sees on every desk, with nothing but the two probabilities. It is the one cohort where cross-model accuracy is comparable, because the prompt is held constant. Reads run with an added reference block are a separate experiment, scored below and never mixed in here.

Accuracy is the share of a model's decided reads where the estimate it chose proved better calibrated (closer to the resolved outcome). Reads where the two estimates were equidistant are a Push and are excluded. A model needs at least 20 decided reads before it carries an accuracy figure; below that it shows its counts only.

Models differ sharply in how often they abstain, so a low decided count is often about caution, not skill. On a numbers-only read, given just the two probabilities, some models judge they have no basis to prefer either estimate and answer indeterminate on most events; others reason from what they know about the event and commit. Abstaining is a legitimate answer, not a wrong one: a model that declines when it has no basis is not penalized, but it also accrues few decided reads and may carry no accuracy figure here. Whether committing more often means reading better is one of the questions this record exists to answer.

ModelMatched better est.DecidedBy choice
anthropic claude-opus-570.0% (42/60)60market 40/51 · fair 2/9
google gemini-3.8-flash52.9% (18/34)34market 17/22 · fair 1/12
xai grok-4.6early8market 1/2 · fair 2/6
openai gpt-6-astraearly4fair 1/4
deepseek deepseek-v4-pro-0813early1fair 1/1

Does context make a model commit?

A separate experiment, not part of the standing record above. On the desks whose fair-value inputs are rights-clean and public domain (weather, catastrophe, rates), the prompt also carries a short reference block: the forecast, climatology, or macro figure the fair value was built from, published verbatim with each read. This panel compares each model's numbers-only reads to its enriched reads on those desks. The first thing context moves is the commit rate: how often a model answers at all instead of abstaining.

ModelCommit rate: numbers-only → enrichedAccuracy (n.o. / enriched)
xai grok-4.65.9% (9/153) → 100.0% (76/76)37.5% (3/8) / 58.9% (43/73)
google gemini-3.8-flash54.3% (38/70) → 100.0% (46/46)52.9% (18/34) / 72.1% (31/43)
anthropic claude-opus-566.7% (62/93) → 100.0% (70/70)70.0% (42/60) / 64.7% (44/68)
deepseek deepseek-v4-pro-081366.7% (2/3) → 100.0% (14/14)100.0% (1/1) / 83.3% (10/12)
openai gpt-6-astra3.3% (5/153) → 65.9% (56/85)25.0% (1/4) / 64.2% (34/53)

Read commit rate first. Accuracy here is on small samples and is not a rating; it is shown only to check that committing more is not the same as reading worse. The enriched desks are the public-domain ones, not the numeric desks (energy, stocks) where the most cautious models abstain most, so this does not touch those models' standing figures.

The question, put to every model

The identical prompt is sent to every model, with the event, the outcome, and the two probabilities filled into the placeholders. Each model is sampled 5 times at temperature 0.3; the majority read is recorded. A model may answer indeterminate when the two numbers give it no basis to prefer either; those abstentions are recorded for transparency but never scored.

The {reference} line is empty on numbers-only reads. On the public-domain desks it carries the reference block described above, and its exact text is published with each enriched read in the list below.

You are grading two independent probability estimates for a FUTURE event that has not yet occurred. You are NOT predicting the outcome and NOT advising a bet.

Event: {event}
Outcome being judged: {outcome}
Two estimates of the probability of this outcome are on record:
  - MARKET (prediction-market implied): {market_pct}
  - FAIR VALUE (an independent benchmark: {context}): {fair_pct}
{reference}
Question: which estimate is better calibrated, that is, closer to the true probability of the outcome, the MARKET or the FAIR VALUE? Use whatever you know about this event to judge. If you genuinely have no basis to prefer either, answer "indeterminate" rather than guessing.

Answer with a single JSON object and nothing else:
{"read": "market" | "fair" | "indeterminate", "confidence": 0.0-1.0, "reason": "<=200 chars"}

How a read is scored

Recent reads, with raw outputs

The most recent reads on record, each with the raw samples the model returned. Expand any row to reproduce the scoring by hand.

WTI ≥ $86.49 · xai abstainedNov 14, 3:01 AM EST
market 76.0% · fair 63.2%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No date, current WTI level, or horizon given; cannot assess which of 76% vs 63.2% is closer to true prob.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, current WTI level, or vol given; cannot judge which of 76% vs 63% is closer to true prob.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No timeframe, current WTI level, or other event details given, so no basis to judge which probability is better calibrated.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No current WTI price, expiry, or vol data given; cannot assess which of 76% vs 63% is closer to true prob.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No date/horizon or current WTI level given; cannot assess vol, mean-reversion or geopolitics to prefer 76% vs 63%.",
    "confidence": 0.85
  }
]
WTI ≥ $86.49 · openai abstainedNov 14, 3:01 AM EST
market 76.0% · fair 63.2%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No event horizon, current price context, or validation evidence for either estimate is provided. The two probabilities alone give no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event deadline, current WTI price, or benchmark methodology is provided. Neither estimate's closeness to the true probability can be established from these numbers alone.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No evaluation date, current WTI price, benchmark methodology, or forecasting track records are provided. The two probabilities alone give no basis to prefer either.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event horizon, current WTI price, or calibration records for either source are provided. The two probabilities alone offer no basis to prefer one.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event deadline, current WTI price, benchmark methodology, or calibration histories are provided. The two probabilities alone give no basis to prefer either.",
    "confidence": 0.99
  }
]
WTI ≥ $86.49 · google abstainedNov 14, 3:01 AM EST
market 76.0% · fair 63.2%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "Lacking the contract expiration date, current spot price, and modeling methodology makes it impossible to reliably determine which probability is better calibrated.",
    "confidence": 0.85
  }
]
Chicago 41–42° · openai abstainedNov 13, 9:03 PM EST
market 60.5% · fair 39.5%· prompt v2-weather
Context shown (public domain)
{
  "city": "Chicago",
  "note": "reference leans toward the fair value, which is derived from the same forecast",
  "source": "NWS api.weather.gov daily-high forecast (US-gov public domain)",
  "market_band_f": "41-42°F",
  "nws_forecast_high_f": 42
}
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "A 42°F point forecast does not establish the probability of a 41–42°F high. Without lead time or forecast-error data, there is insufficient basis to prefer 60.5% over 39.5%, or vice versa.",
    "confidence": 0.9
  },
  {
    "read": "indeterminate",
    "reason": "A 42°F point forecast doesn't determine the probability of 41–42°F. No lead time or forecast-error spread is provided to reliably distinguish 39.5% from 60.5%.",
    "confidence": 0.9
  },
  {
    "read": "fair",
    "reason": "A 2°F band is narrow relative to typical daily-high forecast uncertainty, making 39.5% more plausible than 60.5%. The unspecified lead time limits confidence.",
    "confidence": 0.7
  },
  {
    "read": "indeterminate",
    "reason": "A 42°F point forecast does not establish the probability of a 41–42°F high. The forecast lead time and error distribution are missing, so neither estimate is clearly better supported.",
    "confidence": 0.9
  }
]
WTI ≥ $85.99 · anthropic abstainedNov 13, 9:02 PM EST
market 68.0% · fair 58.5%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No timeframe or current WTI level given; cannot judge whether 68% market or 58.5% model is closer. Markets often better calibrated, but gap is plausible either way.",
    "confidence": 0.3
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, spot level, or vol given; cannot judge whether 68% market or 58.5% statistical FV is closer to the true probability of WTI >= $85.99.",
    "confidence": 0.35
  },
  {
    "read": "market",
    "reason": "Prediction markets aggregate live oil-price information and typically outperform a static statistical benchmark on short-horizon commodity thresholds; no specific evidence the 68% is biased.",
    "confidence": 0.6
  }
]
WTI ≥ $85.99 · xai abstainedNov 13, 3:04 PM EST
market 68.0% · fair 58.5%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No horizon, current WTI level, or vol given; oil geopolitics/OPEC make 68 vs 58.5 equally plausible.",
    "confidence": 0.75
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, current spot, or vol given so cannot form an independent true-prob estimate to compare against either number.",
    "confidence": 0.4
  },
  {
    "read": "indeterminate",
    "reason": "No timeframe, current spot, or vol given for WTI>=85.99; cannot judge market vs statistical model without those.",
    "confidence": 0.55
  },
  {
    "read": "indeterminate",
    "reason": "No expiry, current WTI level or vol given; cannot judge if 85.99 is in/out of money enough to prefer 68% vs 58.5%.",
    "confidence": 0.4
  },
  {
    "read": "indeterminate",
    "reason": "No date, horizon, or current WTI level given; cannot judge calibration of either estimate.",
    "confidence": 0.75
  }
]
WTI ≥ $85.99 · openai abstainedNov 13, 3:03 PM EST
market 68.0% · fair 58.5%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No event horizon, current WTI price, benchmark methodology, or calibration history is provided. The two probability estimates alone give no basis to prefer either.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event deadline, current WTI price, model methodology, or calibration history is provided. The two probabilities alone give no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No observation date, time horizon, current WTI price, or model validation is provided. The two probabilities alone give no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event deadline, price context, or validated forecasting records are provided. The two estimates alone cannot establish which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event date, settlement definition, or evidence validating either estimate is provided. The two probabilities alone cannot establish which is closer to the true probability.",
    "confidence": 0.99
  }
]
Chicago 39–40° · xai read Market · Matched the better estimateNov 13, 3:02 PM EST
market 16.5% · fair 29.1%· prompt v2-weather
Context shown (public domain)
{
  "city": "Chicago",
  "note": "reference leans toward the fair value, which is derived from the same forecast",
  "source": "NWS api.weather.gov daily-high forecast (US-gov public domain)",
  "market_band_f": "39-40°F",
  "nws_forecast_high_f": 42
}
Raw samples
[
  {
    "read": "market",
    "reason": "NWS 42°F; 2°F band 2-3° below has ~15-20% prob at typical ~3°F RMSE (approx. normal). Market 16.5% matches better than 29%.",
    "confidence": 0.68
  },
  {
    "read": "market",
    "reason": "NWS 42°F + typical 2-4°F RMSE (normal errors) yields ~15-20% for the 2°F bin 2-3° below; 16.5% nearer than 29%.",
    "confidence": 0.72
  },
  {
    "read": "market",
    "reason": "NWS 42°F + typical ~3°F error SD (normal) yields ~16-19% for a 2° bin 2-3° below; 16.5% matches, 29% is too high.",
    "confidence": 0.75
  }
]
Chicago 39–40° · openai read Market · Matched the better estimateNov 13, 3:02 PM EST
market 16.5% · fair 29.1%· prompt v2-weather
Context shown (public domain)
{
  "city": "Chicago",
  "note": "reference leans toward the fair value, which is derived from the same forecast",
  "source": "NWS api.weather.gov daily-high forecast (US-gov public domain)",
  "market_band_f": "39-40°F",
  "nws_forecast_high_f": 42
}
Raw samples
[
  {
    "read": "market",
    "reason": "A narrow 39–40°F band below a 42°F forecast generally supports 16.5% more than 29.1% under typical forecast errors. Missing lead time and station details limit confidence.",
    "confidence": 0.65
  },
  {
    "read": "market",
    "reason": "With ordinary uncertainty around a 42°F forecast, the narrow 39–40°F band is more plausibly near 16.5% than 29.1%. Missing lead time and station details limit confidence.",
    "confidence": 0.7
  },
  {
    "read": "market",
    "reason": "The band is 2–3°F below the forecast. Plausible forecast-error spreads support a mid-teens probability more readily than 29.1%, though lead time and station details are missing.",
    "confidence": 0.68
  }
]
WTI ≥ $84.99 · google abstainedNov 13, 9:02 AM EST
market 71.0% · fair 64.7%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "Lacking the specific time horizon and current spot dynamics, there is no sound basis to determine whether market sentiment or the statistical benchmark is better calibrated.",
    "confidence": 0.8
  }
]
Chicago 46–47° · xai read Fair value · Didn'tNov 13, 9:02 AM EST
market 58.0% · fair 39.5%· prompt v2-weather
Context shown (public domain)
{
  "city": "Chicago",
  "note": "reference leans toward the fair value, which is derived from the same forecast",
  "source": "NWS api.weather.gov daily-high forecast (US-gov public domain)",
  "market_band_f": "46-47°F",
  "nws_forecast_high_f": 44
}
Raw samples
[
  {
    "read": "fair",
    "reason": "NWS 44F; 46-47 is +2-3F. Typical NWS high-error SD ~2.5-3.5F yields ~15-25% in that 2F bin, so 39.5% far closer than 58%.",
    "confidence": 0.7
  }
]
WTI ≥ $84.49 · google abstainedNov 13, 9:00 AM EST
market 71.5% · fair 61.7%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "Without contract expiration, current spot price, or statistical model methodology, there is no objective basis to favor either estimate.",
    "confidence": 0.85
  }
]
WTI ≥ $85.49 · xai abstainedNov 13, 3:04 AM EST
market 74.5% · fair 63.4%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No horizon, current WTI level, or vol given; 11pp gap could be either mispricing or model error, so no basis to prefer.",
    "confidence": 0.55
  },
  {
    "read": "indeterminate",
    "reason": "No contract date, current WTI level, or other specifics given; cannot assess which estimate is closer to true prob.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No event horizon, current spot, or model details given; cannot judge calibration of either estimate vs unknown true prob.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No timeframe, current WTI level, or contract details given; cannot judge which of the two estimates is better calibrated.",
    "confidence": 0.75
  },
  {
    "read": "indeterminate",
    "reason": "No date, horizon, or current WTI level given; cannot assess which of the two unanchored probs is closer to truth.",
    "confidence": 0.55
  }
]
WTI ≥ $85.49 · openai abstainedNov 13, 3:04 AM EST
market 74.5% · fair 63.4%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No event date, settlement rules, current WTI context, or calibration records are provided. The two probability estimates alone give no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event horizon, settlement details, or calibration records are provided. The two estimates alone give no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, settlement rules, current WTI price, or evidence of either estimator's reliability is provided. The two probabilities alone cannot establish which is closer to truth.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No settlement date, current WTI price, benchmark methodology, or calibration history is provided. The two probabilities alone give no basis to prefer either estimate.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event horizon, current price context, or historical calibration evidence is provided to justify preferring either probability estimate.",
    "confidence": 0.99
  }
]
WTI ≥ $85.49 · anthropic abstainedNov 13, 3:04 AM EST
market 74.5% · fair 63.4%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No horizon, spot level, or vol given; cannot judge whether 74.5% market or 63.4% statistical fair value is closer. Markets usually well-calibrated, but gap is plausible either way.",
    "confidence": 0.3
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, spot level, or vol given; cannot judge whether 74.5% market or 63.4% statistical fair value is closer. Markets usually better informed, but gap is plausible either way.",
    "confidence": 0.3
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, spot level, or vol given; cannot tell whether market's 74.5% or the statistical 63.4% is closer to the true probability for WTI >= $85.49.",
    "confidence": 0.35
  },
  {
    "read": "indeterminate",
    "reason": "No horizon, spot level, or vol given; cannot judge whether 74.5% market or 63.4% statistical fair value is closer. Markets often better informed, but gap alone isn't decisive.",
    "confidence": 0.3
  }
]
WTI ≥ $85.49 · google abstainedNov 13, 3:04 AM EST
market 74.5% · fair 63.4%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "Lacking the specific time horizon, expiration date, and underlying model parameters makes it impossible to determine which estimate is better calibrated.",
    "confidence": 0.9
  }
]
Chicago ≤ null° · xai abstainedNov 13, 3:03 AM EST
market 2.0% · fair 10.2%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "Event string 'Chicago ≤ null°' is undefined/malformed; no identifiable future event or data to compare 2% vs 10.2%.",
    "confidence": 0.95
  },
  {
    "read": "indeterminate",
    "reason": "Event 'Chicago ≤ null°' is undefined/nonsensical; no basis to judge either estimate.",
    "confidence": 0.95
  },
  {
    "read": "indeterminate",
    "reason": "Event 'Chicago ≤ null°' is undefined/nonsensical; no basis to assess either probability.",
    "confidence": 0.95
  },
  {
    "read": "indeterminate",
    "reason": "Event string 'Chicago ≤ null°' is incomplete/undefined; no identifiable future event or data to compare the two probs.",
    "confidence": 0.95
  },
  {
    "read": "indeterminate",
    "reason": "Event string 'Chicago ≤ null°' is malformed/uninterpretable; no identifiable future event or data to assess calibration of either estimate.",
    "confidence": 0.95
  }
]
Chicago ≤ null° · google abstainedNov 13, 3:03 AM EST
market 2.0% · fair 10.2%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "The event threshold is corrupted/missing ('null°'), making the true probability completely undefined and impossible to evaluate.",
    "confidence": 1
  },
  {
    "read": "indeterminate",
    "reason": "The event specification contains a null temperature threshold, making the event undefined and impossible to evaluate.",
    "confidence": 1
  },
  {
    "read": "indeterminate",
    "reason": "The event specification is corrupted with a null threshold temperature, making the outcome impossible to evaluate.",
    "confidence": 1
  },
  {
    "read": "indeterminate",
    "reason": "The event specification is malformed with a threshold of 'null°', making the outcome undefined and impossible to evaluate.",
    "confidence": 1
  },
  {
    "read": "indeterminate",
    "reason": "The event contains a missing or malformed threshold ('null°'), making it impossible to evaluate the true probability.",
    "confidence": 1
  }
]
Chicago ≤ null° · anthropic read Market · Matched the better estimateNov 13, 3:03 AM EST
market 2.0% · fair 10.2%
Raw samples
[
  {
    "read": "market",
    "reason": "Event threshold is null/unspecified, so no substantive basis; default to market as generally better calibrated than a naive statistical benchmark on temperature thresholds.",
    "confidence": 0.55
  },
  {
    "read": "market",
    "reason": "Threshold is missing (null), so no substantive basis; default to market, which aggregates live weather info and is typically better calibrated than a generic statistical benchmark.",
    "confidence": 0.55
  },
  {
    "read": "market",
    "reason": "Threshold is missing, so no event-specific judgment is possible; defaulting to the generally better-calibrated prediction-market price, especially given longshot bias at low probabilities.",
    "confidence": 0.55
  },
  {
    "read": "market",
    "reason": "Event threshold is null/unspecified; with no resolvable detail, defer to market pricing, which aggregates real info and is typically better calibrated than a generic statistical benchmark.",
    "confidence": 0.55
  },
  {
    "read": "market",
    "reason": "Event threshold is null/unspecified, so no substantive basis; prediction markets on weather thresholds are typically well-calibrated, while the model's 5x higher estimate likely reflects misspecificat",
    "confidence": 0.55
  }
]
Chicago ≤ null° · openai abstainedNov 13, 3:03 AM EST
market 2.0% · fair 10.2%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "The outcome has an undefined threshold ('null°') and no date or measurement details, so there is no basis to assess its true probability or prefer either estimate.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "The event has an undefined threshold ('null°') and no date or measurement details. There is no basis to assess its true probability or prefer either estimate.",
    "confidence": 1
  },
  {
    "read": "indeterminate",
    "reason": "The event has a null temperature threshold and no date or measurement definition. There is no basis to assess its true probability or prefer either estimate.",
    "confidence": 1
  },
  {
    "read": "indeterminate",
    "reason": "The outcome has an undefined threshold (null°) and no date or measurement details. There is no basis to judge whether 2.0% or 10.2% is closer to its true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "The event has an undefined threshold (null°) and no date or temperature metric. There is no basis to determine its true probability or prefer either estimate.",
    "confidence": 1
  }
]
WTI ≥ $84.99 · xai abstainedNov 13, 3:02 AM EST
market 71.0% · fair 64.7%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No resolution date, current WTI level, or other event specifics given; cannot judge which of two close estimates is better calibrated.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No timeframe, current WTI level, vol, or model/market details given; cannot judge which 6pp gap is better calibrated.",
    "confidence": 0.55
  },
  {
    "read": "indeterminate",
    "reason": "No event date, current WTI level, vol, or model details given; cannot judge which of two close estimates is better calibrated.",
    "confidence": 0.55
  },
  {
    "read": "indeterminate",
    "reason": "No timeframe, current WTI level, or other event details given; cannot judge which of two close estimates is better calibrated.",
    "confidence": 0.55
  },
  {
    "read": "indeterminate",
    "reason": "No current WTI price, event horizon, or market details given; cannot judge which estimate is closer to true prob.",
    "confidence": 0.85
  }
]
WTI ≥ $84.99 · anthropic abstainedNov 13, 3:02 AM EST
market 71.0% · fair 64.7%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No timeframe or current WTI level given; 6pp gap is small and both plausible. Markets usually well-calibrated but lack context to judge against statistical fair value.",
    "confidence": 0.3
  },
  {
    "read": "indeterminate",
    "reason": "No date, horizon, or current WTI level given; 71% vs 64.7% gap is small and I lack context to judge which is closer to truth.",
    "confidence": 0.3
  },
  {
    "read": "market",
    "reason": "Prediction markets on oil thresholds typically aggregate futures/options info efficiently; absent horizon details, market edge over a generic statistical fair value is the default.",
    "confidence": 0.55
  }
]
WTI ≥ $84.99 · openai abstainedNov 13, 3:02 AM EST
market 71.0% · fair 64.7%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No settlement date, current WTI price, or benchmark methodology is provided. The two probabilities alone offer no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event date, settlement details, or evidence validating either estimator is provided. The two probabilities alone offer no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event horizon, settlement criteria, or validation evidence is provided. The two quoted probabilities alone give no basis to judge which is closer to the true probability.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event horizon, relevant market context, or validation of either estimator is provided, so there is no defensible basis to prefer 71.0% over 64.7%, or vice versa.",
    "confidence": 0.99
  },
  {
    "read": "indeterminate",
    "reason": "No event deadline, settlement definition, current market context, or benchmark validation is provided. Neither estimate can be judged closer to the true probability from these numbers alone.",
    "confidence": 0.99
  }
]
SPX ≥ 7699.9999 · xai abstainedNov 13, 3:02 AM EST
market 13.5% · fair 24.4%
Raw samples
[
  {
    "read": "indeterminate",
    "reason": "No date/horizon given for SPX>=7700; without tenor or current level context neither 13.5% nor 24.4% can be judged closer to truth.",
    "confidence": 0.55
  },
  {
    "read": "indeterminate",
    "reason": "No date, current SPX level, or horizon given; cannot assess either estimate vs true prob of reaching 7700.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No timeframe, current SPX level, or model details given; cannot assess which of 13.5% vs 24.4% is closer to true prob.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No time horizon, current SPX level, or other contract details given; cannot judge calibration of either estimate.",
    "confidence": 0.85
  },
  {
    "read": "indeterminate",
    "reason": "No current SPX level, horizon, or vol given; cannot independently assess either estimate vs true real-world prob.",
    "confidence": 0.65
  }
]

The estimates being graded are the same ones on the scorecard. This record is separate from the scorecard's own market-vs-fair tally; here the subject is the models, not the estimators.

Next milestone: holding the prompt constant is the first step, but even within one cohort the models are read on different events, since they abstain on different ones. The cleanest comparison is a head-to-head on the same events, scored the same way. That shared-event view is the next step for this record.