Methodology

A displayed 70% really hits ≈ 70%

Left alone, the raw model is overconfident — it says 80% when the real rate is closer to 68%. We learn a per-family monotonic isotonic transform (predicted probability → realized rate) from the settled ledger: prediction-bucketed, Bayesian-shrunk, pooled by adjacent-violators, with family-average and identity fallbacks for thin samples.

It generalizes: in a one-off 70/30 held-out validation (fit on 70% of the ledger, tested on the unseen 30%, run 2026-06), ECE still collapsed 12.6% → 5.4%. A regression test fails the build if the transform ever stops improving ECE, stops being monotonic, or stops being identity-safe.

The model finds edges; curation throws out the bad ones

A market family is proven-negative once it has lost money over a real sample (n ≥ 25, net ≤ −5u). Those families are hard-excluded from every board we surface. This is the single highest-impact lever in the whole system — and it's not modeling, it's discipline.

We publish both numbers — the raw firehose and the curated book — on the Strategy Simulator, where you can replay the entire ledger yourself with any filter and reproduce the track record exactly.

And it survives out-of-sample — held-out validation →

How the daily flagship pick is chosen — the exact chain

Every candidate on the slate (MLB player props + MLB moneylines) runs the same gauntlet, in order:

1. Its market family must not be proven-negative (lost ≥ 5u over n ≥ 25 settled) or overconfident (predicted exceeds realized by ≥ 12pp) — see the curation section above.
2. The raw model probability is calibrated against the family's real settled record (Bayesian blend toward realized rate, isotonic-recalibrated) — the pick page shows raw → calibrated with the family's n and realized rate.
3. The calibrated probability must sit in the conviction band, and calibrated EV at the recorded odds must clear +2%. Nothing qualifies → an honest NO-PICK day, published like any other result.
4. Highest calibrated EV wins; the day's selection locks on first sight in an append-only file and replays from the lock forever — a published day can never be quietly rewritten by later data.

Rule v2 is date-gated at 2026-08-10; earlier days replay under v1 byte-identically and the two records are disclosed separately on the pick page. The lock file is public: alpha_v2_selections.json.

How a model earns its way onto the boards

The CFB desk is the clearest example of what "experimental" means here. Its ratings are Elo with margin-of-victory damping, fit on real ESPN results rather than win-pct, and every constant was measured on the prior season by walk-forward, not borrowed from a football model built for the NFL. Two college-specific facts shaped it: games against FCS opponents average a 34.0-point margin versus 16.4 between FBS teams (so those opponents pool into one rating at reduced weight — beating a cupcake by 40 must not mint rating), and last season's ratings turn out to be worth keeping only ~45% of the way, because college rosters churn far harder than the pros'.

The receipts are published in both regimes, because one of them flatters the model: it predicts 70.3% of games correctly (0.191 Brier) once a season has four weeks of its own data, but only 64.6% (0.206 Brier) predicting off prior-season ratings alone — which is where it stands every September.

And we measured it against the market rather than assuming. Scored head-to-head on 343 identical 2025 games, the closing line posted 0.1866 Brier and 74.1% accuracy; this model posted 0.2026 and 69.1%. The market is sharper than we are, and we land on its side 81% of the time. That result is the entire justification for the desk's posture: the job is to examine the ~19% where the model disagrees, not to claim it out-prices a closing line. Reproduce it with python cfb_ratings.py --vs-market; the cached result ships at cfb_market_benchmark.json.

Nothing from this desk touches the curated boards or the headline record. Its picks are recorded as an experimental family purely to build a settled record, and promotion is gated on published bars — 100 settled picks at non-negative ROI — that are computed against the ledger on every run and shown on the CFB desk, not asserted in prose. Meeting them opens a review; it does not auto-promote anything.

v1 prices moneylines only — no spreads, totals, or player props. Reproduce the constant search with python cfb_ratings.py --tune.

Model overconfidence can't print a fake edge

On game lines, the closing market is sharp. Rather than trust a model probability that diverges wildly from a sharp close, we shrink toward the de-vigged book line — weight w = min(0.80, |gap| × 3). A 53%-vs-67% disagreement gets pulled most of the way to the book; the raw model number is kept alongside, for divergence tracking and learning, but the displayed pick is shaded.

Same mechanism across MLB moneylines and the World Cup match model. Display-only — it never contaminates the raw signal the learning loop trains on.

Did the market move to our side?

Winning a bet can be luck. Beating the close is the metric that survives variance: for every game we lean, did the line move toward our side by the time it closed? We snapshot the de-vigged line open→close from free odds and measure the move. It's the cleanest proof of edge independent of any single result.

And we publish it even when it's below 50%. Right now the MLB game-line model doesn't consistently beat the close — so the CLV page says exactly that. A desk that only shows you its wins isn't a desk.

It gets sharper, without learning from itself

Every few hours: settle fresh results → recompute per-family shrinkage weights → run an integrity audit (Wilson confidence intervals, Brier, ECE, guard checks) → refresh the isotonic map. Crucially, the loop trains on each pick's raw p_predicted, never the calibrated output — so it can't chase its own tail. Guards block any "learning" that a small or noisy sample would otherwise over-fit.