ML Lab — All Models, Side by Side
The full machine learning stack. Four different models predict the same game using the same 30-feature vector. The ensemble combines them. The reliability diagram tells you whether to trust the numbers.
The reliability diagram and scoring metrics (Brier / ECE / log-loss) below are the model's real calibration from the settled results ledger — the bet-relevant numbers. The four architecture visuals (ensemble weights, feature importance, training curve) are an architecture demonstration — the hand-coded backprop / GBM / stacking math shown working on a synthetic development set, not live betting performance. The live, real model edge is the calibration here + the Model Calibration and Learning pages.
Each model uses the same 30-feature input vector. The ensemble's weights are learned from backtest cross-validation. Diversity matters — when one model is wrong, the others can compensate.
Normalized contribution of each base model to the ensemble's final prediction. Learned by gradient descent on a held-out validation set.
How much each feature contributes to model accuracy. Measured by shuffling that column in the test set and watching the loss bump up. Top features are exactly what a sharp baseball analyst would expect.
Training loss falls smoothly; validation loss bottoms out around epoch 10 — classic mild overfit that early stopping handles.
From every settled bet's model probability vs. its actual outcome. Diagonal = perfect calibration. Above = underconfident in that bucket; below = overconfident. Our calibration engine corrects the systematic gap before any pick is published.
| # | Feature | Family | Value | Description |
|---|
Brier Score: —
Mean squared error of predictions vs. outcomes, from real settled bets. Lower is better. Below 0.25 is solid; below 0.22 is a sharp model. Strictly proper scoring rule — can't be gamed.
Log Loss: —
Binary cross-entropy. The natural loss for probabilistic predictions. 0.69 = pure coin flip; below 0.66 means you're learning real structure.
ECE: —
Expected Calibration Error from real settled outcomes — the average gap between predicted probability and actual frequency. The raw model runs hot (overconfident), which is exactly why every displayed pick is passed through our calibration engine before you see it — see the Model Calibration map.
Raw game data
│
▼
┌──────────────────────┐
│ features.py │ 30 engineered features from 8 raw input families
│ extract_features() │ ── team, pitcher, bullpen, park/wx, ump, schedule, lineup, market
└────────┬─────────────┘
│ feature vector
▼
┌────────┼────────┬────────┬────────┐
▼ ▼ ▼ ▼
Bayesian GBM MC Sim NN MLP ← base models predict P(home wins)
│ │ │ │
└───────┴───┬───┴─────────┘
▼
┌───────────────┐
│ ensemble.py │ logistic meta-learner stacks base predictions
│ LogisticMeta │
└───────┬───────┘
│ raw ensemble probability
▼
┌───────────────┐
│ calibration.py│ Platt scaling corrects systematic over/under-confidence
│ platt_scale │
└───────┬───────┘
│ calibrated probability
▼
┌─────────────────────────────┐
│ Decision: edge vs market │
│ - edge = model_p - market_p│
│ - kelly_units = f*p,price │
│ - confidence = bucket │
└─────────────────────────────┘