§ Models · the methodology hub
Six models in escalating sophistication — each one a published, validated rung the next must clear — combined into one ensemble and played out 1.1 million times. This is the whole method, in the open: the equations, the data each model eats, its out-of-sample score, and a clear verdict on what it can and cannot do.
The headline is deliberately modest. The production ensemble scores an out-of-sample RPS of 0.1891, a hair under the de-vigged market's 0.1905 (−0.0014) — but at an effective n = 3 tournaments the confidence intervals overlap heavily. The supported claim is that the model matches the market; it does not beat it. What it earns is a blend weight w* = 0.66 (CI excludes 0): the features carry marginal information the market does not fully price.
Models in the ladder
+ the de-vigged market as the ceiling benchmark
Ensemble OOS RPS (expanding)
vs market 0.1905 — matches, does not beat
Out-of-sample matches
across 3 tournaments with market odds (Copa·Euro·WC 2024/22)
Tournament simulations
of the official 48-team bracket, full FIFA tiebreakers
§ 01
From a reproducible Elo floor to the Monte-Carlo simulator, each rung adds one idea and must beat the rung below it out-of-sample. The market consensus is the ceiling to match. Click any rung for its full page.
Fig. M1 Six rungs · escalation order · skill meter to the ceiling
Read top-down: the simulator wraps the ensemble, which pools the three goal-model kernels (Dixon-Coles, Bayesian, LightGBM) that all sit on the Elo state. The skill meter shows how close each rung's out-of-sample RPS gets to the best achievable, with the market ceiling marked.
Tournament Monte-Carlo simulator
1.1M simulations of the real 48-team bracket with full FIFA tiebreakers.
Combination Ensemble
Simple-average log-opinion pool; beat stacking out-of-sample.
Features LightGBM-Poisson (global)
Gradient boosting on 36 features with monotone constraints; the de-biasing win.
Shrinkage Bayesian hierarchical
Partial pooling so newcomers borrow strength; NUTS/MAP, horseshoe priors.
Baseline Dixon-Coles
Bivariate-Poisson goals with low-score correction and host gating.
Floor Elo
Self-updating strength from results; movement-of-victory and importance weighted.
Floor · every model must clear Elo-only (RPS 0.1938)
Skill meter: fill spans the Elo floor (empty) to the best achievable RPS; the tick marks the market ceiling. Expanding-window out-of-sample RPS (the quoted skill), 152 matches across 3 tournaments.
The kernels are tightly bunched and the ensemble closes the last sliver to the ceiling — gains come from combining diverse-but-comparable models, not from one dominant model.
§ 02
Every model's out-of-sample RPS (expanding window) with its conservative bootstrap/jackknife 95% interval. Sorted best-first. The intervals are wide by design — n = 3 tournaments — and they overlap.
Fig. V11 Expanding window · lower is better · whisker = conservative 95% CI
The ensemble (0.1891) and the market (0.1905) lead a tight pack, but their intervals — 0.1607–0.2164 and 0.1660–0.2151 — overlap almost entirely. The Elo floor (0.1938) is the bar every model clears.
At n = 3 the ranking is suggestive, not decisive: the ensemble's lead over the market is well inside the combined uncertainty. The order is reported as measured, and the blend-weight test carries the inferential load.
§ 03
Three findings from the build — the things that did, and did not, move the needle.
Finding 01 · the ceiling
On out-of-sample RPS the ensemble (0.1891) edges the de-vigged market (0.1905), but the gap (−0.0014) is well inside the bootstrap interval, which includes zero. The defensible claim is parity. Where the model earns its keep is the convex blend: w* = 0.66 with a CI of [0.20, 1.11] that excludes 0 — the features add marginal information beyond the closing line.
See the full validation →Finding 02 · global club coverage
The squad signal was built from top-5-Europe club data, which is sparse for non-European squads — a coverage bias that suppressed the signal it was meant to carry. Swapping to a genuinely global club panel lifted average coverage from 48% to 85% and moved the LightGBM model from worst to best-non-market. The gains land off-UEFA and on low-coverage teams — directionally exactly as predicted, though not significant at n = 3.
How the de-biasing works →Finding 03 · the n = 3 ceiling
A stacked meta-learner scored worse out-of-sample than a plain equal-weight average (+0.0021 RPS, inside the ±0.0022 bootstrap SE), so the production combiner is the simple average — the forecast-combination puzzle, confirmed here. With only three out-of-sample tournaments, every interval is wide; two confirmatory tests were pre-registered, everything else is multiplicity-controlled, and the expanding-window number (never the optimistic LOTO) is quoted as expected skill.
Inside the ensemble →The six models are not competitors to be selected down to one winner on three tournament folds — that is the multiple-testing trap. They are an ensemble basis with controlled disagreement: each is tuned on the match-level backtest (tens of thousands of internationals, large n), then combined and validated once on the tournaments. Dixon-Coles and the Bayesian model disagree precisely on draw-heavy knockout football; the LightGBM model adds non-linear feature interactions; pooling diverse-but-comparable kernels is what sharpens the forecast.
Every RPS on this page is the expanding-window out-of-sample number (train on tournaments before the test fold, never after) over 152 matches in 3 tournaments — the realism protocol. The leave-one-tournament-out (LOTO) number, which leaks future tournaments into past training, is reported on each model page as an optimistic ceiling, never as the headline. The Elo floor is the bar every model must clear before its layer is added.