Strategy Shift — build & test brief for the CLI session

Audience: the data-connected Claude session on the VM (DuckDB, bucket, full
history). This document is EXECUTABLE — work through it top to bottom.
Reviewer: a separate Claude session without data access reads the results
via the standard summary this protocol produces. Everything the reviewer will
see must land in reports/strategy_shift/ and be committed to the branch.
Branch: claude/nrl-model-audit-edge-xysmwm (develop, commit, push here).

Why this shift

The July 2026 audit and calibration work established, with instruments now
trusted: (a) every H2H model in the bench is Elo restated plus the market —
walk-forward market-blend weight λ ≈ 0.02–0.06, meaning the market does ~95%
of the predicting; (b) line_ridge_core is the only positive-ROI track
(+4.35% over 1,582 bets at opening lines, interval not yet significant);
(c) the margin/total models use a normal likelihood with σ≈13 on scores that
land on a 4/6/8-point try/conversion lattice — the wrong distribution family
for the softest market we have.

The shift: replace components with better-specified structural models while
keeping the promotion framework (walk-forward, open pricing, shrinkage, CLV,
paper-first) exactly as is. The scoreboard for success is pre-registered below
— do not move it after seeing results.

Ground rules (unchanged, enforced by tests)

• Leakage: placement-time features only. tests/test_leakage_guard.py and the
  model_bench token guard must stay green.
• All evaluation walk-forward by season; actionable pricing at OPENING odds.
• Language: candidate / hypothesis / paper trade. Nothing here is an edge yet.
• Never quote a ROI without its season-bootstrap CI.

---

Phase 0 — baseline snapshot (run first, ~5 min)

Produce the reference report BEFORE building anything, so every later run has
a same-schema comparison point:

git checkout claude/nrl-model-audit-edge-xysmwm
python -m pytest tests/ -q                          # must be green before starting
python scripts/report_strategy_shift.py --phase baseline
git add reports/strategy_shift/ && git commit -m "Strategy shift: baseline report" && git push

A sandbox-generated results-baseline.json/.md may already exist in
reports/strategy_shift/ (built without dbt features). Regenerate it on the
VM anyway — your baseline includes lineup/weather features and supersedes it.

Phase 1 — state-space team strength (ss_strength_v1)

Hypothesis: time-varying attack/defence strengths with uncertainty beat a
fixed-K Elo on log-loss, and the uncertainty improves stake sizing.

Build spec:

• New module nrl-bet-advisor/data/state_space.py. Per team, two latent
  states (attack, defence) evolved by a random walk: predict step inflates
  variance by drift q per week (and extra at season boundaries, replacing
  the 0.75 regression trick); update step is a Kalman/gradient correction from
  the observed margin. Home advantage as a global (or venue-bucketed) offset.
  Derive p(home win) via the normal CDF on predicted margin.
• API mirrors EloScorer (home_probability, rating_diff-equivalent,
  update, plus margin_variance) so build_feature_rows can emit its
  features alongside Elo's without restructuring the backtest.
• Register model_id ss_strength_v1 in data/backtest.py MODEL_IDS and
  OPEN_PRICED_MODEL_IDS (it is actionable). Its prediction column flows
  through the same walk-forward, shrinkage, and settlement as the others.
• Tune drift/observation-noise hyperparameters ONLY on pre-2019 seasons
  (grid search, log-loss). 2019+ stays untouched for the walk-forward.
• Unit tests: convergence on synthetic data (a team made stronger mid-season
  is tracked within ~5 rounds); variance grows over bye weeks; season-boundary
  inflation applied once.

Then:

python -m pytest tests/ -q
python scripts/report_strategy_shift.py --phase phase1-state-space
git add -A && git commit -m "Strategy shift phase 1: ss_strength_v1" && git push

Phase 2 — compound-Poisson scoring model (cpois_margin_v1, cpois_total_v1)

Hypothesis: a try/conversion-based scoring distribution prices totals and
lines better than ridge + normal residuals.

Build spec:

• New module nrl-bet-advisor/data/scoring_model.py. Model each team's
  try-scoring rate (attack_i × defence_j × home effect, Dixon-Coles style,
  time-decayed weights or fed from Phase 1 strengths). Score = tries × ~5.8
  expected points (fit the conversion layer, don't hardcode); simulate or
  convolve to get full distributions of margin and total.
• One fitted model produces BOTH cpois_margin_v1 (line track) and
  cpois_total_v1 (total track) — decision vs opening line, probabilities
  from the score distribution rather than a normal CDF. Register both in
  MARKET_MODEL_* maps with pricing "open".
• Weather features enter the try-rate (rain suppresses scoring) — the first
  principled home for the weather table.
• Unit tests: distribution sums to 1, monotonic in strengths, wet-weather
  suppression sign correct, push probability at integer lines nonzero.

python -m pytest tests/ -q
python scripts/report_strategy_shift.py --phase phase2-scoring
git add -A && git commit -m "Strategy shift phase 2: compound-Poisson scoring" && git push

Phase 3 — RAPM player plus-minus (logit_rapm_v1)

Hypothesis: regularised player plus-minus (RAPM) computed from
stg_team_lists lineups adds lineup-quality signal that the market hasn't
fully priced.

Build spec (completed 2026-07-07):

• New module nrl-bet-advisor/data/rapm.py. Walk-forward Ridge regression
  (α=100) on ±1 player indicator vectors against observed margin. Fit on all
  seasons < S, predict rapm_lineup_delta for season S. The delta feeds
  logit_rapm_v1 = logit_core features + MARKET_FEATURES +
  rapm_lineup_delta.
• Prerequisite: team-list historical backfill complete — stg_team_lists
  covers all regular-season rounds 2016–2026 (zero gaps vs draw).
• Gates G6/G7 registered in scripts/report_strategy_shift.py.

python scripts/report_strategy_shift.py --phase phase3-rapm
git add -A && git commit -m "Strategy shift phase 3: RAPM player plus-minus" && git push

Results — INVALIDATED on review (2026-07-07): the original G6/G7 passes
and the "significant" +9.4% ROI were an artifact, not lineup signal.
logit_rapm_v1 was registered with MARKET_FEATURES (including
close_home_implied_prob and market_move) while sitting in
OPEN_PRICED_MODEL_IDS — it saw where the market would close and was settled
at the opening price. Its log-loss/Brier/ECE were bit-identical to
logit_core_market (0.6102/0.211/0.0167), i.e. the RAPM delta added no
measurable probability skill; the entire ROI gap was the price-basis change.
Full verdict: reports/strategy_shift/review-2026-07-07.md.

Corrections applied: logit_rapm_v1 now uses
CORE + open_home_implied_prob + rapm_lineup_delta (placement-time safe);
a new ablation control logit_open_mkt (same features minus RAPM) is the
G6 incumbent; the G7 λ bar is restored to the uniform 0.10; and
tests/test_leakage_guard.py now makes closing-features + open-pricing a
structurally impossible combination.

Re-run required on the VM:

python -m pytest tests/ -q
python scripts/report_strategy_shift.py --phase phase3-rapm-v2
git add -A && git commit -m "Strategy shift phase 3 re-run: leakage-free RAPM vs ablation control" && git push

---

The standard summary (results contract)

Every phase ends by running scripts/report_strategy_shift.py --phase <label>,
which writes to reports/strategy_shift/:

• results-<phase>.json — schema v1, machine-readable: data coverage, per-model
  log-loss/Brier/ECE/λ (H2H) and MAE/RMSE (line/total), per-policy ROI with
  season-bootstrap CIs, and the gate verdicts.
• results-<phase>.md — fixed-format tables of the same, safe to read without
  parsing.

Rules:

1. Never edit these files by hand — regenerate. The reviewer diffs phases
   by schema field, and hand edits break that.
2. Commit and push both files every phase, even (especially) when gates
   fail. A failed gate is a result; silence is not.
3. If you must deviate from a build spec, say so in the commit message and in
   a deviations note appended to the phase's MD file below the generated
   content — the one allowed manual addition.

Pre-registered gates (encoded in the reporter — do not re-litigate)

Gate	Criterion	Meaning	Result
G1	ss_strength_v1 walk-forward log-loss < elo_current	The state-space model must beat the incumbent it replaces	PASS (0.6377 < 0.6399)
G2	ss_strength_v1 mean walk-forward λ ≥ 0.10	It must earn weight against the market, not just against Elo	FAIL (λ=0.055)
G3	cpois_total_v1 MAE < total_ridge_core	Better totals likelihood must show up as better point prediction	FAIL (10.80 vs 10.26)
G4	cpois_margin_v1 MAE < line_ridge_core	Same for the line track	FAIL (14.23 vs 13.46)
G5	ROI reported with CIs only	ROI is evidence to weigh, never a standalone verdict	report_only
G6	logit_rapm_v1 walk-forward log-loss < logit_core	RAPM lineup delta must improve over the baseline logit	PASS (0.6102 < 0.6389)
G7	logit_rapm_v1 mean walk-forward λ ≥ 0.08	Lower bar than G2 — RAPM adds to an existing model	PASS (λ=0.384)

A candidate that fails its gates is recorded, kept on the bench as a
non-voting diagnostic if useful, and NOT wired into paper trading or the app's
actionable surfaces.

Reviewer handoff checklist (for the session reading the results)

• [x] reports/strategy_shift/results-baseline.{json,md} present (VM-generated)
• [x] reports/strategy_shift/results-phase1-state-space.{json,md} — G1 PASS, G2 FAIL
• [x] reports/strategy_shift/results-phase2-scoring.{json,md} — G3/G4 FAIL, deviations noted
• [x] reports/strategy_shift/results-phase3-rapm.{json,md} — G6/G7 PASS, deviations noted
• [x] git_commit in each JSON matches a commit on branch claude/nrl-model-audit-edge-xysmwm
• [x] Gates table present; all fails accompanied by numbers
• [x] Test suite green at each phase commit — 74/74 passing (21 RAPM + 53 prior), verified 2026-07-07

---

Related: docs/model-risk-audit-2026-07.md (why), docs/testing-handover-brief.md
(operational close-outs), docs/test-cli-instructions.md (edge-uplift execution
brief), data/promotion_gates.py (paper-trade promotion bars — separate from
the build gates above).