UI deployment — local vs k8s parity review (2026-07-11)

The dashboard behaves differently locally and in the hosted (Docker/k8s)
instance because the two run from different filesystems and env contexts.
This is the parity contract: what differs, what was fixed, what remains.

The three UI stacks (state the obvious once)

Stack	Entry point	Status
Jinja dashboard (CANONICAL)	`nrl-bet-advisor/web/main.py`	What the Docker image serves (`CMD uvicorn web.main:app`, port 8080). All new pages (Models/Strategy/Jobs/Glossary) live here.
React/JSON backend	`nrl-bet-advisor/web/backend/app.py` + Vite frontend	From the `web-ui-migration` branch. Honors env config properly but is NOT what the image serves. Direction decision needed (see repo review).
Streamlit (legacy)	`attic/streamlit-legacy/`	Superseded, archived, and not deployed/imported by FastAPI.

Gaps found and FIXED this round

Repo-root assets missing from the image. The image copied only
nrl-bet-advisor/ + scripts/, but pages read repo-root paths:
/models ← reports/strategy_shift/*.json; /docs ← docs/*.md;
Features ← analytics/dbt_nrl/models/schema.yml; /ops model-artifact
row ← nrl/models/metrics.json. Hosted pages silently degraded (THE root
cause of "data not showing on model cards" on the hosted app; local worked
because the files exist in the repo). → Dockerfile now copies all four.
Keep the Dockerfile COPY list in sync when a page grows a file dependency.
DUCKDB_PATH ignored by the Jinja app's data layer. The image mounts
the DB at /data/nrl_bucket.duckdb and documents DUCKDB_PATH, and
web/backend/settings.py honors it — but data/backtest.py, data/rapm.py,
data/odds_movement.py, data/ops_monitor.py, and
web/services/jobs_service.py hardcoded <repo>/duckdb/nrl_bucket.duckdb.
Hosted: Jobs page all-UNAVAILABLE, RAPM caches empty. → All canonical
data consumers now resolve through data/db_path.py
(NRL_DUCKDB_PATH > DUCKDB_PATH > repo default, matching backend
precedence). The Jobs page also exposes fan-pick and monthly roster-refresh
coverage, so those feeds are visible operationally.

Gaps that REMAIN (operator decisions / VM work)

Report freshness in the image. reports/ is baked at build time, so
hosted /models lags until the next image build. Options: (a) rebuild the
image on each phase commit (CI hook — simplest, recommended); (b) mount
reports on the PVC like the DB; (c) publish reports to the bucket and read
via ODDS_MONITOR_ARTIFACT_PREFIX-style artifacts. Pick one in the next
ops round; until then note the staleness on the card (from <phase> stamp
already renders).
DuckDB writer vs reader. The VM writes the DuckDB file; the k8s pod
reads a PVC copy. How the PVC copy refreshes (sync cadence, tooling) is
undocumented — document/automate it, and surface last-sync age on /ops.
Basic auth / admin boundary: /admin/* is intentionally public and
read-only; it contains diagnostics and paper-trading views, not mutation or
bet placement. A deployment may protect the full dashboard with ingress or
proxy Basic Auth, while /healthz remains public for probes. Verify the
k8s Secret is mounted if that deployment policy is enabled — a missing
secret otherwise leaves the dashboard open.
Two-app drift. Until the React-vs-Jinja direction is decided, every
new feature lands twice or diverges. Recommendation in the repo review:
keep the Jinja app canonical for now (it has all operational pages), treat
the React backend as the future once feature-parity is written down.
Streamlit still imports and ships in the image via nrl-bet-advisor/
COPY — dead weight in the layer (harmless, ~small); archive decision with
the human.

Investigating "data not coming through" on the cluster (2026-07-12 update)

The dashboard now diagnoses itself: open /ops → Instance diagnostics on
the hosted instance. Each red row carries a fix hint; in practice the answer
is one of these four, in likelihood order:

Stale image — header shows build unknown or an old commit: the
deployment predates the Dockerfile asset fixes. Rebuild with
--build-arg GIT_COMMIT=$(git rev-parse --short HEAD) and redeploy.
DuckDB row red — DUCKDB_PATH unset in the Deployment or the PVC not
mounted at /data; the resolved path and its source env var are printed.
DuckDB opens but data is old — the header shows the DB file's last-sync
age; the VM→PVC copy job is behind (gap 4 below).
reports//docs//dbt-schemas rows red — image built before the COPY list
existed; rebuild.

The consolidated /ops page also stamps every feed with its origin
(vm / cluster / manual — from the collector column on odds snapshots, the
scheduling split otherwise) and a Next expected contract column, so
"missing" vs "not due yet" is readable at a glance. /jobs and
/odds-monitor now 308-redirect to /ops. No rerun triggers on the page by
design — commands are listed for the operator instead.

Smoke checklist for the hosted instance (run after each deploy)

/healthz            200 {"status": "ok"}
/models             cards show metrics + gates; header shows latest phase
/ops                diagnostics all green; contracts NOT all UNAVAILABLE;
                    header shows the expected git commit and runtime=cluster
/jobs /odds-monitor 308 → /ops
/docs               list populated, STRATEGY.md renders
/features           dbt layers present
/dq                 loads (elementary redirect or local fallback)
auth                401 without credentials when BASIC_AUTH_* is set

Authenticated smoke — 2026-07-14 (R20 build)

Status: COMPLETED — 2026-07-14. Routes up; DuckDB missing from pod (gap 4).

Route	Expected	Actual	Note
`/`	200	200 ✅ (307 → /matches)	redirect works
`/partials/matches?season=2026&round=Round%2020`	200 + vb-match-card	200 ❌ empty-state	DuckDB not mounted — no data
`/models`	200	200 ✅
`/strategy`	200	200 ✅
`/ops`	200	200 ✅
`/docs`	200	200 ✅

Root cause: DuckDB NOT mounted on the pod. /ops instance diagnostics shows:
- DuckDB: NOT FOUND at /app/duckdb/nrl_bucket.duckdb (via repo default) — red
- Build: unknown — image was built without --build-arg GIT_COMMIT=...
- runtime=cluster ✅ (origin chip correct)
- All data-contract rows: UNAVAILABLE (no DB → nothing to check)

This is gap 4 (DuckDB writer vs reader / PVC sync). The image is also stale
(build=unknown means it predates the GIT_COMMIT arg addition).

Fix actions required (operator):
1. Rebuild the image with --build-arg GIT_COMMIT=$(git rev-parse --short HEAD)
and --build-arg IMAGE_BUILT_AT=$(date -u +%Y-%m-%dT%H:%MZ) to stamp the build.
2. Mount the DuckDB PVC at /data in the Deployment and set
NRL_DUCKDB_PATH=/data/nrl_bucket.duckdb in the Deployment env.
3. Ensure the VM→PVC copy job has run since last staging (gap 4 — undocumented/manual).
4. Re-run six-route smoke after redeploy to confirm vb-match-card appears and
/ops shows all data-contract rows green.

# Re-run smoke after fixing deployment (operator runs from VM)
for p in "/" "/partials/matches?season=2026&round=Round%2020" "/models" "/strategy" "/ops" "/docs"; do
  curl -sS -u "$UI_SMOKE_AUTH" -o /dev/null -w "%{http_code} $p\n" \
    "https://scrapers-ui.doks-ci-cd.hungee.cloud$p"
done
# Gate: all 200; /partials/matches body contains vb-match-card