feat(SH01): bootstrap storia full-history + last_block_only (punto-10: regime live non robusto)

Ri-validazione sh01_trainwindow_validate.py (rolling train_window == regime live):
- tw=8760 (live 365g): BTC FULL +82% ma fee-2x -42% (6/9 anni), ETH Sharpe -0.02,
  trade-rate 21.7-26.4% vs 9.8% validato -> NON robusto. Diagnosi sweep confermata
  (LogReg over-confident su train corto, th 0.58 inerte).
- progressione MONOTONA con la memoria (2y: Sh 2.05; 3y: 3.09; expanding: ROBUSTO
  8/9) -> l'edge di SH01 e' la memoria lunga.
- sweep soglia nel regime corto: instabile/incoerente fra asset -> NON ri-tunare.

Fix (decisione utente): ripristinare in live il regime validato.
- ml_wf_entries(last_block_only=True): fitta/predice solo l'ultimo blocco del WF
  (confini deterministici start+k*retrain) -> entries IDENTICHE per costruzione
  al tail del WF completo (parity test esatto); 0.6s/tick su 73k barre vs ~140 fit.
- runner._with_history: tick degli sleeve ml su parquet locale + feed live
  (dedup timestamp, gap-guard con WARN una-tantum se parquet stantio).
- _defs.py: params {last_block_only: true} sugli sleeve SHAPE (solo path live;
  il backtest canonico resta WF completo).

Effetto atteso live: trade-rate SH01 ~25% -> ~10% delle barre, selettivita'
ripristinata. Manutenzione: parquet fresco via download_all (oggi 2026-05-28,
margine ~11 mesi col feed 365g).

Test: 87/87 (4 nuovi: parity last-block esatta, merge storia, gap fallback).
Diario docs/diary/2026-06-07-sh01-trainwindow.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Adriano Dal Pastro
2026-06-07 16:48:22 +00:00
parent be85a454e0
commit 8c5372c487
8 changed files with 268 additions and 6 deletions
+60
View File
@@ -0,0 +1,60 @@
"""SH01 path live: last_block_only == tail del walk-forward completo (parity by
construction) + merge storia parquet/feed del runner."""
import numpy as np
import pandas as pd
import pytest
from src.portfolio.runner import _with_history
def _df(n=6500, seed=0, t0=1_600_000_000_000):
rng = np.random.default_rng(seed)
c = 100 * np.exp(np.cumsum(rng.normal(0, 0.01, n)))
h = c * (1 + np.abs(rng.normal(0, 0.004, n)))
l = c * (1 - np.abs(rng.normal(0, 0.004, n)))
o = np.roll(c, 1); o[0] = c[0]
ts = t0 + 3_600_000 * np.arange(n)
return pd.DataFrame({"timestamp": ts, "open": o, "high": h, "low": l,
"close": c, "volume": 1.0})
def test_last_block_only_matches_full_wf_tail():
from scripts.analysis.shape_ml_research import ml_wf_entries
df = _df()
cfg = dict(W=24, H=12, model="logit", thresh=0.55, train_min=4000, retrain=500)
full = ml_wf_entries(df, **cfg)
last = ml_wf_entries(df, last_block_only=True, **cfg)
n = len(df)
# confine dell'ultimo blocco: start + k*retrain (deterministico)
start = max(cfg["train_min"], cfg["W"] - 1)
B = start
while B + cfg["retrain"] < n - 1:
B += cfg["retrain"]
full_tail = [e for e in full if e["i"] >= B]
assert last == full_tail # identita' esatta, non solo simile
assert all(e["i"] >= B for e in last)
def test_with_history_merges_contiguous():
hist = _df(n=100)
live = _df(n=50, t0=int(hist["timestamp"].iloc[60])) # overlap: live parte dentro hist
out = _with_history(hist, live)
assert len(out) == 60 + 50 # hist pre-overlap + live completo
ts = out["timestamp"].values
assert (np.diff(ts) > 0).all() # serie strettamente crescente
def test_with_history_gap_falls_back_to_live():
hist = _df(n=100)
gap_start = int(hist["timestamp"].iloc[-1]) + 10 * 3_600_000 # buco di 10 barre
live = _df(n=50, t0=gap_start)
warned = set()
out = _with_history(hist, live, warned, "BTC")
assert len(out) == 50 and "BTC" in warned # solo feed + warn deduplicato
out2 = _with_history(hist, live, warned, "BTC")
assert len(out2) == 50 # secondo giro: nessun nuovo warn
def test_with_history_none_hist_passthrough():
live = _df(n=50)
assert _with_history(None, live) is live