feat(phase-2.6): Walk-Forward Validation + min-trades filter parametrico

Due fondamenta scientifiche per filtrare overfit e lucky-shot:

1) undertrading_threshold parametrico (era hardcoded 10):
   - AdversarialAgent.__init__(undertrading_threshold=10)
   - CLI flag --undertrading-threshold
   - Aggiunto a hard_kill_findings v2 default
     {"no_trades", "degenerate", "undertrading"}: ora un genome con 1 trade
     fortunato (es. genome 80be6bcc-1trade-fit-0.21 di fitness-v2-combo) viene
     killato anche sotto fitness v2 soft-kill.
   - Test parametric: undertrading_threshold=25 → 15 trade triggerano HIGH.

2) Walk-Forward Validation (WFA):
   - RunConfig.wfa_train_split (None=off, 0<x<1=on) + wfa_top_k=5
   - run_phase1: split ohlcv in train/test; GA usa solo train; a fine GA
     i top_k genomi (by fitness in-sample, fitness>0) vengono rivalutati
     sul test_ohlcv via falsification+adversarial+compute_fitness.
   - Schema migration: evaluations + fitness_oos, sharpe_oos, return_oos,
     max_dd_oos, n_trades_oos (ALTER TABLE con try/except per DB pre-2.6).
   - Repository.update_evaluation_oos helper per popolare colonne OOS.
   - CLI flags --wfa-train-split, --wfa-top-k.
   - Test integration: train_split=0.7 → fitness_oos popolato per top_k.

Motivazione: la fase 2.5 ha generato 17 run con fitness fino a 0.36 + DSR
positivo, ma OOS test su 7 anni mostra che flat-ablation top crolla -37%
mentre fitness-v2 top regge (+143%). WFA in-run permette ora di vedere
direttamente il degradation train→test senza eseguire backtest separati,
rendendo possibile filtrare overfit early durante l'ottimizzazione.

Tests (+2 → 193 totale):
- test_undertrading_threshold_parametric
- test_e2e_wfa_populates_fitness_oos

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-12 17:31:22 +02:00
parent 4c184bb5f7
commit 242724ba05
7 changed files with 198 additions and 7 deletions
+32
View File
@@ -100,3 +100,35 @@ def test_e2e_minimal_run_completes(
assert len(gens) == 2
evals = repo.list_evaluations(run_id)
assert len(evals) >= 5 # almeno una popolazione
def test_e2e_wfa_populates_fitness_oos(
tmp_path: Path,
synthetic_ohlcv,
fake_llm,
mocker,
):
"""WFA: train_split=0.7 → top genomi devono avere fitness_oos popolato."""
cfg = RunConfig(
run_name="e2e-wfa-test",
population_size=5,
n_generations=2,
elite_k=1,
tournament_k=2,
p_crossover=0.5,
seed=42,
model_tier=ModelTier.C,
symbol="BTC/USDT",
timeframe="1h",
fees_bp=5.0,
n_trials_dsr=10,
db_path=tmp_path / "runs.db",
wfa_train_split=0.7,
wfa_top_k=3,
)
run_id = run_phase1(cfg, ohlcv=synthetic_ohlcv, llm=fake_llm)
repo = Repository(db_path=tmp_path / "runs.db")
evals = repo.list_evaluations(run_id)
# Almeno 1 genome con fitness > 0 deve avere fitness_oos popolato.
oos_evals = [e for e in evals if e.get("fitness_oos") is not None]
assert len(oos_evals) >= 1, f"Nessun OOS popolato; evals={evals}"
+43
View File
@@ -194,6 +194,49 @@ def test_undertrading_under_10_is_high(monkeypatch: pytest.MonkeyPatch,
)
def test_undertrading_threshold_parametric(monkeypatch: pytest.MonkeyPatch,
ohlcv: pd.DataFrame) -> None:
"""undertrading_threshold=25 → 15 trade vengono killati come HIGH."""
fake_trades = [
_make_trade(
ohlcv.index[i * 30],
ohlcv.index[i * 30 + 10],
entry_price=100.0,
exit_price=101.0,
)
for i in range(15)
]
fake_signals = pd.Series(
[Side.LONG] * 250 + [Side.FLAT] * 250, index=ohlcv.index, dtype=object
)
def fake_run(self, ohlcv: pd.DataFrame, signals: pd.Series) -> BacktestResult: # type: ignore[no-untyped-def]
return BacktestResult(
equity_curve=pd.Series([0.0] * len(ohlcv), index=ohlcv.index, name="equity"),
returns=pd.Series([0.0] * len(ohlcv), index=ohlcv.index, name="returns"),
trades=fake_trades,
)
def fake_compile(strategy): # type: ignore[no-untyped-def]
return lambda df: fake_signals
monkeypatch.setattr("multi_swarm.agents.adversarial.BacktestEngine.run", fake_run)
monkeypatch.setattr("multi_swarm.agents.adversarial.compile_strategy", fake_compile)
ast = parse_strategy(_MINIMAL_STRATEGY_SRC)
# Default threshold 10: 15 trade NON killato
agent_default = AdversarialAgent()
rep_default = agent_default.review(ast, ohlcv)
assert not any(f.name == "undertrading" for f in rep_default.findings)
# Threshold 25: 15 trade killato
agent_strict = AdversarialAgent(undertrading_threshold=25)
rep_strict = agent_strict.review(ast, ohlcv)
assert any(
f.name == "undertrading" and f.severity == Severity.HIGH
for f in rep_strict.findings
)
def test_overtrading_with_tighter_threshold(monkeypatch: pytest.MonkeyPatch,
ohlcv: pd.DataFrame) -> None:
"""n_trades > n_bars/20 -> MEDIUM overtrading (Phase 1.5: era /5)."""