feat(fitness): multi-objective combined = alpha*IS + (1-alpha)*OOS opt-in
Aggiunti due meccanismi per selection multi-objective: 1) Helper compute_combined_fitness(fit_train, fit_oos, alpha): formula = alpha*IS + (1-alpha)*OOS, fallback a IS se OOS è None/NaN. 2) RunConfig.eval_oos_during_loop (default False) + fitness_combined_alpha (default 0.5). Quando True E wfa_train_split attivo, ogni genome con fitness IS > 0 viene rivalutato su test_ohlcv DURANTE il loop GA e la fitness usata per tournament_select/elite_select è quella combinata. 2x costo backtest engine (richiede 2 evaluation per genome). 3) CLI flags --eval-oos-during-loop e --fitness-combined-alpha. Motivazione: il run phase2-7-max7y-v2-wfa-001 ha mostrato che il top by fitness_IS (4e1be9fa, ratio OOS 0.31) NON è il top per performance OOS reale (634111992702, ratio 1.42, ret_OOS +105% / 2.2y). Selezionare durante GA con combined fitness orienta l'evoluzione verso strategie OOS-robust by design invece di filtrarle a posteriori. Backward compat: default eval_oos_during_loop=False → comportamento invariato per run senza il flag. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -81,6 +81,21 @@ def parse_args() -> argparse.Namespace:
|
||||
default=5,
|
||||
help="Walk-forward: quanti top genomi rivalutare OOS (default 5)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--eval-oos-during-loop",
|
||||
action="store_true",
|
||||
help=(
|
||||
"Multi-objective: eval ogni genome anche su test_ohlcv durante "
|
||||
"il loop e usa combined = alpha*IS + (1-alpha)*OOS per selection. "
|
||||
"Richiede --wfa-train-split. 2x costo backtest engine."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--fitness-combined-alpha",
|
||||
type=float,
|
||||
default=0.5,
|
||||
help="Multi-objective: peso IS (1-alpha = OOS). 1.0=solo IS, 0.5=bilanciato, 0.0=solo OOS",
|
||||
)
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
@@ -144,6 +159,8 @@ def main() -> None:
|
||||
fitness_adversarial_soft_penalty=args.fitness_soft_penalty,
|
||||
wfa_train_split=args.wfa_train_split,
|
||||
wfa_top_k=args.wfa_top_k,
|
||||
eval_oos_during_loop=args.eval_oos_during_loop,
|
||||
fitness_combined_alpha=args.fitness_combined_alpha,
|
||||
)
|
||||
|
||||
run_id = run_phase1(cfg, ohlcv=ohlcv, llm=llm)
|
||||
|
||||
@@ -32,6 +32,26 @@ from ..agents.adversarial import AdversarialReport, Severity
|
||||
from ..agents.falsification import FalsificationReport
|
||||
|
||||
|
||||
def compute_combined_fitness(
|
||||
fitness_train: float,
|
||||
fitness_oos: float | None,
|
||||
alpha: float = 0.5,
|
||||
) -> float:
|
||||
"""Combina fitness IS e OOS in uno scalare per selection multi-objective.
|
||||
|
||||
Formula::
|
||||
|
||||
combined = alpha * fitness_train + (1 - alpha) * fitness_oos
|
||||
|
||||
Se ``fitness_oos`` è ``None`` o NaN, ritorna ``fitness_train`` (fallback).
|
||||
alpha=1.0 → solo IS (= comportamento default). alpha=0.0 → solo OOS.
|
||||
alpha=0.5 → bilanciato.
|
||||
"""
|
||||
if fitness_oos is None or fitness_oos != fitness_oos: # noqa: PLR0124 (NaN check)
|
||||
return fitness_train
|
||||
return alpha * fitness_train + (1.0 - alpha) * fitness_oos
|
||||
|
||||
|
||||
def compute_fitness(
|
||||
falsification: FalsificationReport,
|
||||
adversarial: AdversarialReport,
|
||||
|
||||
@@ -61,6 +61,12 @@ class RunConfig:
|
||||
# dei top genomi sui restanti. None/0 = no WFA (eval full ohlcv).
|
||||
wfa_train_split: float | None = None
|
||||
wfa_top_k: int = 5 # quanti top genomi rivalutare OOS
|
||||
# Multi-objective selection: se True, ogni genome viene valutato anche su
|
||||
# test_ohlcv durante il loop e la fitness usata per tournament/elite è
|
||||
# combined = alpha*IS + (1-alpha)*OOS. Richiede wfa_train_split attivo.
|
||||
# 2x costo backtest engine.
|
||||
eval_oos_during_loop: bool = False
|
||||
fitness_combined_alpha: float = 0.5 # peso IS (1-alpha = OOS)
|
||||
|
||||
|
||||
def run_phase1(
|
||||
@@ -176,6 +182,31 @@ def run_phase1(
|
||||
hard_kill_findings=cfg.fitness_hard_kill_findings,
|
||||
adversarial_soft_penalty=cfg.fitness_adversarial_soft_penalty,
|
||||
)
|
||||
# Multi-objective: se attivo, eval OOS subito e combina via alpha.
|
||||
if (
|
||||
cfg.eval_oos_during_loop
|
||||
and test_ohlcv is not None
|
||||
and len(test_ohlcv) >= 100
|
||||
and fit > 0
|
||||
):
|
||||
try:
|
||||
fals_oos_inloop = falsification_agent.evaluate(
|
||||
proposal.strategy, test_ohlcv
|
||||
)
|
||||
adv_oos_inloop = adversarial_agent.review(
|
||||
proposal.strategy, test_ohlcv
|
||||
)
|
||||
fit_oos_inloop = compute_fitness(
|
||||
fals_oos_inloop, adv_oos_inloop,
|
||||
hard_kill_findings=cfg.fitness_hard_kill_findings,
|
||||
adversarial_soft_penalty=cfg.fitness_adversarial_soft_penalty,
|
||||
)
|
||||
fit = (
|
||||
cfg.fitness_combined_alpha * fit
|
||||
+ (1.0 - cfg.fitness_combined_alpha) * fit_oos_inloop
|
||||
)
|
||||
except Exception: # noqa: BLE001
|
||||
pass # fallback: usa solo IS
|
||||
repo.save_evaluation(
|
||||
run_id=run_id,
|
||||
genome_id=genome.id,
|
||||
|
||||
Reference in New Issue
Block a user