9742df3a1f
Incident: extended run elite (Sharpe IS 1.93) net-negativo su 7y
continuous (fees=101% del gross). Multi-fold validation NON sufficiente:
ogni fold restarta equity, mascherando accumulo fees compound.
Correzioni:
1) Default --start esteso a 2018-09-01 (7.3 anni)
- Copre bear 2018-19, halving 2020, COVID, ATH 2021, winter 2022,
ETF rally 2024, regime corrente.
- Una finestra corta (2y) lasciava il GA libero di overfit single-regime.
2) fees_eat_alpha promosso a hard-kill in fitness v2
- Da soft-penalty 0.4x a hard-kill 0 fitness.
- Una strategia con fees > 50% del gross non e' recuperabile via
selection: il prodotto del GA non puo' deployare con quel cost burden.
3) Nuovo finding negative_net_pnl (HIGH, hard-kill)
- Fires quando sum(trade.net_pnl) < 0 sul training window.
- Cattura: gross negativo (no edge direzionale) E gross positivo ma
fees > gross (edge insufficiente).
- Sintesi del net-after-fees su finestra continua come "deal-breaker"
non negoziabile via soft penalty.
CLI:
- --fitness-hard-kill <csv> per override esplicito.
- Default v2: no_trades,degenerate,undertrading,fees_eat_alpha,negative_net_pnl.
Test:
- 252 pass (251 + 1 nuovo: test_negative_net_pnl_fires_on_negative_gross).
- Test e2e WFA aggiornato: passa fitness_hard_kill_findings=('no_trades',)
perche' il fixture sintetico non produce strategie profittevoli.
- test_no_findings_on_reasonable_strategy rinominato:
test_rsi_mean_reversion_loses_money_on_synthetic_data (riflette
semantica reale: RSI mean-rev su synthetic ohlcv ha net negativo).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
246 lines
8.7 KiB
Python
246 lines
8.7 KiB
Python
from __future__ import annotations
|
|
|
|
import argparse
|
|
import importlib.resources
|
|
from datetime import datetime
|
|
from pathlib import Path
|
|
|
|
from multi_swarm_core.cerbero.client import CerberoClient
|
|
from multi_swarm_core.config import load_settings
|
|
from multi_swarm_core.data.cerbero_ohlcv import CerberoOHLCVLoader, OHLCVRequest
|
|
from multi_swarm_core.genome.hypothesis import ModelTier
|
|
from multi_swarm_core.genome.prompt_library import PromptLibrary
|
|
from multi_swarm_core.llm.client import LLMClient
|
|
from multi_swarm_core.orchestrator.run import RunConfig, run_phase1
|
|
|
|
|
|
def _default_prompt_library_path() -> Path:
|
|
"""Default: prompts.json shippato col package strategy_crypto."""
|
|
return Path(str(importlib.resources.files("strategy_crypto") / "prompts.json"))
|
|
|
|
|
|
# Default v2 hard-kill list: oltre ai degenerate originali, fees_eat_alpha e
|
|
# negative_net_pnl sono deal-breaker non recuperabili via soft penalty (vedi
|
|
# 7y-overfit incident 2026-05-16: elite IS Sharpe 1.93 -> net -5% su 7y per fees).
|
|
_DEFAULT_V2_HARD_KILL: tuple[str, ...] = (
|
|
"no_trades",
|
|
"degenerate",
|
|
"undertrading",
|
|
"fees_eat_alpha",
|
|
"negative_net_pnl",
|
|
)
|
|
|
|
|
|
def _resolve_hard_kill(args) -> tuple[str, ...] | None:
|
|
"""Resolve la lista hard-kill da CLI args.
|
|
|
|
Priority: ``--fitness-hard-kill`` esplicito > default v2 > ``None`` (v1).
|
|
"""
|
|
if args.fitness_hard_kill:
|
|
return tuple(s.strip() for s in args.fitness_hard_kill.split(",") if s.strip())
|
|
if args.fitness_v2:
|
|
return _DEFAULT_V2_HARD_KILL
|
|
return None
|
|
|
|
|
|
def parse_args() -> argparse.Namespace:
|
|
p = argparse.ArgumentParser(description="Multi-Swarm Phase 1 runner")
|
|
p.add_argument("--name", default="phase1-spike-001")
|
|
p.add_argument("--population-size", type=int, default=20)
|
|
p.add_argument("--n-generations", type=int, default=10)
|
|
p.add_argument("--elite-k", type=int, default=2)
|
|
p.add_argument("--tournament-k", type=int, default=3)
|
|
p.add_argument("--p-crossover", type=float, default=0.5)
|
|
p.add_argument("--seed", type=int, default=42)
|
|
p.add_argument(
|
|
"--exchange",
|
|
default="deribit",
|
|
choices=["deribit", "bybit", "hyperliquid"],
|
|
)
|
|
p.add_argument("--symbol", default="BTC-PERPETUAL")
|
|
p.add_argument("--timeframe", default="1h")
|
|
# Default esteso a 7.3 anni: copre bear 2018-19, halving 2020, COVID,
|
|
# ATH 2021, winter 2022, ETF rally 2024, regime corrente. Una finestra
|
|
# corta lascia il GA libero di overfit a un singolo regime.
|
|
p.add_argument("--start", default="2018-09-01T00:00:00+00:00")
|
|
p.add_argument("--end", default="2026-01-01T00:00:00+00:00")
|
|
p.add_argument("--fees-bp", type=float, default=5.0)
|
|
p.add_argument("--n-trials-dsr", type=int, default=50)
|
|
p.add_argument(
|
|
"--prompt-mutation-weight",
|
|
type=float,
|
|
default=0.0,
|
|
help="Phase 2.5: probabilità (0-1) che la mutazione invochi LLM mutator tier B",
|
|
)
|
|
p.add_argument(
|
|
"--fees-eat-alpha-threshold",
|
|
type=float,
|
|
default=0.5,
|
|
help="Adversarial gate: kill se fees/gross_pnl > soglia (default 0.5, ablation 0.7)",
|
|
)
|
|
p.add_argument(
|
|
"--flat-too-long-threshold",
|
|
type=float,
|
|
default=0.95,
|
|
help="Adversarial gate: kill se signal flat > soglia delle bar (default 0.95, ablation 0.98)",
|
|
)
|
|
p.add_argument(
|
|
"--undertrading-threshold",
|
|
type=int,
|
|
default=10,
|
|
help="Adversarial: kill se n_trades < soglia (default 10, bump per filtrare lucky-shot)",
|
|
)
|
|
p.add_argument(
|
|
"--fitness-v2",
|
|
action="store_true",
|
|
help=(
|
|
"Attiva fitness v2: hard-kill su {no_trades, degenerate, undertrading, "
|
|
"fees_eat_alpha, negative_net_pnl}; gli altri HIGH applicano soft penalty "
|
|
"multiplicativa. Versione hardened post 7y-overfit incident: fees + "
|
|
"net negativo non sono recuperabili."
|
|
),
|
|
)
|
|
p.add_argument(
|
|
"--fitness-soft-penalty",
|
|
type=float,
|
|
default=0.4,
|
|
help="v2: fattore soft penalty per HIGH non-hard (default 0.4 → 1 HIGH → 0.71x)",
|
|
)
|
|
p.add_argument(
|
|
"--fitness-hard-kill",
|
|
type=str,
|
|
default=None,
|
|
help=(
|
|
"Override comma-separated della lista di finding name che azzerano la "
|
|
"fitness in modalita' v2. Es: 'no_trades,degenerate'. Default v2: "
|
|
"no_trades,degenerate,undertrading,fees_eat_alpha,negative_net_pnl."
|
|
),
|
|
)
|
|
p.add_argument(
|
|
"--wfa-train-split",
|
|
type=float,
|
|
default=None,
|
|
help="Walk-forward: frazione bar usate per training (es. 0.7 = primi 70%% in-sample, ultimi 30%% OOS)",
|
|
)
|
|
p.add_argument(
|
|
"--wfa-top-k",
|
|
type=int,
|
|
default=5,
|
|
help="Walk-forward: quanti top genomi rivalutare OOS (default 5)",
|
|
)
|
|
p.add_argument(
|
|
"--eval-oos-during-loop",
|
|
action="store_true",
|
|
help=(
|
|
"Multi-objective: eval ogni genome anche su test_ohlcv durante "
|
|
"il loop e usa combined = alpha*IS + (1-alpha)*OOS per selection. "
|
|
"Richiede --wfa-train-split. 2x costo backtest engine."
|
|
),
|
|
)
|
|
p.add_argument(
|
|
"--fitness-combined-alpha",
|
|
type=float,
|
|
default=0.5,
|
|
help="Multi-objective: peso IS (1-alpha = OOS). 1.0=solo IS, 0.5=bilanciato, 0.0=solo OOS",
|
|
)
|
|
p.add_argument(
|
|
"--prompt-library",
|
|
type=Path,
|
|
default=None,
|
|
help=(
|
|
"Path al file JSON con stili cognitivi + direttive system_prompt. "
|
|
"Default: strategy_crypto/prompts.json shippato col package. "
|
|
"Schema: {styles: {<name>: {directive: <testo>}}}"
|
|
),
|
|
)
|
|
p.add_argument(
|
|
"--llm-concurrency",
|
|
type=int,
|
|
default=1,
|
|
help=(
|
|
"Numero di propose() LLM concorrenti per generazione (default 1 = "
|
|
"serial). 6-10 tipicamente accettati da OpenRouter qwen-2.5 senza "
|
|
"rate-limit; riduce wall time GA loop di 5-8x."
|
|
),
|
|
)
|
|
return p.parse_args()
|
|
|
|
|
|
def main() -> None:
|
|
args = parse_args()
|
|
settings = load_settings()
|
|
|
|
prompt_lib_path = args.prompt_library or _default_prompt_library_path()
|
|
prompt_library = PromptLibrary.from_json(prompt_lib_path)
|
|
print(
|
|
f"PromptLibrary loaded from {prompt_lib_path}: "
|
|
f"{len(prompt_library.styles)} stili ({', '.join(prompt_library.cognitive_styles)})"
|
|
)
|
|
|
|
token = (
|
|
settings.cerbero_mainnet_token.get_secret_value()
|
|
if settings.cerbero_mainnet_token
|
|
else settings.cerbero_testnet_token.get_secret_value()
|
|
)
|
|
cerbero = CerberoClient(
|
|
base_url=settings.cerbero_base_url,
|
|
token=token,
|
|
bot_tag=settings.cerbero_bot_tag,
|
|
)
|
|
|
|
loader = CerberoOHLCVLoader(client=cerbero, cache_dir=settings.series_dir)
|
|
req = OHLCVRequest(
|
|
symbol=args.symbol,
|
|
timeframe=args.timeframe,
|
|
start=datetime.fromisoformat(args.start),
|
|
end=datetime.fromisoformat(args.end),
|
|
exchange=args.exchange,
|
|
)
|
|
ohlcv = loader.load(req)
|
|
print(f"OHLCV loaded: {len(ohlcv)} bars from {ohlcv.index[0]} to {ohlcv.index[-1]}")
|
|
|
|
llm = LLMClient(
|
|
openrouter_api_key=settings.openrouter_api_key.get_secret_value(),
|
|
model_tier_s=settings.llm_model_tier_s,
|
|
model_tier_a=settings.llm_model_tier_a,
|
|
model_tier_b=settings.llm_model_tier_b,
|
|
model_tier_c=settings.llm_model_tier_c,
|
|
model_tier_d=settings.llm_model_tier_d,
|
|
openrouter_base_url=settings.openrouter_base_url,
|
|
)
|
|
|
|
cfg = RunConfig(
|
|
run_name=args.name,
|
|
population_size=args.population_size,
|
|
n_generations=args.n_generations,
|
|
elite_k=args.elite_k,
|
|
tournament_k=args.tournament_k,
|
|
p_crossover=args.p_crossover,
|
|
seed=args.seed,
|
|
model_tier=ModelTier.C,
|
|
symbol=args.symbol,
|
|
timeframe=args.timeframe,
|
|
fees_bp=args.fees_bp,
|
|
n_trials_dsr=args.n_trials_dsr,
|
|
db_path=settings.db_path,
|
|
prompt_mutation_weight=args.prompt_mutation_weight,
|
|
fees_eat_alpha_threshold=args.fees_eat_alpha_threshold,
|
|
flat_too_long_threshold=args.flat_too_long_threshold,
|
|
undertrading_threshold=args.undertrading_threshold,
|
|
fitness_hard_kill_findings=_resolve_hard_kill(args),
|
|
fitness_adversarial_soft_penalty=args.fitness_soft_penalty,
|
|
wfa_train_split=args.wfa_train_split,
|
|
wfa_top_k=args.wfa_top_k,
|
|
eval_oos_during_loop=args.eval_oos_during_loop,
|
|
fitness_combined_alpha=args.fitness_combined_alpha,
|
|
prompt_library=prompt_library,
|
|
llm_concurrency=args.llm_concurrency,
|
|
)
|
|
|
|
run_id = run_phase1(cfg, ohlcv=ohlcv, llm=llm)
|
|
print(f"Run completed: {run_id}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|