chore(reset): v2.0.0 — storico certificato Deribit mainnet, ripartenza pulita
Reset del progetto su fondamenta verificate dopo la scoperta che l'intera libreria "validata OOS" era artefatto di feed contaminato (print fantasma del feed Cerbero TESTNET + storico Binance/USDT). - Storico ricostruito da Deribit MAINNET (ccxt pubblico, tokenless) e CERTIFICATO (certify_feed.py): BTC/ETH puliti su TUTTA la storia (mediana 2-6 bps vs Coinbase USD), integrita' OHLC + coerenza resample (maxΔ 0.00) + cross-venue OK. Alt esclusi (illiquidi/divergenti: LTC/DOGE 50-82% barre flat; XRP/BNB non certificabili). - Verdetto sul feed pulito: FADE / PAIRS / XS01 / TSM01 morti (ogni portafoglio Sharpe -2.3..-3.0, DD ~40%); solo SH01 e frammenti HONEST con segnale residuo, da ri-validare in isolamento. - Cleanup "restart pulito": strategie, stack live (src/live, src/portfolio, runner/executor, yml, docker), ~100 script ricerca/gate, waste/games/ portfolios, dati non certificati + cache e 60+ diari -> archiviati in Old/ (preservati, non cancellati). Diario consolidato in un unico documento. - Skeleton ricerca tenuto: Strategy ABC + indicatori + src/fractal + src/backtest/engine + load_data; tool dati certificati (rebuild_history, certify_feed, audit_feed, multi_source_check). - Universo dati ATTIVO: solo BTC/ETH (5m/15m/1h); guardrail fisico (load_data su alt -> FileNotFoundError). Esecuzione DISABILITATA, conto flat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Adriano Dal Pastro
@@ -0,0 +1,266 @@
|
||||
"""ML01 — Squeeze + GBM (Gradient Boosting Machine) Walk-Forward.
|
||||
|
||||
Strategia ibrida: squeeze breakout come pre-filtro (QUANDO tradare),
|
||||
GradientBoosting su features strutturali come conferma (QUALE direzione).
|
||||
|
||||
Pipeline:
|
||||
1. Rileva squeeze release (Bollinger esce da Keltner)
|
||||
2. Estrai 44 features dalla finestra (structural multi-window + squeeze
|
||||
metadata + price position + ATR + momentum breakout)
|
||||
3. GBM walk-forward: train su 50% rolling, step 10%, predice direzione
|
||||
4. Trade solo se ML ha confidenza ≥ ml_threshold
|
||||
|
||||
IN:
|
||||
- OHLCV DataFrame
|
||||
- Parametri: bb_window (14), sq_threshold (0.8), brk_bars (3),
|
||||
ml_threshold (0.70), leverage (3), position_pct (0.15)
|
||||
|
||||
OUT:
|
||||
- BacktestResult con metriche walk-forward (no data leakage)
|
||||
- Solo periodo di test (seconda metà dati)
|
||||
|
||||
Risultati tipici:
|
||||
ETH 15m bb14 ml=0.70: 76.9% acc, 1213 trades, DD 4.2%, €13.78/day
|
||||
BTC 15m bb14 ml=0.70: 78.8% acc, 1964 trades, DD 7.0%, €5.51/day
|
||||
BTC 1h bb14 ml=0.70: 77.3% acc, 617 trades, DD 6.7%, €3.85/day
|
||||
|
||||
Note:
|
||||
- GBM = GradientBoostingClassifier di scikit-learn
|
||||
- Walk-forward: nessun look-ahead, train sempre prima di test
|
||||
- Il baseline squeeze puro ha accuracy più alta (~79.5%) ma DD peggiore
|
||||
- Il valore del ML è filtrare breakout deboli → DD ridotto
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import GradientBoostingClassifier
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
|
||||
from src.strategies.base import Strategy, Signal, BacktestResult, YearlyStats, TF_MINUTES
|
||||
from src.strategies.indicators import keltner_ratio, detect_squeezes
|
||||
from src.data.downloader import load_data
|
||||
|
||||
|
||||
def _build_features(df: pd.DataFrame, i: int, squeeze_info: dict) -> np.ndarray | None:
|
||||
"""44 features per il punto di squeeze release."""
|
||||
if i < 100:
|
||||
return None
|
||||
o, h, l, c, v = (df["open"].values, df["high"].values, df["low"].values,
|
||||
df["close"].values, df["volume"].values)
|
||||
feats = []
|
||||
for w in [12, 24, 48]:
|
||||
wc, wo = c[i-w:i], o[i-w:i]
|
||||
wh, wl, wv = h[i-w:i], l[i-w:i], v[i-w:i]
|
||||
mn, mx = wl.min(), max(wh.max(), wc.max())
|
||||
rng = mx - mn if mx - mn > 0 else 1e-10
|
||||
total = np.where(wh - wl == 0, 1e-10, wh - wl)
|
||||
body = np.abs(wc - wo) / total
|
||||
direction = np.sign(wc - wo)
|
||||
log_c = np.log(np.where(wc == 0, 1e-10, wc))
|
||||
rets = np.diff(log_c)
|
||||
v_mean = np.mean(wv)
|
||||
feats.extend([
|
||||
np.mean(rets) if len(rets) > 0 else 0,
|
||||
np.std(rets) if len(rets) > 0 else 0,
|
||||
np.sum(rets) if len(rets) > 0 else 0,
|
||||
float(pd.Series(rets).skew()) if len(rets) > 2 else 0,
|
||||
float(pd.Series(rets).kurtosis()) if len(rets) > 3 else 0,
|
||||
np.mean(body), np.std(body),
|
||||
np.mean(direction), np.mean(direction[-min(3, w):]),
|
||||
(wc[-1] - mn) / rng,
|
||||
wv[-1] / v_mean if v_mean > 0 else 1,
|
||||
np.corrcoef(rets[:-1], rets[1:])[0, 1] if len(rets) > 1 and np.std(rets) > 0 else 0,
|
||||
])
|
||||
sq = squeeze_info
|
||||
feats.extend([
|
||||
sq["dur"], sq["dur"] / 24, sq["kcr_at_release"],
|
||||
v[i-1] / sq.get("avg_vol", 1) if sq.get("avg_vol", 0) > 0 else 1,
|
||||
np.mean(v[i:min(i+3, len(v))]) / sq.get("avg_vol", 1) if sq.get("avg_vol", 0) > 0 else 1,
|
||||
])
|
||||
h48, l48 = np.max(h[max(0, i-48):i]), np.min(l[max(0, i-48):i])
|
||||
r48 = h48 - l48
|
||||
feats.append((c[i-1] - l48) / r48 if r48 > 0 else 0.5)
|
||||
tr = np.maximum(h[i-14:i] - l[i-14:i],
|
||||
np.maximum(np.abs(h[i-14:i] - np.roll(c[i-14:i], 1)),
|
||||
np.abs(l[i-14:i] - np.roll(c[i-14:i], 1))))
|
||||
feats.append(np.mean(tr[1:]) / c[i-1] if c[i-1] > 0 else 0)
|
||||
feats.append((c[i] - c[i-1]) / c[i-1] if c[i-1] > 0 else 0)
|
||||
return np.nan_to_num(np.array(feats), nan=0, posinf=1e6, neginf=-1e6)
|
||||
|
||||
|
||||
class SqueezeGBM(Strategy):
|
||||
name = "ML01_squeeze_gbm"
|
||||
description = "Squeeze + GBM walk-forward — ML filtra breakout deboli"
|
||||
default_assets = ["BTC", "ETH"]
|
||||
default_timeframes = ["15m", "1h"]
|
||||
fee_ml = 0.001
|
||||
|
||||
def generate_signals(self, df, ts, **params):
|
||||
raise NotImplementedError("ML01 usa backtest custom con walk-forward")
|
||||
|
||||
def backtest(self, asset: str, tf: str, hold: int = 3, **params) -> BacktestResult | None:
|
||||
bb_w = params.get("bb_window", 14)
|
||||
sq_thr = params.get("sq_threshold", 0.8 if tf == "1h" else 0.9)
|
||||
brk = params.get("brk_bars", hold)
|
||||
ml_thr = params.get("ml_threshold", 0.70)
|
||||
lev = params.get("leverage", self.leverage)
|
||||
pos = params.get("position_pct", self.position_size)
|
||||
|
||||
df = load_data(asset, tf)
|
||||
close = df["close"].values
|
||||
high = df["high"].values
|
||||
low = df["low"].values
|
||||
volume = df["volume"].values
|
||||
n = len(df)
|
||||
ts = pd.to_datetime(df["timestamp"], unit="ms", utc=True)
|
||||
|
||||
kcr = keltner_ratio(close, high, low, bb_w)
|
||||
raw_events = detect_squeezes(close, high, low, kcr, sq_thr)
|
||||
|
||||
# Aggiungi avg_vol a ogni evento
|
||||
events = []
|
||||
for ev in raw_events:
|
||||
ev["avg_vol"] = float(np.mean(volume[ev["sq_start"]:ev["idx"]]))
|
||||
events.append(ev)
|
||||
|
||||
X_all, y_all, ev_all = [], [], []
|
||||
for ev in events:
|
||||
i = ev["idx"]
|
||||
if i + brk >= n or i < 100:
|
||||
continue
|
||||
feats = _build_features(df, i, ev)
|
||||
if feats is None:
|
||||
continue
|
||||
actual_ret = (close[i + brk - 1] - close[i - 1]) / close[i - 1]
|
||||
X_all.append(feats)
|
||||
y_all.append(1 if actual_ret > 0 else 0)
|
||||
ev_all.append(ev)
|
||||
|
||||
if len(X_all) < 50:
|
||||
return None
|
||||
|
||||
X, y = np.array(X_all), np.array(y_all)
|
||||
TRAIN_SIZE = max(int(len(X) * 0.5), 50)
|
||||
STEP_SIZE = max(int(len(X) * 0.1), 10)
|
||||
|
||||
yearly: dict[int, dict] = {}
|
||||
capital = float(self.initial_capital)
|
||||
peak = capital
|
||||
max_dd = 0.0
|
||||
total_bars = 0
|
||||
all_t = all_w = 0
|
||||
|
||||
start = 0
|
||||
while start + TRAIN_SIZE + STEP_SIZE <= len(X):
|
||||
train_end = start + TRAIN_SIZE
|
||||
test_end = min(train_end + STEP_SIZE, len(X))
|
||||
X_tr, y_tr = X[start:train_end], y[start:train_end]
|
||||
X_te = X[train_end:test_end]
|
||||
|
||||
if len(np.unique(y_tr)) < 2:
|
||||
start += STEP_SIZE
|
||||
continue
|
||||
|
||||
scaler = StandardScaler()
|
||||
X_tr_s = scaler.fit_transform(X_tr)
|
||||
X_te_s = scaler.transform(X_te)
|
||||
|
||||
model = GradientBoostingClassifier(
|
||||
n_estimators=150, max_depth=4, min_samples_leaf=10,
|
||||
learning_rate=0.05, subsample=0.8, random_state=42,
|
||||
)
|
||||
model.fit(X_tr_s, y_tr)
|
||||
up_idx = list(model.classes_).index(1) if 1 in model.classes_ else -1
|
||||
if up_idx < 0:
|
||||
start += STEP_SIZE
|
||||
continue
|
||||
|
||||
for j in range(len(X_te)):
|
||||
proba = model.predict_proba(X_te_s[j:j+1])[0]
|
||||
p_up = proba[up_idx]
|
||||
ev = ev_all[train_end + j]
|
||||
i = ev["idx"]
|
||||
actual_ret = (close[i + brk - 1] - close[i - 1]) / close[i - 1]
|
||||
|
||||
if p_up >= ml_thr:
|
||||
direction = 1
|
||||
elif p_up <= (1 - ml_thr):
|
||||
direction = -1
|
||||
else:
|
||||
continue
|
||||
|
||||
is_correct = (direction == 1 and actual_ret > 0) or (direction == -1 and actual_ret < 0)
|
||||
trade_ret = actual_ret * direction
|
||||
net = trade_ret * lev - self.fee_ml * 2 * lev
|
||||
capital += capital * pos * net
|
||||
capital = max(capital, 10)
|
||||
if capital > peak:
|
||||
peak = capital
|
||||
dd = (peak - capital) / peak
|
||||
max_dd = max(max_dd, dd)
|
||||
total_bars += brk
|
||||
|
||||
all_t += 1
|
||||
if is_correct:
|
||||
all_w += 1
|
||||
|
||||
year = ts.iloc[i].year
|
||||
if year not in yearly:
|
||||
yearly[year] = {"w": 0, "t": 0, "pnl": 0.0}
|
||||
yearly[year]["t"] += 1
|
||||
if is_correct:
|
||||
yearly[year]["w"] += 1
|
||||
yearly[year]["pnl"] += net * self.initial_capital
|
||||
|
||||
start += STEP_SIZE
|
||||
|
||||
if all_t == 0:
|
||||
return None
|
||||
|
||||
yearly_stats = [
|
||||
YearlyStats(year=y, trades=d["t"], wins=d["w"], pnl=d["pnl"])
|
||||
for y, d in sorted(yearly.items())
|
||||
]
|
||||
|
||||
return BacktestResult(
|
||||
strategy_name=self.name,
|
||||
asset=asset,
|
||||
timeframe=tf,
|
||||
params={"bb_w": bb_w, "sq_thr": sq_thr, "ml_thr": ml_thr,
|
||||
"brk": brk, "lev": lev, "pos": pos},
|
||||
trades=all_t,
|
||||
wins=all_w,
|
||||
pnl=sum(d["pnl"] for d in yearly.values()),
|
||||
capital=capital,
|
||||
initial_capital=self.initial_capital,
|
||||
max_dd=max_dd * 100,
|
||||
time_in_market_pct=total_bars / n * 100,
|
||||
avg_trade_duration_h=brk * TF_MINUTES.get(tf, 60) / 60,
|
||||
years_active=len(yearly),
|
||||
yearly=yearly_stats,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
strategy = SqueezeGBM()
|
||||
print("Training ML models...\n")
|
||||
results = []
|
||||
for asset in ["ETH", "BTC"]:
|
||||
for tf in ["15m", "1h"]:
|
||||
for ml_thr in [0.65, 0.70]:
|
||||
r = strategy.backtest(asset, tf, ml_threshold=ml_thr)
|
||||
if r and r.trades >= 20:
|
||||
r.strategy_name = f"ML01 {asset} {tf} ml={ml_thr}"
|
||||
results.append(r)
|
||||
results.sort(key=lambda r: r.accuracy, reverse=True)
|
||||
|
||||
print(f"{'=' * 120}")
|
||||
print(f" ML01 SQUEEZE+GBM — RISULTATI")
|
||||
print(f"{'=' * 120}")
|
||||
for r in results:
|
||||
r.print_summary()
|
||||
if results:
|
||||
results[0].print_yearly()
|
||||
Reference in New Issue
Block a user