feat: strategie 1-10, framework analisi frattale, download dati storici BTC/ETH
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+18
@@ -0,0 +1,18 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
.venv/
|
||||
.env
|
||||
!.env.example
|
||||
.vscode/
|
||||
.idea/
|
||||
.DS_Store
|
||||
data/raw/
|
||||
data/processed/
|
||||
*.log
|
||||
*.pkl
|
||||
*.pt
|
||||
*.pth
|
||||
notebooks/.ipynb_checkpoints/
|
||||
@@ -0,0 +1 @@
|
||||
3.11
|
||||
@@ -0,0 +1,92 @@
|
||||
# 2026-05-26 — Giorno 1: Setup e download dati
|
||||
|
||||
### 23:15 — Inizializzazione progetto
|
||||
|
||||
**Cosa:** creato struttura progetto Python con uv, git init, moduli base
|
||||
**Perché:** servono fondamenta solide per ricerca iterativa. Struttura: src/data (download/storage), src/fractal (analisi pattern), src/strategies (strategie trading), src/backtest (engine di test), src/nn (reti neurali), src/utils (utility)
|
||||
**Atteso:** progetto funzionante con dipendenze installate
|
||||
**Reale:** in corso
|
||||
|
||||
### 23:20 — Verifica Cerbero MCP
|
||||
|
||||
**Cosa:** testato accesso API Cerbero su cerbero-mcp.tielogic.xyz per dati storici crypto
|
||||
**Perché:** verificare se può fornire dati dal 2018
|
||||
**Atteso:** dati storici cross-exchange (consensus multi-sorgente)
|
||||
**Reale:** API funziona, dati recenti OK. Per storico 2018→oggi uso Binance via ccxt (copertura temporale maggiore, dati 1m disponibili)
|
||||
|
||||
### 23:25 — Script download dati
|
||||
|
||||
**Cosa:** creato src/data/downloader.py — scarica OHLCV da Binance per BTC/USDT e ETH/USDT su 4 timeframe (1m, 5m, 15m, 1h) dal 2018-01-01 a oggi. Formato: parquet (veloce, compresso). Supporta resume in caso di interruzione.
|
||||
**Perché:** dati locali per iterazione veloce. Parquet per caricamento istantaneo vs CSV.
|
||||
**Atteso:** ~4.2M candele 1m per asset, ~70K candele 1h per asset. Download 1m stimato ~30-60 min per asset.
|
||||
**Reale:** in corso (avvio download)
|
||||
|
||||
### Metriche target
|
||||
|
||||
| Metrica | Valore target |
|
||||
|---|---|
|
||||
| Accuratezza previsione direzione | >80% |
|
||||
| ROI annuo (con fees) | >30% |
|
||||
| Capitale iniziale | €1.000 |
|
||||
| Obiettivo giornaliero (steady state) | €50/giorno |
|
||||
| Fee considerate | 0.1% maker/taker (Binance standard) |
|
||||
|
||||
### Approccio
|
||||
|
||||
1. **Focus frattali**: pattern ricorrenti multi-scala, non indicatori classici
|
||||
2. **Multi-timeframe**: conferma segnali su scale diverse (1m→1h)
|
||||
3. **Fuori dagli schemi**: combinare Fourier, auto-similarità, entropia di Shannon, dimensione frattale di Hausdorff
|
||||
4. **Pragmatismo**: se un approccio non funziona, pivotare veloce. Misurare tutto.
|
||||
|
||||
### 23:40 — Analisi baseline completata
|
||||
|
||||
**Cosa:** analisi distribuzione pattern discreti (U/D/0) su BTC 1h, 73.557 candele 2018→2026
|
||||
**Perché:** baseline per capire se pattern candlestick semplici hanno potere predittivo
|
||||
**Atteso:** almeno alcuni pattern con >60% accuracy direzionale
|
||||
**Reale:** NESSUN pattern supera 55% accuracy a 24h. Max: DDD→58.5% a 1h, ma scende a 53.6% a 24h. Pattern discreti semplici NON hanno edge significativo.
|
||||
**Lezione:**
|
||||
- Distribuzione candele quasi uniforme: UP 42.1%, DOWN 40.8%, DOJI 17.1%
|
||||
- 1080 pattern unici (esattamente lo spazio teorico 3^3 + 3^4 + 3^5 + 3^6)
|
||||
- Pattern alternanti (UDU, DUD) più frequenti → mercato mean-reverting a scala oraria
|
||||
- Serve andare oltre: features continue (body/shadow ratios, volume), Fourier, self-similarity, ML
|
||||
**Benchmark:** Buy & Hold test period: +110%, annualizzato 34.3%, Sharpe 0.52
|
||||
|
||||
### 23:40 — Download dati completato
|
||||
|
||||
**Cosa:** scaricati dati storici BTC + ETH, 3 timeframe (5m, 15m, 1h) dal 2018-01-01
|
||||
**Fonti:** Cerbero MCP (Deribit) per set 2018+, Binance/ccxt per gap iniziale
|
||||
**Reale:**
|
||||
|
||||
| Asset | TF | Candele | Peso |
|
||||
|-------|-----|---------|---------|
|
||||
| BTC | 5m | 882.630 | 23.6 MB |
|
||||
| BTC | 15m | 294.213 | 9.1 MB |
|
||||
| BTC | 1h | 73.557 | 2.8 MB |
|
||||
| ETH | 5m | 882.312 | 19.4 MB |
|
||||
| ETH | 15m | 294.107 | 7.9 MB |
|
||||
| ETH | 1h | 73.531 | 2.5 MB |
|
||||
|
||||
**Note:** 1m rimandato (troppo pesante per primo round). 5m sufficiente per analisi fine-grained.
|
||||
|
||||
### 23:50 — Strategia 3: Fourier projection — FALLITA
|
||||
|
||||
**Cosa:** proiezione FFT naive su BTC 1h (ispirata dal paper Pythagoras)
|
||||
**Atteso:** almeno 55% accuracy direzionale
|
||||
**Reale:** 49.8% accuracy (=random), -99.9% return. Tutte le varianti parametri (W=144-588, N=5-50) identicamente pessime.
|
||||
**Lezione:** FFT extrapola sinusoidi che non continuano fuori finestra. Il paper Pythagoras non fa proiezione naive — usa trasformazioni geometriche (centro inversione, riflessioni). Approccio sbagliato, non la tecnica in sé.
|
||||
|
||||
### 00:05 — Strategia 4: Regime-aware fractal ML — PARZIALE SUCCESSO
|
||||
|
||||
**Cosa:** RandomForest + GradientBoosting su features frattali (Hurst, fractal dim, self-similarity, vol ratio, momentum, candle patterns)
|
||||
**Atteso:** >55% accuracy con ML su features ricche
|
||||
**Reale:**
|
||||
- RF: 38% accuracy (3 classi), pochissimi segnali ad alta confidenza (8 @ thr 0.55 → 100% acc)
|
||||
- GB: 41.6% accuracy, MA a threshold sweep:
|
||||
- thr=0.65: **63.6% accuracy**, 66 segnali, **+5.7% return**, Sharpe 0.21
|
||||
- thr=0.80: **80% accuracy**, 5 segnali
|
||||
- Feature importance: volatility (21%) > momentum (10%) > fractal features (6%)
|
||||
**Lezione:**
|
||||
1. Classificazione 3-classi troppo dispersiva → switch a binario
|
||||
2. Features frattali contribuiscono ma non dominano — serve combinarle meglio
|
||||
3. Trade filtering ad alta confidenza funziona: meno trade, più precisi
|
||||
4. Direzione giusta: ML su features frattali produce edge reale, anche se piccolo
|
||||
@@ -0,0 +1,72 @@
|
||||
# 2026-05-27 — Giorno 2: Strategie e risultati
|
||||
|
||||
### 00:00 — Strategia 5: Enhanced fractal (DATA LEAKAGE trovata!)
|
||||
|
||||
**Cosa:** GBM con features multi-window (4 finestre × 9 features), classification binaria, BTC + ETH su 3 lookahead
|
||||
**Atteso:** miglioramento rispetto a #4 con più features e classificazione binaria
|
||||
**Reale:** risultati iniziali troppo belli (84.5% accuracy BTC, 85% ETH) → **DATA LEAKAGE TROVATA**
|
||||
**Bug:** `returns[i-w : i]` includeva `returns[i-1]` che usa `close[i]` (1 candle nel futuro)
|
||||
**Fix:** cambiato a `returns[i-w : i-1]` — re-run in corso
|
||||
**Lezione:** SEMPRE verificare che nessuna feature usi dati oltre il timestamp di decisione. Returns ha off-by-one insidioso.
|
||||
|
||||
### 00:10 — Strategia 6: Structural Pattern KNN + GBM
|
||||
|
||||
**Cosa:** features normalizzate da finestra OHLC (close norm, body, direction, shadow, volume), con KNN e GBM
|
||||
**Reale:**
|
||||
- KNN: max 55.9% accuracy (K=100, thr=0.65) → edge minimo
|
||||
- **GBM: thr=0.65, 795 trades, 58.6% accuracy, +57.5% return** ← MIGLIOR SINGOLO (senza leakage)
|
||||
**Lezione:** features strutturali normalizzate battono features raw. GBM >> KNN per questo tipo di dati.
|
||||
|
||||
### 00:20 — Strategia 7: LSTM
|
||||
|
||||
**Cosa:** LSTM (2 layer, 64 hidden, dropout 0.3) su sequenze di 48 candele × 6 features per-candle
|
||||
**Reale:**
|
||||
- BTC test: 51.9% base, ma thr=0.60: **58.4% accuracy, 214 trades, +4.3%**
|
||||
- BTC thr=0.65: **64.3% accuracy** ma solo 14 trade
|
||||
- ETH: 52.6% base, thr=0.55: **54.5%, +19.9%**
|
||||
- Training su CPU (CUDA non disponibile) → 14 epoch con early stopping
|
||||
**Lezione:** LSTM cattura pattern ma non aggiunge molto rispetto a GBM su features ingegnerizzate. Edge comparabile (~58-64%) con molte meno features. CPU training lento.
|
||||
|
||||
### 00:30 — Strategia 8: Ensemble multi-timeframe
|
||||
|
||||
**Cosa:** 3 modelli (structural 1h, multi-tf 15m, combined) con voting e media probabilità
|
||||
**Reale:**
|
||||
- M1_structural thr=0.65: 829 trades, **58.3% acc, +53.4%, 17.8% annualizzato**
|
||||
- M2_multi_tf: scarso (15m features da sole non bastano)
|
||||
- Ensemble agree≥2, thr=0.65: 520 trades, **59.2% accuracy, +19.9%**
|
||||
- Ensemble agree≥3, thr=0.65: 27 trades, **70.4% accuracy** ma pochi trade
|
||||
**Lezione:**
|
||||
1. Multi-timeframe aggiunge margine (+1% accuracy nell'ensemble)
|
||||
2. Consensus forte (3/3) raggiunge 70%+ ma troppo pochi trade
|
||||
3. Il collo di bottiglia è la frequenza segnali ad alta confidenza
|
||||
|
||||
### 00:45 — Strategie 9 e 10 in esecuzione
|
||||
|
||||
- **#9**: Walk-forward validation con GBM, features combinate structural+fractal
|
||||
- **#10**: High precision (target >80%) con ensemble 5 modelli (2×GBM, RF, ExtraTrees, LogReg), consensus voting, leva 3x
|
||||
|
||||
### Riepilogo risultati validi (no leakage)
|
||||
|
||||
| # | Nome | Accuracy | Return | Ann. | Trades | Note |
|
||||
|---|------|----------|--------|------|--------|------|
|
||||
| 6 | GBM structural | 58.6% | +57.5% | ~20% | 795 | Miglior singolo |
|
||||
| 8/M1 | Structural WF | 58.3% | +53.4% | 17.8% | 829 | Robusto |
|
||||
| 8/ens | Ensemble 2/3 | 59.2% | +19.9% | 7.2% | 520 | Più filtrato |
|
||||
| 8/ens3 | Ensemble 3/3 | 70.4% | +11.3% | 4.2% | 27 | Alta acc, pochi trade |
|
||||
| 4 | GBM fractal | 63.6% | +5.7% | ~3% | 66 | Pochi ma precisi |
|
||||
| 7 | LSTM | 58.4% | +4.3% | 3.1% | 214 | Comparabile a GBM |
|
||||
|
||||
### Analisi gap verso target
|
||||
|
||||
| Target | Attuale | Gap |
|
||||
|--------|---------|-----|
|
||||
| Accuracy >80% | max 70.4% (ens 3/3) | serve +10% |
|
||||
| ROI annuo >30% | max ~20% (structural) | serve +10% |
|
||||
| €50/giorno da €1000 | richiede ~5% daily | richiede crescita capitale su 6 mesi |
|
||||
|
||||
### Prossimi passi
|
||||
|
||||
1. Verificare strategia 5 corretta (senza leakage)
|
||||
2. Risultati strategia 9 (walk-forward) e 10 (high precision ensemble)
|
||||
3. Se accuracy ancora insufficiente: provare features da 5m aggregati, o approach completamente diverso (reinforcement learning?)
|
||||
4. Valutare combinazione: multi-asset (BTC+ETH) per diversificazione
|
||||
@@ -0,0 +1,17 @@
|
||||
# Diario di Ricerca — PythagorasGoal
|
||||
|
||||
Registro cronologico di ogni passo del progetto: decisioni, esperimenti, risultati attesi e reali.
|
||||
|
||||
## Formato entry
|
||||
|
||||
Ogni entry segue il formato:
|
||||
|
||||
```
|
||||
### YYYY-MM-DD HH:MM — Titolo
|
||||
|
||||
**Cosa:** descrizione azione
|
||||
**Perché:** motivazione
|
||||
**Atteso:** risultato previsto
|
||||
**Reale:** risultato effettivo
|
||||
**Note:** osservazioni, lezioni apprese
|
||||
```
|
||||
@@ -0,0 +1,28 @@
|
||||
[project]
|
||||
name = "pythagoras-goal"
|
||||
version = "0.1.0"
|
||||
description = "Fractal pattern recognition and prediction for crypto trading (BTC, ETH)"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"pandas>=2.0",
|
||||
"numpy>=1.24",
|
||||
"requests>=2.31",
|
||||
"ccxt>=4.0",
|
||||
"pyarrow>=15.0",
|
||||
"scipy>=1.11",
|
||||
"scikit-learn>=1.3",
|
||||
"torch>=2.0",
|
||||
"matplotlib>=3.7",
|
||||
"tqdm>=4.65",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"pytest-asyncio>=0.24",
|
||||
"ipython>=8.0",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
asyncio_mode = "auto"
|
||||
@@ -0,0 +1,110 @@
|
||||
"""Analisi baseline: distribuzione pattern frattali e prima strategia naive."""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.patterns import encode_candles, find_patterns, pattern_frequency
|
||||
from src.backtest.engine import run_backtest, BacktestResult
|
||||
|
||||
print("=" * 60)
|
||||
print(" ANALISI BASELINE — BTC 1H")
|
||||
print("=" * 60)
|
||||
|
||||
df = load_data("BTC", "1h")
|
||||
print(f"\nDati: {len(df)} candele [{df['datetime'].iloc[0]} → {df['datetime'].iloc[-1]}]")
|
||||
|
||||
# 1. Distribuzione pattern
|
||||
print("\n--- DISTRIBUZIONE PATTERN (3-6 candele) ---")
|
||||
candle_types = encode_candles(df)
|
||||
unique, counts = np.unique(candle_types, return_counts=True)
|
||||
type_map = {-1: "DOWN", 0: "DOJI", 1: "UP"}
|
||||
for t, c in zip(unique, counts):
|
||||
print(f" {type_map[t]}: {c} ({c/len(df)*100:.1f}%)")
|
||||
|
||||
patterns = find_patterns(df, min_len=3, max_len=6)
|
||||
freq = pattern_frequency(patterns)
|
||||
print(f"\nPattern unici: {len(freq)}")
|
||||
print(f"\nTop 20 pattern più frequenti:")
|
||||
print(freq.head(20).to_string(index=False))
|
||||
|
||||
# 2. Analisi predittiva: dopo ogni pattern, il prezzo sale o scende?
|
||||
print("\n\n--- ANALISI PREDITTIVA PER PATTERN ---")
|
||||
print("Per ogni pattern: % volte che il prezzo sale nelle N candele successive")
|
||||
|
||||
LOOKAHEAD = [1, 3, 6, 12, 24]
|
||||
top_patterns = freq.head(30)["pattern"].tolist()
|
||||
|
||||
results = []
|
||||
for code in top_patterns:
|
||||
matching = [p for p in patterns if p.code == code]
|
||||
if len(matching) < 50:
|
||||
continue
|
||||
|
||||
row = {"pattern": code, "count": len(matching)}
|
||||
for ahead in LOOKAHEAD:
|
||||
ups = 0
|
||||
valid = 0
|
||||
for p in matching:
|
||||
future_idx = p.end_idx + ahead
|
||||
if future_idx >= len(df):
|
||||
continue
|
||||
valid += 1
|
||||
if df["close"].iloc[future_idx] > df["close"].iloc[p.end_idx - 1]:
|
||||
ups += 1
|
||||
if valid > 0:
|
||||
row[f"up_{ahead}h"] = round(ups / valid * 100, 1)
|
||||
else:
|
||||
row[f"up_{ahead}h"] = None
|
||||
results.append(row)
|
||||
|
||||
pred_df = pd.DataFrame(results)
|
||||
print(pred_df.to_string(index=False))
|
||||
|
||||
# 3. Strategia naive: compra quando il pattern più bullish si presenta
|
||||
print("\n\n--- STRATEGIA 1: PATTERN BULLISH NAIVE ---")
|
||||
# Trova pattern con up_24h > 55%
|
||||
bullish_patterns = pred_df[pred_df["up_24h"] > 55]["pattern"].tolist()
|
||||
bearish_patterns = pred_df[pred_df["up_24h"] < 45]["pattern"].tolist()
|
||||
print(f"Pattern bullish (>55% up in 24h): {len(bullish_patterns)}")
|
||||
print(f"Pattern bearish (<45% up in 24h): {len(bearish_patterns)}")
|
||||
|
||||
# Genera segnali
|
||||
signals = pd.Series(0, index=df.index)
|
||||
all_patterns = find_patterns(df, min_len=3, max_len=6)
|
||||
for p in all_patterns:
|
||||
if p.code in bullish_patterns:
|
||||
signals.iloc[p.end_idx - 1] = 1
|
||||
elif p.code in bearish_patterns:
|
||||
if signals.iloc[p.end_idx - 1] == 0:
|
||||
signals.iloc[p.end_idx - 1] = -1
|
||||
|
||||
# Train/test split: 70/30
|
||||
split_idx = int(len(df) * 0.7)
|
||||
train_df = df.iloc[:split_idx].reset_index(drop=True)
|
||||
test_df = df.iloc[split_idx:].reset_index(drop=True)
|
||||
train_signals = signals.iloc[:split_idx].reset_index(drop=True)
|
||||
test_signals = signals.iloc[split_idx:].reset_index(drop=True)
|
||||
|
||||
train_result = run_backtest(train_df, train_signals, initial_capital=1000, fee_pct=0.001)
|
||||
test_result = run_backtest(test_df, test_signals, initial_capital=1000, fee_pct=0.001)
|
||||
|
||||
print("\nRISULTATI TRAIN (70%):")
|
||||
for k, v in train_result.summary().items():
|
||||
print(f" {k}: {v}")
|
||||
|
||||
print("\nRISULTATI TEST (30%):")
|
||||
for k, v in test_result.summary().items():
|
||||
print(f" {k}: {v}")
|
||||
|
||||
# 4. Buy & Hold come benchmark
|
||||
print("\n\n--- BENCHMARK: BUY & HOLD ---")
|
||||
bh_signals = pd.Series(0, index=test_df.index)
|
||||
bh_signals.iloc[0] = 1 # Compra al primo candle
|
||||
bh_result = run_backtest(test_df, bh_signals, initial_capital=1000, fee_pct=0.001, max_hold_candles=len(test_df))
|
||||
print("Buy & Hold (test period):")
|
||||
for k, v in bh_result.summary().items():
|
||||
print(f" {k}: {v}")
|
||||
@@ -0,0 +1,110 @@
|
||||
"""Strategia 2: DTW pattern matching.
|
||||
Idea: per ogni finestra di N candele, cerca le K finestre più simili nel passato
|
||||
via DTW sui prezzi normalizzati. Se la maggioranza delle match passate è salita
|
||||
dopo, vai long. Se è scesa, vai short.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.similarity import dtw_distance
|
||||
from src.fractal.patterns import normalize_pattern_window
|
||||
from src.backtest.engine import run_backtest
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 2: DTW PATTERN MATCHING — BTC 1H")
|
||||
print("=" * 60)
|
||||
|
||||
df = load_data("BTC", "1h")
|
||||
close = df["close"].values
|
||||
|
||||
WINDOW = 12
|
||||
LOOKAHEAD = 6
|
||||
K_NEIGHBORS = 20
|
||||
LOOKBACK = 2000
|
||||
THRESHOLD = 0.65
|
||||
|
||||
split_idx = int(len(df) * 0.7)
|
||||
|
||||
def normalize_window(arr: np.ndarray) -> np.ndarray:
|
||||
mn, mx = arr.min(), arr.max()
|
||||
if mx - mn == 0:
|
||||
return np.zeros_like(arr)
|
||||
return (arr - mn) / (mx - mn)
|
||||
|
||||
def compute_returns(close_arr: np.ndarray, idx: int, ahead: int) -> float:
|
||||
if idx + ahead >= len(close_arr):
|
||||
return 0.0
|
||||
return (close_arr[idx + ahead] - close_arr[idx]) / close_arr[idx]
|
||||
|
||||
print(f"\nParametri: window={WINDOW}, lookahead={LOOKAHEAD}, K={K_NEIGHBORS}")
|
||||
print(f"Lookback: {LOOKBACK} candele, threshold: {THRESHOLD}")
|
||||
print(f"Train: 0→{split_idx}, Test: {split_idx}→{len(df)}")
|
||||
|
||||
signals = pd.Series(0, index=df.index)
|
||||
accuracies = []
|
||||
|
||||
step = 6
|
||||
test_range = range(split_idx, len(df) - LOOKAHEAD, step)
|
||||
total_steps = len(list(test_range))
|
||||
print(f"\nValutazione: {total_steps} punti (step={step})...")
|
||||
|
||||
for count, i in enumerate(test_range):
|
||||
if count % 500 == 0:
|
||||
print(f" Progresso: {count}/{total_steps} ({count/total_steps*100:.0f}%)")
|
||||
|
||||
current = normalize_window(close[i - WINDOW : i])
|
||||
|
||||
search_start = max(WINDOW, i - LOOKBACK)
|
||||
search_end = i - LOOKAHEAD
|
||||
|
||||
if search_end - search_start < K_NEIGHBORS:
|
||||
continue
|
||||
|
||||
distances = []
|
||||
for j in range(search_start, search_end):
|
||||
candidate = normalize_window(close[j - WINDOW : j])
|
||||
if len(candidate) != len(current):
|
||||
continue
|
||||
d = dtw_distance(current, candidate)
|
||||
future_ret = compute_returns(close, j, LOOKAHEAD)
|
||||
distances.append((d, future_ret))
|
||||
|
||||
if len(distances) < K_NEIGHBORS:
|
||||
continue
|
||||
|
||||
distances.sort(key=lambda x: x[0])
|
||||
top_k = distances[:K_NEIGHBORS]
|
||||
up_count = sum(1 for _, ret in top_k if ret > 0)
|
||||
up_ratio = up_count / K_NEIGHBORS
|
||||
|
||||
if up_ratio >= THRESHOLD:
|
||||
signals.iloc[i] = 1
|
||||
elif up_ratio <= (1 - THRESHOLD):
|
||||
signals.iloc[i] = -1
|
||||
|
||||
actual_ret = compute_returns(close, i, LOOKAHEAD)
|
||||
predicted_up = up_ratio >= THRESHOLD
|
||||
predicted_down = up_ratio <= (1 - THRESHOLD)
|
||||
if predicted_up:
|
||||
accuracies.append(1 if actual_ret > 0 else 0)
|
||||
elif predicted_down:
|
||||
accuracies.append(1 if actual_ret < 0 else 0)
|
||||
|
||||
print(f"\nSegnali generati: {(signals != 0).sum()}")
|
||||
print(f" Long: {(signals == 1).sum()}, Short: {(signals == -1).sum()}")
|
||||
|
||||
if accuracies:
|
||||
print(f"Accuratezza direzione: {np.mean(accuracies)*100:.1f}% su {len(accuracies)} segnali")
|
||||
|
||||
test_df = df.iloc[split_idx:].reset_index(drop=True)
|
||||
test_signals = signals.iloc[split_idx:].reset_index(drop=True)
|
||||
|
||||
result = run_backtest(test_df, test_signals, initial_capital=1000, fee_pct=0.001, max_hold_candles=LOOKAHEAD)
|
||||
|
||||
print("\nRISULTATI TEST:")
|
||||
for k, v in result.summary().items():
|
||||
print(f" {k}: {v}")
|
||||
@@ -0,0 +1,134 @@
|
||||
"""Strategia 3: Fourier decomposition e proiezione.
|
||||
Ispirata al paper Pythagoras Trading Prediction.
|
||||
Idea: scomponi il prezzo in componenti sinusoidali via FFT,
|
||||
ricostruisci con le N componenti più forti, proietta nel futuro.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from src.data.downloader import load_data
|
||||
from src.backtest.engine import run_backtest
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 3: FOURIER PROJECTION — BTC 1H")
|
||||
print("=" * 60)
|
||||
|
||||
df = load_data("BTC", "1h")
|
||||
close = df["close"].values
|
||||
n_total = len(close)
|
||||
|
||||
WINDOW = 588 # dal paper: 588 candele per l'indicatore H-C
|
||||
N_COMPONENTS = 25 # dal paper: 25 linee verticali
|
||||
LOOKAHEAD = 6
|
||||
STEP = 6
|
||||
|
||||
split_idx = int(n_total * 0.7)
|
||||
|
||||
def fourier_project(series: np.ndarray, n_components: int, ahead: int) -> np.ndarray:
|
||||
"""Ricostruisci serie con top-N componenti Fourier e proietta avanti."""
|
||||
n = len(series)
|
||||
detrended = series - np.linspace(series[0], series[-1], n)
|
||||
fft_vals = np.fft.fft(detrended)
|
||||
freqs = np.fft.fftfreq(n)
|
||||
|
||||
magnitudes = np.abs(fft_vals)
|
||||
magnitudes[0] = 0
|
||||
top_indices = np.argsort(magnitudes)[-n_components * 2:]
|
||||
|
||||
fft_filtered = np.zeros_like(fft_vals)
|
||||
fft_filtered[top_indices] = fft_vals[top_indices]
|
||||
|
||||
t_extended = np.arange(n + ahead)
|
||||
reconstruction = np.zeros(n + ahead)
|
||||
for idx in top_indices:
|
||||
amp = np.abs(fft_vals[idx]) / n
|
||||
phase = np.angle(fft_vals[idx])
|
||||
freq = freqs[idx]
|
||||
reconstruction += amp * np.cos(2 * np.pi * freq * t_extended / 1 + phase)
|
||||
|
||||
trend_slope = (series[-1] - series[0]) / n
|
||||
trend_extended = series[0] + trend_slope * t_extended
|
||||
reconstruction += trend_extended
|
||||
|
||||
return reconstruction
|
||||
|
||||
|
||||
print(f"\nParametri: window={WINDOW}, components={N_COMPONENTS}, lookahead={LOOKAHEAD}")
|
||||
print(f"Train: 0→{split_idx}, Test: {split_idx}→{n_total}")
|
||||
|
||||
signals = pd.Series(0, index=df.index)
|
||||
accuracies = []
|
||||
|
||||
test_range = range(max(split_idx, WINDOW), n_total - LOOKAHEAD, STEP)
|
||||
total_steps = len(list(test_range))
|
||||
print(f"Valutazione: {total_steps} punti (step={STEP})...")
|
||||
|
||||
for count, i in enumerate(test_range):
|
||||
if count % 500 == 0:
|
||||
print(f" Progresso: {count}/{total_steps} ({count/total_steps*100:.0f}%)")
|
||||
|
||||
window_data = close[i - WINDOW : i]
|
||||
projected = fourier_project(window_data, N_COMPONENTS, LOOKAHEAD)
|
||||
|
||||
current_price = close[i - 1]
|
||||
projected_price = projected[-1]
|
||||
change_pct = (projected_price - current_price) / current_price
|
||||
|
||||
if change_pct > 0.005:
|
||||
signals.iloc[i] = 1
|
||||
elif change_pct < -0.005:
|
||||
signals.iloc[i] = -1
|
||||
|
||||
actual_ret = (close[i + LOOKAHEAD - 1] - current_price) / current_price
|
||||
if signals.iloc[i] == 1:
|
||||
accuracies.append(1 if actual_ret > 0 else 0)
|
||||
elif signals.iloc[i] == -1:
|
||||
accuracies.append(1 if actual_ret < 0 else 0)
|
||||
|
||||
print(f"\nSegnali generati: {(signals != 0).sum()}")
|
||||
print(f" Long: {(signals == 1).sum()}, Short: {(signals == -1).sum()}")
|
||||
|
||||
if accuracies:
|
||||
print(f"Accuratezza direzione: {np.mean(accuracies)*100:.1f}% su {len(accuracies)} segnali")
|
||||
|
||||
test_df = df.iloc[split_idx:].reset_index(drop=True)
|
||||
test_signals = signals.iloc[split_idx:].reset_index(drop=True)
|
||||
|
||||
result = run_backtest(test_df, test_signals, initial_capital=1000, fee_pct=0.001, max_hold_candles=LOOKAHEAD)
|
||||
|
||||
print("\nRISULTATI TEST:")
|
||||
for k, v in result.summary().items():
|
||||
print(f" {k}: {v}")
|
||||
|
||||
# Varianti con parametri diversi
|
||||
print("\n\n--- VARIANTI PARAMETRI ---")
|
||||
for n_comp in [5, 10, 15, 25, 50]:
|
||||
for window in [144, 288, 588]:
|
||||
sigs = pd.Series(0, index=df.index)
|
||||
accs = []
|
||||
test_r = range(max(split_idx, window), n_total - LOOKAHEAD, STEP)
|
||||
for i in test_r:
|
||||
w = close[i - window : i]
|
||||
proj = fourier_project(w, n_comp, LOOKAHEAD)
|
||||
cp = close[i - 1]
|
||||
pp = proj[-1]
|
||||
ch = (pp - cp) / cp
|
||||
if ch > 0.005:
|
||||
sigs.iloc[i] = 1
|
||||
elif ch < -0.005:
|
||||
sigs.iloc[i] = -1
|
||||
ar = (close[i + LOOKAHEAD - 1] - cp) / cp
|
||||
if sigs.iloc[i] == 1:
|
||||
accs.append(1 if ar > 0 else 0)
|
||||
elif sigs.iloc[i] == -1:
|
||||
accs.append(1 if ar < 0 else 0)
|
||||
|
||||
if not accs:
|
||||
continue
|
||||
t_sigs = sigs.iloc[split_idx:].reset_index(drop=True)
|
||||
res = run_backtest(test_df, t_sigs, initial_capital=1000, fee_pct=0.001, max_hold_candles=LOOKAHEAD)
|
||||
acc = np.mean(accs) * 100
|
||||
print(f" W={window:3d} N={n_comp:2d} → acc={acc:.1f}% trades={res.total_trades} ret={res.total_return*100:+.1f}% sharpe={res.sharpe_ratio:.2f}")
|
||||
@@ -0,0 +1,231 @@
|
||||
"""Strategia 4: Regime-aware fractal ML.
|
||||
Combina:
|
||||
1. Hurst exponent per regime detection (trend vs mean-revert vs random)
|
||||
2. Feature engineering da indicatori frattali
|
||||
3. RandomForest per predizione direzione
|
||||
4. Trade filtering aggressivo (solo alta confidenza)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
|
||||
from sklearn.metrics import accuracy_score, classification_report
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.indicators import (
|
||||
hurst_exponent,
|
||||
fractal_dimension_higuchi,
|
||||
self_similarity_score,
|
||||
volatility_ratio,
|
||||
)
|
||||
from src.fractal.patterns import encode_candles, extract_body_ratios, extract_shadow_ratios
|
||||
from src.backtest.engine import run_backtest
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 4: REGIME-AWARE FRACTAL ML — BTC 1H")
|
||||
print("=" * 60)
|
||||
|
||||
df = load_data("BTC", "1h")
|
||||
close = df["close"].values
|
||||
n = len(close)
|
||||
|
||||
LOOKBACK = 48
|
||||
LOOKAHEAD = 6
|
||||
MIN_CONFIDENCE = 0.60
|
||||
|
||||
print(f"\nDati: {n} candele")
|
||||
print(f"Lookback: {LOOKBACK}, Lookahead: {LOOKAHEAD}")
|
||||
|
||||
# --- Feature engineering ---
|
||||
print("\nCalcolo features...")
|
||||
|
||||
features_list = []
|
||||
labels = []
|
||||
indices = []
|
||||
|
||||
returns = np.diff(np.log(np.where(close == 0, 1e-10, close)))
|
||||
candle_types = encode_candles(df)
|
||||
body_ratios = extract_body_ratios(df)
|
||||
shadow_ratios = extract_shadow_ratios(df)
|
||||
|
||||
for i in range(LOOKBACK, n - LOOKAHEAD, 3):
|
||||
if i % 5000 == 0:
|
||||
print(f" Feature extraction: {i}/{n}")
|
||||
|
||||
window = close[i - LOOKBACK : i]
|
||||
ret_window = returns[i - LOOKBACK : i - 1]
|
||||
|
||||
if len(ret_window) < 10:
|
||||
continue
|
||||
|
||||
h = hurst_exponent(ret_window, max_lag=min(len(ret_window) // 4, 20))
|
||||
fd = fractal_dimension_higuchi(ret_window, k_max=min(8, len(ret_window) // 4))
|
||||
|
||||
larger_window = close[max(0, i - LOOKBACK * 6) : i]
|
||||
ss = self_similarity_score(larger_window, min(LOOKBACK, len(larger_window)))
|
||||
vr = volatility_ratio(window, fast=12, slow=LOOKBACK)
|
||||
|
||||
# Candle pattern features
|
||||
ct = candle_types[i - 6 : i]
|
||||
br = body_ratios[i - 6 : i]
|
||||
sr = shadow_ratios[i - 6 : i]
|
||||
|
||||
recent_returns = ret_window[-12:]
|
||||
momentum_short = np.sum(recent_returns[-3:])
|
||||
momentum_mid = np.sum(recent_returns[-6:])
|
||||
momentum_long = np.sum(recent_returns)
|
||||
|
||||
vol_short = np.std(recent_returns[-6:]) if len(recent_returns) >= 6 else 0
|
||||
vol_long = np.std(ret_window) if len(ret_window) > 0 else 0
|
||||
|
||||
volume_window = df["volume"].values[i - 12 : i]
|
||||
vol_avg = np.mean(volume_window) if len(volume_window) > 0 else 0
|
||||
vol_last = df["volume"].values[i - 1] if i > 0 else 0
|
||||
vol_ratio = vol_last / vol_avg if vol_avg > 0 else 1.0
|
||||
|
||||
up_count_6 = np.sum(ct[-6:] == 1) / 6
|
||||
down_count_6 = np.sum(ct[-6:] == -1) / 6
|
||||
|
||||
features = [
|
||||
h, # Hurst exponent
|
||||
fd, # Fractal dimension
|
||||
ss, # Self-similarity
|
||||
vr, # Volatility ratio
|
||||
momentum_short, # 3-candle momentum
|
||||
momentum_mid, # 6-candle momentum
|
||||
momentum_long, # Full window momentum
|
||||
vol_short, # Short-term volatility
|
||||
vol_long, # Long-term volatility
|
||||
vol_ratio, # Volume spike ratio
|
||||
up_count_6, # Bullish ratio (last 6)
|
||||
down_count_6, # Bearish ratio (last 6)
|
||||
np.mean(br[-6:]), # Avg body ratio
|
||||
np.mean(sr[-6:]), # Avg shadow ratio
|
||||
np.mean(br[-3:]), # Avg body ratio (last 3)
|
||||
np.std(br[-6:]), # Body ratio std
|
||||
close[i - 1] / np.mean(window), # Price vs MA
|
||||
]
|
||||
|
||||
# Label: 1 if price goes up in next LOOKAHEAD candles, 0 otherwise
|
||||
future_ret = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
label = 1 if future_ret > 0.002 else (0 if future_ret > -0.002 else -1)
|
||||
|
||||
features_list.append(features)
|
||||
labels.append(label)
|
||||
indices.append(i)
|
||||
|
||||
X = np.array(features_list)
|
||||
y = np.array(labels)
|
||||
idx_arr = np.array(indices)
|
||||
|
||||
print(f"\nDataset: {len(X)} samples")
|
||||
print(f"Label distribution: up={np.sum(y==1)}, flat={np.sum(y==0)}, down={np.sum(y==-1)}")
|
||||
|
||||
# Train/test split cronologico
|
||||
split_point = int(len(X) * 0.7)
|
||||
X_train, X_test = X[:split_point], X[split_point:]
|
||||
y_train, y_test = y[:split_point], y[split_point:]
|
||||
idx_train, idx_test = idx_arr[:split_point], idx_arr[split_point:]
|
||||
|
||||
# Handle NaN/Inf
|
||||
X_train = np.nan_to_num(X_train, nan=0.0, posinf=1e6, neginf=-1e6)
|
||||
X_test = np.nan_to_num(X_test, nan=0.0, posinf=1e6, neginf=-1e6)
|
||||
|
||||
# --- Modelli ---
|
||||
print("\n--- TRAINING ---")
|
||||
|
||||
models = {
|
||||
"RandomForest": RandomForestClassifier(
|
||||
n_estimators=200, max_depth=8, min_samples_leaf=20,
|
||||
class_weight="balanced", random_state=42, n_jobs=-1,
|
||||
),
|
||||
"GradientBoosting": GradientBoostingClassifier(
|
||||
n_estimators=200, max_depth=5, min_samples_leaf=20,
|
||||
learning_rate=0.05, random_state=42,
|
||||
),
|
||||
}
|
||||
|
||||
for name, model in models.items():
|
||||
print(f"\n{'='*40}")
|
||||
print(f" {name}")
|
||||
print(f"{'='*40}")
|
||||
|
||||
model.fit(X_train, y_train)
|
||||
|
||||
# Feature importance
|
||||
if hasattr(model, "feature_importances_"):
|
||||
feat_names = [
|
||||
"hurst", "fractal_dim", "self_sim", "vol_ratio",
|
||||
"mom_3", "mom_6", "mom_full", "vol_short", "vol_long",
|
||||
"vol_spike", "up_ratio", "down_ratio", "body_avg",
|
||||
"shadow_avg", "body_3", "body_std", "price_vs_ma"
|
||||
]
|
||||
imp = model.feature_importances_
|
||||
sorted_idx = np.argsort(imp)[::-1]
|
||||
print("\nFeature importance (top 10):")
|
||||
for j in sorted_idx[:10]:
|
||||
print(f" {feat_names[j]:15s}: {imp[j]:.4f}")
|
||||
|
||||
# Prediction con probabilità
|
||||
y_pred = model.predict(X_test)
|
||||
proba = model.predict_proba(X_test)
|
||||
|
||||
print(f"\nAccuracy: {accuracy_score(y_test, y_pred)*100:.1f}%")
|
||||
print(classification_report(y_test, y_pred, target_names=["down", "flat", "up"], zero_division=0))
|
||||
|
||||
# Genera segnali filtrati per confidenza
|
||||
signals = pd.Series(0, index=df.index)
|
||||
accuracies_filtered = []
|
||||
classes = model.classes_
|
||||
|
||||
up_class_idx = list(classes).index(1) if 1 in classes else -1
|
||||
down_class_idx = list(classes).index(-1) if -1 in classes else -1
|
||||
|
||||
for k, i in enumerate(idx_test):
|
||||
p = proba[k]
|
||||
if up_class_idx >= 0 and p[up_class_idx] >= MIN_CONFIDENCE:
|
||||
signals.iloc[i] = 1
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
accuracies_filtered.append(1 if actual > 0 else 0)
|
||||
elif down_class_idx >= 0 and p[down_class_idx] >= MIN_CONFIDENCE:
|
||||
signals.iloc[i] = -1
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
accuracies_filtered.append(1 if actual < 0 else 0)
|
||||
|
||||
n_signals = (signals != 0).sum()
|
||||
print(f"\nSegnali filtrati (conf>={MIN_CONFIDENCE}): {n_signals}")
|
||||
if accuracies_filtered:
|
||||
print(f"Accuratezza filtrata: {np.mean(accuracies_filtered)*100:.1f}%")
|
||||
|
||||
# Backtest
|
||||
split_idx = int(len(df) * 0.7)
|
||||
test_df = df.iloc[split_idx:].reset_index(drop=True)
|
||||
test_signals = signals.iloc[split_idx:].reset_index(drop=True)
|
||||
|
||||
result = run_backtest(test_df, test_signals, initial_capital=1000, fee_pct=0.001, max_hold_candles=LOOKAHEAD)
|
||||
print(f"\nBACKTEST:")
|
||||
for kk, v in result.summary().items():
|
||||
print(f" {kk}: {v}")
|
||||
|
||||
# Prova con soglie diverse
|
||||
print(f"\n Varianti soglia:")
|
||||
for threshold in [0.55, 0.60, 0.65, 0.70, 0.75, 0.80]:
|
||||
sigs = pd.Series(0, index=df.index)
|
||||
accs = []
|
||||
for k, i in enumerate(idx_test):
|
||||
p = proba[k]
|
||||
if up_class_idx >= 0 and p[up_class_idx] >= threshold:
|
||||
sigs.iloc[i] = 1
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
elif down_class_idx >= 0 and p[down_class_idx] >= threshold:
|
||||
sigs.iloc[i] = -1
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
|
||||
t_sigs = sigs.iloc[split_idx:].reset_index(drop=True)
|
||||
res = run_backtest(test_df, t_sigs, initial_capital=1000, fee_pct=0.001, max_hold_candles=LOOKAHEAD)
|
||||
acc = np.mean(accs) * 100 if accs else 0
|
||||
print(f" thr={threshold:.2f}: signals={len(accs):4d} acc={acc:.1f}% ret={res.total_return*100:+.1f}% wr={res.win_rate*100:.0f}% sharpe={res.sharpe_ratio:.2f}")
|
||||
@@ -0,0 +1,202 @@
|
||||
"""Strategia 5: Enhanced fractal features + binary classification + position management.
|
||||
Miglioramenti rispetto a #4:
|
||||
- Binary classification (up vs down, ignora flat)
|
||||
- Feature engineering esteso: multi-window fractal indicators
|
||||
- Migliore filtraggio segnali
|
||||
- Position sizing basato su confidenza
|
||||
- Trailing stop
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import GradientBoostingClassifier
|
||||
from sklearn.metrics import accuracy_score
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.indicators import (
|
||||
hurst_exponent,
|
||||
fractal_dimension_higuchi,
|
||||
self_similarity_score,
|
||||
volatility_ratio,
|
||||
)
|
||||
from src.fractal.patterns import encode_candles, extract_body_ratios, extract_shadow_ratios
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 5: ENHANCED FRACTAL — BTC + ETH 1H")
|
||||
print("=" * 60)
|
||||
|
||||
LOOKAHEADS = [3, 6, 12]
|
||||
MIN_RETURN = 0.003 # 0.3% threshold for "up" label
|
||||
|
||||
for asset in ["BTC", "ETH"]:
|
||||
for LOOKAHEAD in LOOKAHEADS:
|
||||
print(f"\n{'#'*60}")
|
||||
print(f" {asset} 1H — LOOKAHEAD={LOOKAHEAD}")
|
||||
print(f"{'#'*60}")
|
||||
|
||||
df = load_data(asset, "1h")
|
||||
close = df["close"].values
|
||||
volume = df["volume"].values
|
||||
n = len(close)
|
||||
log_close = np.log(np.where(close == 0, 1e-10, close))
|
||||
returns = np.diff(log_close)
|
||||
|
||||
candle_types = encode_candles(df)
|
||||
body_ratios = extract_body_ratios(df)
|
||||
shadow_ratios = extract_shadow_ratios(df)
|
||||
|
||||
WINDOWS = [24, 48, 96, 192]
|
||||
features_list = []
|
||||
labels = []
|
||||
indices = []
|
||||
|
||||
max_window = max(WINDOWS) + 50
|
||||
|
||||
for i in range(max_window, n - LOOKAHEAD, 2):
|
||||
feats = []
|
||||
|
||||
for w in WINDOWS:
|
||||
ret_w = returns[i - w : i - 1]
|
||||
close_w = close[i - w : i]
|
||||
|
||||
h = hurst_exponent(ret_w, max_lag=min(len(ret_w) // 4, 20))
|
||||
fd = fractal_dimension_higuchi(ret_w, k_max=min(6, len(ret_w) // 4))
|
||||
vr = volatility_ratio(close_w, fast=min(12, w // 4), slow=w)
|
||||
|
||||
mom = np.sum(ret_w)
|
||||
vol = np.std(ret_w)
|
||||
skew = float(pd.Series(ret_w).skew()) if len(ret_w) > 2 else 0
|
||||
kurt = float(pd.Series(ret_w).kurtosis()) if len(ret_w) > 3 else 0
|
||||
|
||||
ma = np.mean(close_w)
|
||||
price_vs_ma = close[i - 1] / ma if ma > 0 else 1
|
||||
|
||||
# Autocorrelation lag-1
|
||||
if len(ret_w) > 1 and np.std(ret_w) > 0:
|
||||
ac1 = np.corrcoef(ret_w[:-1], ret_w[1:])[0, 1]
|
||||
if not np.isfinite(ac1):
|
||||
ac1 = 0
|
||||
else:
|
||||
ac1 = 0
|
||||
|
||||
feats.extend([h, fd, vr, mom, vol, skew, kurt, price_vs_ma, ac1])
|
||||
|
||||
# Self-similarity multi-scale
|
||||
large_window = close[max(0, i - 192 * 4) : i]
|
||||
ss = self_similarity_score(large_window, 48)
|
||||
feats.append(ss)
|
||||
|
||||
# Candle pattern features (last 12 candles)
|
||||
ct = candle_types[i - 12 : i]
|
||||
br = body_ratios[i - 12 : i]
|
||||
sr = shadow_ratios[i - 12 : i]
|
||||
|
||||
feats.extend([
|
||||
np.mean(ct[-3:]),
|
||||
np.mean(ct[-6:]),
|
||||
np.mean(ct[-12:]),
|
||||
np.std(br[-6:]),
|
||||
np.mean(br[-3:]),
|
||||
np.mean(sr[-6:]),
|
||||
np.max(br[-6:]),
|
||||
np.min(br[-6:]),
|
||||
])
|
||||
|
||||
# Volume features
|
||||
vol_w = volume[i - 24 : i]
|
||||
if np.mean(vol_w) > 0:
|
||||
feats.append(volume[i - 1] / np.mean(vol_w))
|
||||
feats.append(np.std(vol_w) / np.mean(vol_w))
|
||||
else:
|
||||
feats.extend([1.0, 0.0])
|
||||
|
||||
# Range/ATR proxy
|
||||
h_arr = df["high"].values[i - 14 : i]
|
||||
l_arr = df["low"].values[i - 14 : i]
|
||||
c_arr = close[i - 14 : i]
|
||||
tr = np.maximum(h_arr - l_arr, np.maximum(np.abs(h_arr - np.roll(c_arr, 1)), np.abs(l_arr - np.roll(c_arr, 1))))
|
||||
atr = np.mean(tr[1:])
|
||||
feats.append(atr / close[i - 1] if close[i - 1] > 0 else 0)
|
||||
|
||||
# Label
|
||||
future_ret = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
if abs(future_ret) < MIN_RETURN:
|
||||
continue # skip flat zones
|
||||
|
||||
label = 1 if future_ret > 0 else 0
|
||||
|
||||
features_list.append(feats)
|
||||
labels.append(label)
|
||||
indices.append(i)
|
||||
|
||||
X = np.array(features_list)
|
||||
y = np.array(labels)
|
||||
idx_arr = np.array(indices)
|
||||
|
||||
X = np.nan_to_num(X, nan=0.0, posinf=1e6, neginf=-1e6)
|
||||
|
||||
# Split
|
||||
split = int(len(X) * 0.7)
|
||||
X_train, X_test = X[:split], X[split:]
|
||||
y_train, y_test = y[:split], y[split:]
|
||||
idx_test = idx_arr[split:]
|
||||
|
||||
print(f"Samples: {len(X)} (train={split}, test={len(X)-split})")
|
||||
print(f"Label balance: up={np.mean(y)*100:.1f}%")
|
||||
|
||||
# Train
|
||||
model = GradientBoostingClassifier(
|
||||
n_estimators=300, max_depth=5, min_samples_leaf=30,
|
||||
learning_rate=0.03, subsample=0.8, random_state=42,
|
||||
)
|
||||
model.fit(X_train, y_train)
|
||||
|
||||
y_pred = model.predict(X_test)
|
||||
proba = model.predict_proba(X_test)
|
||||
|
||||
base_acc = accuracy_score(y_test, y_pred)
|
||||
print(f"Base accuracy: {base_acc*100:.1f}%")
|
||||
|
||||
# Threshold sweep
|
||||
print(f"\n Threshold sweep:")
|
||||
for thr in [0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80]:
|
||||
up_idx = model.classes_.tolist().index(1)
|
||||
|
||||
sigs = []
|
||||
accs = []
|
||||
for k in range(len(X_test)):
|
||||
p_up = proba[k][up_idx]
|
||||
i = idx_test[k]
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
|
||||
if p_up >= thr:
|
||||
sigs.append(("long", i))
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
elif p_up <= (1 - thr):
|
||||
sigs.append(("short", i))
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
|
||||
if not accs:
|
||||
print(f" thr={thr:.2f}: no signals")
|
||||
continue
|
||||
|
||||
acc = np.mean(accs) * 100
|
||||
# Simple PnL estimate
|
||||
pnl = 0
|
||||
capital = 1000
|
||||
for direction, i in sigs:
|
||||
entry = close[i - 1]
|
||||
exit_ = close[i + LOOKAHEAD - 1]
|
||||
if direction == "long":
|
||||
ret = (exit_ - entry) / entry
|
||||
else:
|
||||
ret = (entry - exit_) / entry
|
||||
ret -= 0.002 # fees round-trip
|
||||
pnl += capital * ret * 0.5 # 50% per trade
|
||||
capital += capital * ret * 0.5
|
||||
|
||||
total_ret = (capital - 1000) / 1000 * 100
|
||||
trades_per_year = len(sigs) / ((n - max_window) / (24 * 365))
|
||||
print(f" thr={thr:.2f}: signals={len(sigs):5d} acc={acc:.1f}% ret={total_ret:+.1f}% trades/yr={trades_per_year:.0f}")
|
||||
@@ -0,0 +1,201 @@
|
||||
"""Strategia 6: Structural Pattern Matching con DTW veloce.
|
||||
Idea diversa: invece di ML generico, cerca nel passato le finestre OHLC
|
||||
più simili alla finestra corrente usando una versione veloce (reduced DTW).
|
||||
Vota sulla direzione basandosi sui K-nearest neighbors nel passato.
|
||||
Usa features normalizzate (non DTW puro sul prezzo che è lento).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.neighbors import KNeighborsClassifier
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.patterns import normalize_pattern_window
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 6: STRUCTURAL PATTERN KNN — BTC 1H")
|
||||
print("=" * 60)
|
||||
|
||||
df = load_data("BTC", "1h")
|
||||
close = df["close"].values
|
||||
n = len(close)
|
||||
|
||||
WINDOW = 24
|
||||
LOOKAHEAD = 6
|
||||
MIN_RETURN = 0.003
|
||||
|
||||
def extract_structural_features(df: pd.DataFrame, idx: int, window: int) -> np.ndarray | None:
|
||||
"""Extract normalized structural features from OHLC window."""
|
||||
if idx < window:
|
||||
return None
|
||||
|
||||
o = df["open"].values[idx - window : idx]
|
||||
h = df["high"].values[idx - window : idx]
|
||||
l = df["low"].values[idx - window : idx]
|
||||
c = df["close"].values[idx - window : idx]
|
||||
v = df["volume"].values[idx - window : idx]
|
||||
|
||||
# Normalize price to [0,1]
|
||||
all_prices = np.concatenate([o, h, l, c])
|
||||
mn, mx = all_prices.min(), all_prices.max()
|
||||
if mx - mn == 0:
|
||||
return None
|
||||
o_n = (o - mn) / (mx - mn)
|
||||
h_n = (h - mn) / (mx - mn)
|
||||
l_n = (l - mn) / (mx - mn)
|
||||
c_n = (c - mn) / (mx - mn)
|
||||
|
||||
# Body and shadow ratios (already normalized)
|
||||
total = h - l
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
body = np.abs(c - o) / total
|
||||
upper_shadow = (h - np.maximum(o, c)) / total
|
||||
lower_shadow = (np.minimum(o, c) - l) / total
|
||||
direction = np.sign(c - o)
|
||||
|
||||
# Returns
|
||||
log_c = np.log(np.where(c == 0, 1e-10, c))
|
||||
returns = np.diff(log_c)
|
||||
|
||||
# Volume profile (normalized)
|
||||
v_mean = np.mean(v)
|
||||
v_n = v / v_mean if v_mean > 0 else np.ones_like(v)
|
||||
|
||||
# Downsample to fixed-size feature vector
|
||||
# Take every N-th candle if window is large
|
||||
step = max(1, window // 12)
|
||||
sampled_idx = np.arange(0, window, step)[:12]
|
||||
|
||||
features = np.concatenate([
|
||||
c_n[sampled_idx], # 12: normalized close
|
||||
body[sampled_idx], # 12: body ratios
|
||||
direction[sampled_idx], # 12: direction
|
||||
upper_shadow[sampled_idx], # 12: upper shadow
|
||||
lower_shadow[sampled_idx], # 12: lower shadow
|
||||
v_n[sampled_idx], # 12: volume profile
|
||||
[np.mean(returns), np.std(returns), np.sum(returns)], # 3: return stats
|
||||
[np.mean(body), np.std(body)], # 2: body stats
|
||||
])
|
||||
return features
|
||||
|
||||
|
||||
print("Extracting features...")
|
||||
features_all = []
|
||||
labels_all = []
|
||||
indices_all = []
|
||||
|
||||
for i in range(WINDOW, n - LOOKAHEAD, 1):
|
||||
feats = extract_structural_features(df, i, WINDOW)
|
||||
if feats is None:
|
||||
continue
|
||||
|
||||
future_ret = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
if abs(future_ret) < MIN_RETURN:
|
||||
continue
|
||||
|
||||
features_all.append(feats)
|
||||
labels_all.append(1 if future_ret > 0 else 0)
|
||||
indices_all.append(i)
|
||||
|
||||
X = np.array(features_all)
|
||||
y = np.array(labels_all)
|
||||
idx_arr = np.array(indices_all)
|
||||
|
||||
X = np.nan_to_num(X, nan=0.0, posinf=1e6, neginf=-1e6)
|
||||
|
||||
split = int(len(X) * 0.7)
|
||||
X_train, X_test = X[:split], X[split:]
|
||||
y_train, y_test = y[:split], y[split:]
|
||||
idx_test = idx_arr[split:]
|
||||
|
||||
print(f"Samples: {len(X)} (train={split}, test={len(X)-split})")
|
||||
print(f"Label balance: up={np.mean(y)*100:.1f}%")
|
||||
|
||||
scaler = StandardScaler()
|
||||
X_train_s = scaler.fit_transform(X_train)
|
||||
X_test_s = scaler.transform(X_test)
|
||||
|
||||
# Test diversi K
|
||||
print("\n--- KNN SWEEP ---")
|
||||
for K in [5, 10, 20, 50, 100, 200]:
|
||||
knn = KNeighborsClassifier(n_neighbors=K, weights="distance", n_jobs=-1)
|
||||
knn.fit(X_train_s, y_train)
|
||||
|
||||
proba = knn.predict_proba(X_test_s)
|
||||
up_idx = list(knn.classes_).index(1)
|
||||
|
||||
for thr in [0.55, 0.60, 0.65, 0.70]:
|
||||
sigs = []
|
||||
accs = []
|
||||
for j in range(len(X_test)):
|
||||
p_up = proba[j][up_idx]
|
||||
i = idx_test[j]
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
|
||||
if p_up >= thr:
|
||||
sigs.append(1)
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
elif p_up <= (1 - thr):
|
||||
sigs.append(-1)
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
|
||||
if not accs:
|
||||
continue
|
||||
|
||||
acc = np.mean(accs) * 100
|
||||
# PnL
|
||||
capital = 1000
|
||||
for direction, j in zip(sigs, range(len(accs))):
|
||||
i_idx = idx_test[[k for k in range(len(X_test)) if proba[k][up_idx] >= thr or proba[k][up_idx] <= (1 - thr)][j]]
|
||||
entry = close[i_idx - 1]
|
||||
exit_ = close[i_idx + LOOKAHEAD - 1]
|
||||
if direction == 1:
|
||||
ret = (exit_ - entry) / entry
|
||||
else:
|
||||
ret = (entry - exit_) / entry
|
||||
ret -= 0.002
|
||||
capital *= (1 + ret * 0.5)
|
||||
|
||||
total_ret = (capital - 1000) / 1000 * 100
|
||||
print(f" K={K:3d} thr={thr:.2f}: signals={len(accs):5d} acc={acc:.1f}% ret={total_ret:+.1f}%")
|
||||
|
||||
# Best combo: try with Gradient Boosting on same features
|
||||
print("\n\n--- GRADIENT BOOSTING SU STRUCTURAL FEATURES ---")
|
||||
from sklearn.ensemble import GradientBoostingClassifier
|
||||
|
||||
gb = GradientBoostingClassifier(
|
||||
n_estimators=300, max_depth=5, min_samples_leaf=30,
|
||||
learning_rate=0.03, subsample=0.8, random_state=42,
|
||||
)
|
||||
gb.fit(X_train_s, y_train)
|
||||
proba_gb = gb.predict_proba(X_test_s)
|
||||
up_idx_gb = list(gb.classes_).index(1)
|
||||
|
||||
for thr in [0.50, 0.55, 0.60, 0.65, 0.70, 0.75]:
|
||||
accs = []
|
||||
capital = 1000
|
||||
n_trades = 0
|
||||
for j in range(len(X_test)):
|
||||
p_up = proba_gb[j][up_idx_gb]
|
||||
i = idx_test[j]
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
|
||||
if p_up >= thr:
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
ret = actual - 0.002
|
||||
capital *= (1 + ret * 0.5)
|
||||
n_trades += 1
|
||||
elif p_up <= (1 - thr):
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
ret = -actual - 0.002
|
||||
capital *= (1 + ret * 0.5)
|
||||
n_trades += 1
|
||||
|
||||
if not accs:
|
||||
continue
|
||||
acc = np.mean(accs) * 100
|
||||
total_ret = (capital - 1000) / 1000 * 100
|
||||
print(f" thr={thr:.2f}: trades={n_trades:5d} acc={acc:.1f}% ret={total_ret:+.1f}%")
|
||||
@@ -0,0 +1,320 @@
|
||||
"""Strategia 7: LSTM su features frattali multi-timeframe.
|
||||
Usa sequenze di features frattali come input a un LSTM
|
||||
per predire la direzione del prezzo.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
from torch.utils.data import DataLoader, TensorDataset
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.indicators import hurst_exponent, fractal_dimension_higuchi, volatility_ratio
|
||||
from src.fractal.patterns import encode_candles, extract_body_ratios, extract_shadow_ratios
|
||||
|
||||
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
||||
print(f"Device: {DEVICE}")
|
||||
|
||||
|
||||
class FractalLSTM(nn.Module):
|
||||
def __init__(self, input_size: int, hidden_size: int = 64, num_layers: int = 2, dropout: float = 0.3):
|
||||
super().__init__()
|
||||
self.lstm = nn.LSTM(input_size, hidden_size, num_layers, batch_first=True, dropout=dropout)
|
||||
self.classifier = nn.Sequential(
|
||||
nn.Linear(hidden_size, 32),
|
||||
nn.ReLU(),
|
||||
nn.Dropout(dropout),
|
||||
nn.Linear(32, 1),
|
||||
)
|
||||
|
||||
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
||||
_, (h_n, _) = self.lstm(x)
|
||||
out = self.classifier(h_n[-1])
|
||||
return out.squeeze(-1)
|
||||
|
||||
|
||||
def extract_candle_features(df: pd.DataFrame, i: int) -> np.ndarray:
|
||||
"""Extract per-candle features at index i."""
|
||||
o, h, l, c = df["open"].values[i], df["high"].values[i], df["low"].values[i], df["close"].values[i]
|
||||
v = df["volume"].values[i]
|
||||
total = h - l if h - l > 0 else 1e-10
|
||||
body = abs(c - o) / total
|
||||
upper_s = (h - max(o, c)) / total
|
||||
lower_s = (min(o, c) - l) / total
|
||||
direction = 1 if c > o else (-1 if c < o else 0)
|
||||
|
||||
# Log return from previous candle
|
||||
if i > 0:
|
||||
prev_c = df["close"].values[i - 1]
|
||||
log_ret = np.log(c / prev_c) if prev_c > 0 else 0
|
||||
else:
|
||||
log_ret = 0
|
||||
|
||||
return np.array([body, upper_s, lower_s, direction, log_ret, v])
|
||||
|
||||
|
||||
def build_dataset(df: pd.DataFrame, seq_len: int = 48, lookahead: int = 6, min_ret: float = 0.003):
|
||||
"""Build sequences of candle features with labels."""
|
||||
close = df["close"].values
|
||||
n = len(df)
|
||||
|
||||
vol_mean = pd.Series(df["volume"].values).rolling(100, min_periods=1).mean().values
|
||||
|
||||
sequences = []
|
||||
labels = []
|
||||
indices = []
|
||||
|
||||
# Pre-compute additional features
|
||||
candle_types = encode_candles(df)
|
||||
body_ratios = extract_body_ratios(df)
|
||||
shadow_ratios = extract_shadow_ratios(df)
|
||||
|
||||
for i in range(seq_len, n - lookahead, 2):
|
||||
seq = []
|
||||
for j in range(i - seq_len, i):
|
||||
feats = extract_candle_features(df, j)
|
||||
# Normalize volume by rolling mean
|
||||
feats[5] = feats[5] / vol_mean[j] if vol_mean[j] > 0 else 1.0
|
||||
seq.append(feats)
|
||||
|
||||
future_ret = (close[i + lookahead - 1] - close[i - 1]) / close[i - 1]
|
||||
if abs(future_ret) < min_ret:
|
||||
continue
|
||||
|
||||
sequences.append(seq)
|
||||
labels.append(1 if future_ret > 0 else 0)
|
||||
indices.append(i)
|
||||
|
||||
return np.array(sequences), np.array(labels), np.array(indices)
|
||||
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 7: LSTM FRACTAL — BTC 1H")
|
||||
print("=" * 60)
|
||||
|
||||
df = load_data("BTC", "1h")
|
||||
close = df["close"].values
|
||||
|
||||
SEQ_LEN = 48
|
||||
LOOKAHEAD = 6
|
||||
EPOCHS = 30
|
||||
BATCH_SIZE = 256
|
||||
LR = 0.001
|
||||
|
||||
print(f"\nSeq length: {SEQ_LEN}, Lookahead: {LOOKAHEAD}")
|
||||
print("Building dataset...")
|
||||
|
||||
X, y, idx_arr = build_dataset(df, seq_len=SEQ_LEN, lookahead=LOOKAHEAD)
|
||||
print(f"Samples: {len(X)}, Features per candle: {X.shape[2]}, Up ratio: {np.mean(y)*100:.1f}%")
|
||||
|
||||
# Chronological split
|
||||
split = int(len(X) * 0.7)
|
||||
val_split = int(len(X) * 0.85)
|
||||
|
||||
X_train, X_val, X_test = X[:split], X[split:val_split], X[val_split:]
|
||||
y_train, y_val, y_test = y[:split], y[split:val_split], y[val_split:]
|
||||
idx_test_arr = idx_arr[val_split:]
|
||||
|
||||
# Normalize features per-feature across time
|
||||
n_features = X.shape[2]
|
||||
for f in range(n_features):
|
||||
scaler = StandardScaler()
|
||||
X_train[:, :, f] = scaler.fit_transform(X_train[:, :, f])
|
||||
X_val[:, :, f] = scaler.transform(X_val[:, :, f])
|
||||
X_test[:, :, f] = scaler.transform(X_test[:, :, f])
|
||||
|
||||
# To tensors
|
||||
X_train_t = torch.FloatTensor(X_train).to(DEVICE)
|
||||
y_train_t = torch.FloatTensor(y_train).to(DEVICE)
|
||||
X_val_t = torch.FloatTensor(X_val).to(DEVICE)
|
||||
y_val_t = torch.FloatTensor(y_val).to(DEVICE)
|
||||
X_test_t = torch.FloatTensor(X_test).to(DEVICE)
|
||||
|
||||
train_ds = TensorDataset(X_train_t, y_train_t)
|
||||
train_dl = DataLoader(train_ds, batch_size=BATCH_SIZE, shuffle=True)
|
||||
|
||||
# Model
|
||||
model = FractalLSTM(input_size=n_features, hidden_size=64, num_layers=2, dropout=0.3).to(DEVICE)
|
||||
optimizer = torch.optim.Adam(model.parameters(), lr=LR, weight_decay=1e-5)
|
||||
criterion = nn.BCEWithLogitsLoss()
|
||||
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, patience=5, factor=0.5)
|
||||
|
||||
print(f"\nTraining on {DEVICE}...")
|
||||
best_val_acc = 0
|
||||
patience_counter = 0
|
||||
|
||||
for epoch in range(EPOCHS):
|
||||
model.train()
|
||||
total_loss = 0
|
||||
for xb, yb in train_dl:
|
||||
optimizer.zero_grad()
|
||||
pred = model(xb)
|
||||
loss = criterion(pred, yb)
|
||||
loss.backward()
|
||||
torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
|
||||
optimizer.step()
|
||||
total_loss += loss.item()
|
||||
|
||||
# Validation
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
val_pred = model(X_val_t)
|
||||
val_loss = criterion(val_pred, y_val_t).item()
|
||||
val_proba = torch.sigmoid(val_pred).cpu().numpy()
|
||||
val_acc = np.mean((val_proba > 0.5) == y_val)
|
||||
|
||||
scheduler.step(val_loss)
|
||||
|
||||
if val_acc > best_val_acc:
|
||||
best_val_acc = val_acc
|
||||
torch.save(model.state_dict(), "data/processed/best_lstm.pt")
|
||||
patience_counter = 0
|
||||
else:
|
||||
patience_counter += 1
|
||||
|
||||
if epoch % 5 == 0 or patience_counter > 8:
|
||||
print(f" Epoch {epoch:2d}: train_loss={total_loss/len(train_dl):.4f} val_loss={val_loss:.4f} val_acc={val_acc*100:.1f}% best={best_val_acc*100:.1f}%")
|
||||
|
||||
if patience_counter > 10:
|
||||
print(f" Early stopping at epoch {epoch}")
|
||||
break
|
||||
|
||||
# Load best model and test
|
||||
model.load_state_dict(torch.load("data/processed/best_lstm.pt", weights_only=True))
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
test_pred = model(X_test_t)
|
||||
test_proba = torch.sigmoid(test_pred).cpu().numpy()
|
||||
|
||||
test_acc = np.mean((test_proba > 0.5) == y_test)
|
||||
print(f"\nTest accuracy (base): {test_acc*100:.1f}%")
|
||||
|
||||
# Threshold sweep
|
||||
print("\n--- THRESHOLD SWEEP ---")
|
||||
for thr in [0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80]:
|
||||
accs = []
|
||||
capital = 1000
|
||||
n_trades = 0
|
||||
|
||||
for j in range(len(X_test)):
|
||||
p = test_proba[j]
|
||||
i = idx_test_arr[j]
|
||||
actual = (close[i + LOOKAHEAD - 1] - close[i - 1]) / close[i - 1]
|
||||
|
||||
if p >= thr:
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
ret = actual - 0.002
|
||||
capital *= (1 + ret * 0.3)
|
||||
n_trades += 1
|
||||
elif p <= (1 - thr):
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
ret = -actual - 0.002
|
||||
capital *= (1 + ret * 0.3)
|
||||
n_trades += 1
|
||||
|
||||
if not accs:
|
||||
print(f" thr={thr:.2f}: no signals")
|
||||
continue
|
||||
|
||||
acc = np.mean(accs) * 100
|
||||
total_ret = (capital - 1000) / 1000 * 100
|
||||
# Annualized
|
||||
test_days = (idx_test_arr[-1] - idx_test_arr[0]) / 24
|
||||
years = test_days / 365.25 if test_days > 0 else 1
|
||||
ann_ret = ((capital / 1000) ** (1 / years) - 1) * 100 if years > 0 and capital > 0 else -100
|
||||
trades_yr = n_trades / years if years > 0 else 0
|
||||
print(f" thr={thr:.2f}: trades={n_trades:5d} acc={acc:.1f}% ret={total_ret:+.1f}% ann={ann_ret:+.1f}% trades/yr={trades_yr:.0f}")
|
||||
|
||||
# Also try ETH
|
||||
print("\n\n" + "=" * 60)
|
||||
print(" LSTM SU ETH 1H (same model architecture)")
|
||||
print("=" * 60)
|
||||
|
||||
df_eth = load_data("ETH", "1h")
|
||||
close_eth = df_eth["close"].values
|
||||
|
||||
X_eth, y_eth, idx_eth = build_dataset(df_eth, seq_len=SEQ_LEN, lookahead=LOOKAHEAD)
|
||||
print(f"ETH samples: {len(X_eth)}, Up ratio: {np.mean(y_eth)*100:.1f}%")
|
||||
|
||||
split_e = int(len(X_eth) * 0.7)
|
||||
val_e = int(len(X_eth) * 0.85)
|
||||
X_train_e, X_val_e, X_test_e = X_eth[:split_e], X_eth[split_e:val_e], X_eth[val_e:]
|
||||
y_train_e, y_val_e, y_test_e = y_eth[:split_e], y_eth[split_e:val_e], y_eth[val_e:]
|
||||
idx_test_e = idx_eth[val_e:]
|
||||
|
||||
for f in range(n_features):
|
||||
sc = StandardScaler()
|
||||
X_train_e[:, :, f] = sc.fit_transform(X_train_e[:, :, f])
|
||||
X_val_e[:, :, f] = sc.transform(X_val_e[:, :, f])
|
||||
X_test_e[:, :, f] = sc.transform(X_test_e[:, :, f])
|
||||
|
||||
X_tr_e = torch.FloatTensor(X_train_e).to(DEVICE)
|
||||
y_tr_e = torch.FloatTensor(y_train_e).to(DEVICE)
|
||||
X_va_e = torch.FloatTensor(X_val_e).to(DEVICE)
|
||||
y_va_e = torch.FloatTensor(y_val_e).to(DEVICE)
|
||||
X_te_e = torch.FloatTensor(X_test_e).to(DEVICE)
|
||||
|
||||
model_eth = FractalLSTM(input_size=n_features, hidden_size=64, num_layers=2, dropout=0.3).to(DEVICE)
|
||||
opt_e = torch.optim.Adam(model_eth.parameters(), lr=LR, weight_decay=1e-5)
|
||||
ds_e = TensorDataset(X_tr_e, y_tr_e)
|
||||
dl_e = DataLoader(ds_e, batch_size=BATCH_SIZE, shuffle=True)
|
||||
sch_e = torch.optim.lr_scheduler.ReduceLROnPlateau(opt_e, patience=5, factor=0.5)
|
||||
|
||||
best_e = 0
|
||||
pc = 0
|
||||
for epoch in range(EPOCHS):
|
||||
model_eth.train()
|
||||
tl = 0
|
||||
for xb, yb in dl_e:
|
||||
opt_e.zero_grad()
|
||||
p = model_eth(xb)
|
||||
loss = criterion(p, yb)
|
||||
loss.backward()
|
||||
torch.nn.utils.clip_grad_norm_(model_eth.parameters(), 1.0)
|
||||
opt_e.step()
|
||||
tl += loss.item()
|
||||
|
||||
model_eth.eval()
|
||||
with torch.no_grad():
|
||||
vp = model_eth(X_va_e)
|
||||
vl = criterion(vp, y_va_e).item()
|
||||
va = np.mean((torch.sigmoid(vp).cpu().numpy() > 0.5) == y_val_e)
|
||||
|
||||
sch_e.step(vl)
|
||||
if va > best_e:
|
||||
best_e = va
|
||||
torch.save(model_eth.state_dict(), "data/processed/best_lstm_eth.pt")
|
||||
pc = 0
|
||||
else:
|
||||
pc += 1
|
||||
if epoch % 5 == 0:
|
||||
print(f" Epoch {epoch:2d}: val_acc={va*100:.1f}% best={best_e*100:.1f}%")
|
||||
if pc > 10:
|
||||
break
|
||||
|
||||
model_eth.load_state_dict(torch.load("data/processed/best_lstm_eth.pt", weights_only=True))
|
||||
model_eth.eval()
|
||||
with torch.no_grad():
|
||||
tp_e = torch.sigmoid(model_eth(X_te_e)).cpu().numpy()
|
||||
|
||||
print(f"\nETH Test accuracy: {np.mean((tp_e > 0.5) == y_test_e)*100:.1f}%")
|
||||
|
||||
for thr in [0.55, 0.60, 0.65, 0.70]:
|
||||
accs = []
|
||||
capital = 1000
|
||||
for j in range(len(X_test_e)):
|
||||
p = tp_e[j]
|
||||
i = idx_test_e[j]
|
||||
actual = (close_eth[i + LOOKAHEAD - 1] - close_eth[i - 1]) / close_eth[i - 1]
|
||||
if p >= thr:
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
capital *= (1 + (actual - 0.002) * 0.3)
|
||||
elif p <= (1 - thr):
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
capital *= (1 + (-actual - 0.002) * 0.3)
|
||||
if accs:
|
||||
print(f" thr={thr:.2f}: trades={len(accs):5d} acc={np.mean(accs)*100:.1f}% ret={(capital-1000)/10:+.1f}%")
|
||||
@@ -0,0 +1,290 @@
|
||||
"""Strategia 8: Ensemble multi-timeframe.
|
||||
Combina i migliori approcci:
|
||||
1. GBM su structural features (miglior singolo finora: 58.6%/+57.5%)
|
||||
2. GBM su fractal indicators
|
||||
3. Multi-timeframe: 1h features + 15m aggregati
|
||||
Vota con consensus: trade solo quando almeno 2/3 modelli concordano.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import GradientBoostingClassifier
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.patterns import encode_candles, extract_body_ratios, extract_shadow_ratios
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 8: ENSEMBLE MULTI-TF — BTC")
|
||||
print("=" * 60)
|
||||
|
||||
# Load both timeframes
|
||||
df_1h = load_data("BTC", "1h")
|
||||
df_15m = load_data("BTC", "15m")
|
||||
|
||||
close_1h = df_1h["close"].values
|
||||
ts_1h = df_1h["timestamp"].values
|
||||
|
||||
WINDOW_1H = 24
|
||||
LOOKAHEAD = 6
|
||||
MIN_RETURN = 0.003
|
||||
|
||||
def structural_features_1h(df: pd.DataFrame, i: int, window: int = 24) -> np.ndarray | None:
|
||||
if i < window:
|
||||
return None
|
||||
|
||||
o = df["open"].values[i - window : i]
|
||||
h = df["high"].values[i - window : i]
|
||||
l = df["low"].values[i - window : i]
|
||||
c = df["close"].values[i - window : i]
|
||||
v = df["volume"].values[i - window : i]
|
||||
|
||||
all_p = np.concatenate([o, h, l, c])
|
||||
mn, mx = all_p.min(), all_p.max()
|
||||
if mx - mn == 0:
|
||||
return None
|
||||
o_n = (o - mn) / (mx - mn)
|
||||
h_n = (h - mn) / (mx - mn)
|
||||
l_n = (l - mn) / (mx - mn)
|
||||
c_n = (c - mn) / (mx - mn)
|
||||
|
||||
total = h - l
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
body = np.abs(c - o) / total
|
||||
u_shadow = (h - np.maximum(o, c)) / total
|
||||
l_shadow = (np.minimum(o, c) - l) / total
|
||||
direction = np.sign(c - o)
|
||||
|
||||
log_c = np.log(np.where(c == 0, 1e-10, c))
|
||||
rets = np.diff(log_c)
|
||||
|
||||
v_mean = np.mean(v)
|
||||
v_n = v / v_mean if v_mean > 0 else np.ones_like(v)
|
||||
|
||||
step = max(1, window // 12)
|
||||
idx = np.arange(0, window, step)[:12]
|
||||
|
||||
features = np.concatenate([
|
||||
c_n[idx], body[idx], direction[idx],
|
||||
u_shadow[idx], l_shadow[idx], v_n[idx],
|
||||
[np.mean(rets), np.std(rets), np.sum(rets),
|
||||
np.mean(body), np.std(body),
|
||||
np.max(body[-6:]) - np.min(body[-6:])],
|
||||
])
|
||||
return features
|
||||
|
||||
|
||||
def multi_tf_features(ts_current: int, df_15m: pd.DataFrame, n_bars: int = 48) -> np.ndarray | None:
|
||||
"""Extract aggregated features from 15m data aligned to current 1h candle."""
|
||||
ts_15m = df_15m["timestamp"].values
|
||||
mask = ts_15m <= ts_current
|
||||
end_idx = np.sum(mask)
|
||||
|
||||
if end_idx < n_bars:
|
||||
return None
|
||||
|
||||
start = end_idx - n_bars
|
||||
chunk = df_15m.iloc[start:end_idx]
|
||||
|
||||
c = chunk["close"].values
|
||||
h = chunk["high"].values
|
||||
l = chunk["low"].values
|
||||
v = chunk["volume"].values
|
||||
|
||||
if len(c) < n_bars:
|
||||
return None
|
||||
|
||||
log_c = np.log(np.where(c == 0, 1e-10, c))
|
||||
rets = np.diff(log_c)
|
||||
|
||||
# Micro-structure features
|
||||
mom_12 = np.sum(rets[-12:])
|
||||
mom_24 = np.sum(rets[-24:])
|
||||
vol_12 = np.std(rets[-12:])
|
||||
vol_48 = np.std(rets)
|
||||
|
||||
# Candle pattern stats
|
||||
ct = encode_candles(chunk)
|
||||
up_ratio_12 = np.mean(ct[-12:] == 1)
|
||||
up_ratio_24 = np.mean(ct[-24:] == 1)
|
||||
|
||||
# Intra-bar volatility (high-low range)
|
||||
ranges = (h - l) / np.where(c == 0, 1e-10, c)
|
||||
avg_range_12 = np.mean(ranges[-12:])
|
||||
avg_range_48 = np.mean(ranges)
|
||||
|
||||
# Volume profile
|
||||
v_mean = np.mean(v)
|
||||
v_recent = np.mean(v[-12:])
|
||||
vol_surge = v_recent / v_mean if v_mean > 0 else 1.0
|
||||
|
||||
# Autocorrelation
|
||||
if np.std(rets) > 0 and len(rets) > 1:
|
||||
ac1 = np.corrcoef(rets[:-1], rets[1:])[0, 1]
|
||||
ac1 = 0 if not np.isfinite(ac1) else ac1
|
||||
else:
|
||||
ac1 = 0
|
||||
|
||||
return np.array([
|
||||
mom_12, mom_24, vol_12, vol_48,
|
||||
up_ratio_12, up_ratio_24,
|
||||
avg_range_12, avg_range_48,
|
||||
vol_surge, ac1,
|
||||
vol_12 / vol_48 if vol_48 > 0 else 1.0,
|
||||
])
|
||||
|
||||
|
||||
print("Extracting features...")
|
||||
n_1h = len(df_1h)
|
||||
|
||||
X_struct = []
|
||||
X_multi = []
|
||||
y_all = []
|
||||
indices = []
|
||||
|
||||
for i in range(WINDOW_1H, n_1h - LOOKAHEAD, 1):
|
||||
if i % 5000 == 0:
|
||||
print(f" {i}/{n_1h}")
|
||||
|
||||
sf = structural_features_1h(df_1h, i, WINDOW_1H)
|
||||
if sf is None:
|
||||
continue
|
||||
|
||||
mf = multi_tf_features(ts_1h[i - 1], df_15m)
|
||||
if mf is None:
|
||||
continue
|
||||
|
||||
future_ret = (close_1h[i + LOOKAHEAD - 1] - close_1h[i - 1]) / close_1h[i - 1]
|
||||
if abs(future_ret) < MIN_RETURN:
|
||||
continue
|
||||
|
||||
X_struct.append(sf)
|
||||
X_multi.append(mf)
|
||||
y_all.append(1 if future_ret > 0 else 0)
|
||||
indices.append(i)
|
||||
|
||||
X_s = np.nan_to_num(np.array(X_struct), nan=0, posinf=1e6, neginf=-1e6)
|
||||
X_m = np.nan_to_num(np.array(X_multi), nan=0, posinf=1e6, neginf=-1e6)
|
||||
X_combined = np.hstack([X_s, X_m])
|
||||
y = np.array(y_all)
|
||||
idx_arr = np.array(indices)
|
||||
|
||||
print(f"\nSamples: {len(y)}, struct_feats: {X_s.shape[1]}, multi_feats: {X_m.shape[1]}, combined: {X_combined.shape[1]}")
|
||||
print(f"Up ratio: {np.mean(y)*100:.1f}%")
|
||||
|
||||
split = int(len(y) * 0.7)
|
||||
|
||||
# 3 models
|
||||
configs = {
|
||||
"M1_structural": X_s,
|
||||
"M2_multi_tf": X_m,
|
||||
"M3_combined": X_combined,
|
||||
}
|
||||
|
||||
probas = {}
|
||||
for name, X_data in configs.items():
|
||||
X_tr, X_te = X_data[:split], X_data[split:]
|
||||
y_tr, y_te = y[:split], y[split:]
|
||||
|
||||
sc = StandardScaler()
|
||||
X_tr_s = sc.fit_transform(X_tr)
|
||||
X_te_s = sc.transform(X_te)
|
||||
|
||||
model = GradientBoostingClassifier(
|
||||
n_estimators=300, max_depth=5, min_samples_leaf=30,
|
||||
learning_rate=0.03, subsample=0.8, random_state=42,
|
||||
)
|
||||
model.fit(X_tr_s, y_tr)
|
||||
proba = model.predict_proba(X_te_s)
|
||||
up_idx = list(model.classes_).index(1)
|
||||
|
||||
probas[name] = proba[:, up_idx]
|
||||
|
||||
# Individual results
|
||||
for thr in [0.55, 0.60, 0.65, 0.70]:
|
||||
accs = []
|
||||
capital = 1000
|
||||
for j in range(len(X_te)):
|
||||
p = proba[j][up_idx]
|
||||
i = idx_arr[split + j]
|
||||
actual = (close_1h[i + LOOKAHEAD - 1] - close_1h[i - 1]) / close_1h[i - 1]
|
||||
if p >= thr:
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
capital *= (1 + (actual - 0.002) * 0.5)
|
||||
elif p <= (1 - thr):
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
capital *= (1 + (-actual - 0.002) * 0.5)
|
||||
|
||||
if accs:
|
||||
acc = np.mean(accs) * 100
|
||||
ret = (capital - 1000) / 1000 * 100
|
||||
test_days = (idx_arr[-1] - idx_arr[split]) / 24
|
||||
years = test_days / 365.25
|
||||
ann = ((capital / 1000) ** (1 / years) - 1) * 100 if years > 0 and capital > 0 else -100
|
||||
print(f" {name:15s} thr={thr:.2f}: trades={len(accs):5d} acc={acc:.1f}% ret={ret:+.1f}% ann={ann:+.1f}%")
|
||||
|
||||
|
||||
# Ensemble voting
|
||||
print("\n\n--- ENSEMBLE VOTING ---")
|
||||
y_test = y[split:]
|
||||
idx_test = idx_arr[split:]
|
||||
|
||||
for min_agree in [2, 3]:
|
||||
for thr in [0.55, 0.60, 0.65, 0.70]:
|
||||
accs = []
|
||||
capital = 1000
|
||||
for j in range(len(y_test)):
|
||||
votes_up = sum(1 for p in probas.values() if p[j] >= thr)
|
||||
votes_down = sum(1 for p in probas.values() if p[j] <= (1 - thr))
|
||||
|
||||
i = idx_test[j]
|
||||
actual = (close_1h[i + LOOKAHEAD - 1] - close_1h[i - 1]) / close_1h[i - 1]
|
||||
|
||||
if votes_up >= min_agree:
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
capital *= (1 + (actual - 0.002) * 0.5)
|
||||
elif votes_down >= min_agree:
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
capital *= (1 + (-actual - 0.002) * 0.5)
|
||||
|
||||
if accs:
|
||||
acc = np.mean(accs) * 100
|
||||
ret = (capital - 1000) / 1000 * 100
|
||||
test_days = (idx_test[-1] - idx_test[0]) / 24
|
||||
years = test_days / 365.25
|
||||
ann = ((capital / 1000) ** (1 / years) - 1) * 100 if years > 0 and capital > 0 else -100
|
||||
trades_yr = len(accs) / years if years > 0 else 0
|
||||
print(f" agree>={min_agree} thr={thr:.2f}: trades={len(accs):5d} acc={acc:.1f}% ret={ret:+.1f}% ann={ann:+.1f}% trades/yr={trades_yr:.0f}")
|
||||
|
||||
|
||||
# Average probability ensemble
|
||||
print("\n--- ENSEMBLE AVERAGE PROBABILITY ---")
|
||||
avg_proba = np.mean([p for p in probas.values()], axis=0)
|
||||
|
||||
for thr in [0.55, 0.60, 0.65, 0.70, 0.75]:
|
||||
accs = []
|
||||
capital = 1000
|
||||
for j in range(len(y_test)):
|
||||
p = avg_proba[j]
|
||||
i = idx_test[j]
|
||||
actual = (close_1h[i + LOOKAHEAD - 1] - close_1h[i - 1]) / close_1h[i - 1]
|
||||
|
||||
if p >= thr:
|
||||
accs.append(1 if actual > 0 else 0)
|
||||
capital *= (1 + (actual - 0.002) * 0.5)
|
||||
elif p <= (1 - thr):
|
||||
accs.append(1 if actual < 0 else 0)
|
||||
capital *= (1 + (-actual - 0.002) * 0.5)
|
||||
|
||||
if accs:
|
||||
acc = np.mean(accs) * 100
|
||||
ret = (capital - 1000) / 1000 * 100
|
||||
test_days = (idx_test[-1] - idx_test[0]) / 24
|
||||
years = test_days / 365.25
|
||||
ann = ((capital / 1000) ** (1 / years) - 1) * 100 if years > 0 and capital > 0 else -100
|
||||
trades_yr = len(accs) / years if years > 0 else 0
|
||||
daily_ret = capital ** (1 / (test_days)) - 1 if test_days > 0 and capital > 0 else 0
|
||||
daily_pnl_on_1k = 1000 * daily_ret
|
||||
print(f" thr={thr:.2f}: trades={len(accs):5d} acc={acc:.1f}% ret={ret:+.1f}% ann={ann:+.1f}% trades/yr={trades_yr:.0f} daily_pnl=€{daily_pnl_on_1k:.2f}")
|
||||
@@ -0,0 +1,309 @@
|
||||
"""Strategia 9: Refined walk-forward with adaptive features.
|
||||
Combina le lezioni apprese:
|
||||
- Structural features (migliore singolo)
|
||||
- Walk-forward validation (no single split bias)
|
||||
- XGBoost (più potente di GBM per dati tabulari)
|
||||
- Dynamic exit: trailing stop + take profit
|
||||
- Multi-asset: BTC + ETH in portafoglio
|
||||
- Position sizing basato su confidenza
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import GradientBoostingClassifier
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.patterns import encode_candles, extract_body_ratios, extract_shadow_ratios
|
||||
from src.fractal.indicators import hurst_exponent, volatility_ratio
|
||||
|
||||
print("=" * 60)
|
||||
print(" STRATEGIA 9: WALK-FORWARD REFINATA")
|
||||
print("=" * 60)
|
||||
|
||||
|
||||
def build_features(df: pd.DataFrame, i: int) -> np.ndarray | None:
|
||||
"""All features from structural + fractal, no leakage."""
|
||||
if i < 200:
|
||||
return None
|
||||
|
||||
o = df["open"].values
|
||||
h = df["high"].values
|
||||
l = df["low"].values
|
||||
c = df["close"].values
|
||||
v = df["volume"].values
|
||||
|
||||
feats = []
|
||||
|
||||
# Structural features (3 windows)
|
||||
for w in [12, 24, 48]:
|
||||
win_c = c[i - w : i]
|
||||
win_o = o[i - w : i]
|
||||
win_h = h[i - w : i]
|
||||
win_l = l[i - w : i]
|
||||
win_v = v[i - w : i]
|
||||
|
||||
mn, mx = min(win_l.min(), win_o.min()), max(win_h.max(), win_o.max())
|
||||
if mx - mn == 0:
|
||||
feats.extend([0] * 15)
|
||||
continue
|
||||
|
||||
c_n = (win_c - mn) / (mx - mn)
|
||||
total = win_h - win_l
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
|
||||
body = np.abs(win_c - win_o) / total
|
||||
direction = np.sign(win_c - win_o)
|
||||
|
||||
log_c = np.log(np.where(win_c == 0, 1e-10, win_c))
|
||||
rets = np.diff(log_c)
|
||||
|
||||
v_mean = np.mean(win_v)
|
||||
v_n = win_v / v_mean if v_mean > 0 else np.ones_like(win_v)
|
||||
|
||||
feats.extend([
|
||||
np.mean(rets),
|
||||
np.std(rets),
|
||||
np.sum(rets),
|
||||
float(pd.Series(rets).skew()) if len(rets) > 2 else 0,
|
||||
float(pd.Series(rets).kurtosis()) if len(rets) > 3 else 0,
|
||||
np.mean(body),
|
||||
np.std(body),
|
||||
np.mean(direction[-6:]),
|
||||
np.mean(direction),
|
||||
c_n[-1],
|
||||
np.mean(c_n[-6:]),
|
||||
v_n[-1],
|
||||
np.mean(v_n[-6:]),
|
||||
np.max(body[-6:]),
|
||||
np.corrcoef(rets[:-1], rets[1:])[0, 1] if len(rets) > 1 and np.std(rets) > 0 else 0,
|
||||
])
|
||||
|
||||
# Fractal features
|
||||
ret_long = np.diff(np.log(np.where(c[i-96:i] == 0, 1e-10, c[i-96:i])))
|
||||
if len(ret_long) > 20:
|
||||
h_exp = hurst_exponent(ret_long, max_lag=min(len(ret_long)//4, 20))
|
||||
else:
|
||||
h_exp = 0.5
|
||||
|
||||
feats.append(h_exp)
|
||||
feats.append(volatility_ratio(c[i-48:i], fast=12, slow=48))
|
||||
|
||||
# ATR
|
||||
tr_arr = np.maximum(h[i-14:i] - l[i-14:i],
|
||||
np.maximum(np.abs(h[i-14:i] - np.roll(c[i-14:i], 1)),
|
||||
np.abs(l[i-14:i] - np.roll(c[i-14:i], 1))))
|
||||
atr = np.mean(tr_arr[1:])
|
||||
feats.append(atr / c[i-1] if c[i-1] > 0 else 0)
|
||||
|
||||
# Price position relative to recent range
|
||||
high_48 = np.max(h[i-48:i])
|
||||
low_48 = np.min(l[i-48:i])
|
||||
range_48 = high_48 - low_48
|
||||
feats.append((c[i-1] - low_48) / range_48 if range_48 > 0 else 0.5)
|
||||
|
||||
return np.nan_to_num(np.array(feats), nan=0, posinf=1e6, neginf=-1e6)
|
||||
|
||||
|
||||
def walk_forward_backtest(
|
||||
df: pd.DataFrame,
|
||||
train_size: int = 10000,
|
||||
step_size: int = 2000,
|
||||
lookahead: int = 6,
|
||||
min_return: float = 0.003,
|
||||
threshold: float = 0.60,
|
||||
fee_pct: float = 0.001,
|
||||
position_pct: float = 0.3,
|
||||
) -> dict:
|
||||
"""Walk-forward validation with rolling train window."""
|
||||
close = df["close"].values
|
||||
n = len(df)
|
||||
|
||||
all_trades = []
|
||||
capital = 1000.0
|
||||
equity = [capital]
|
||||
|
||||
start = 200
|
||||
features_cache: dict[int, np.ndarray] = {}
|
||||
|
||||
def get_features(idx: int) -> np.ndarray | None:
|
||||
if idx not in features_cache:
|
||||
features_cache[idx] = build_features(df, idx)
|
||||
return features_cache[idx]
|
||||
|
||||
# Pre-compute all features
|
||||
print(" Pre-computing features...")
|
||||
for i in range(start, n - lookahead, 2):
|
||||
get_features(i)
|
||||
|
||||
fold = 0
|
||||
train_start = start
|
||||
total_signals = 0
|
||||
total_correct = 0
|
||||
|
||||
while train_start + train_size + step_size + lookahead < n:
|
||||
train_end = train_start + train_size
|
||||
test_end = min(train_end + step_size, n - lookahead)
|
||||
|
||||
# Build train set
|
||||
X_train, y_train = [], []
|
||||
for i in range(train_start, train_end, 2):
|
||||
f = get_features(i)
|
||||
if f is None:
|
||||
continue
|
||||
ret = (close[i + lookahead - 1] - close[i - 1]) / close[i - 1]
|
||||
if abs(ret) < min_return:
|
||||
continue
|
||||
X_train.append(f)
|
||||
y_train.append(1 if ret > 0 else 0)
|
||||
|
||||
if len(X_train) < 100:
|
||||
train_start += step_size
|
||||
continue
|
||||
|
||||
X_tr = np.array(X_train)
|
||||
y_tr = np.array(y_train)
|
||||
|
||||
scaler = StandardScaler()
|
||||
X_tr_s = scaler.fit_transform(X_tr)
|
||||
|
||||
model = GradientBoostingClassifier(
|
||||
n_estimators=200, max_depth=5, min_samples_leaf=30,
|
||||
learning_rate=0.05, subsample=0.8, random_state=42,
|
||||
)
|
||||
model.fit(X_tr_s, y_tr)
|
||||
|
||||
up_idx = list(model.classes_).index(1)
|
||||
|
||||
# Test on next step
|
||||
fold_trades = 0
|
||||
fold_correct = 0
|
||||
for i in range(train_end, test_end, 2):
|
||||
f = get_features(i)
|
||||
if f is None:
|
||||
continue
|
||||
|
||||
f_s = scaler.transform(f.reshape(1, -1))
|
||||
proba = model.predict_proba(f_s)[0]
|
||||
p_up = proba[up_idx]
|
||||
|
||||
actual_ret = (close[i + lookahead - 1] - close[i - 1]) / close[i - 1]
|
||||
if abs(actual_ret) < min_return:
|
||||
continue
|
||||
|
||||
direction = None
|
||||
if p_up >= threshold:
|
||||
direction = "long"
|
||||
elif p_up <= (1 - threshold):
|
||||
direction = "short"
|
||||
|
||||
if direction:
|
||||
if direction == "long":
|
||||
trade_ret = actual_ret
|
||||
else:
|
||||
trade_ret = -actual_ret
|
||||
|
||||
net_ret = trade_ret - fee_pct * 2
|
||||
pnl = capital * position_pct * net_ret
|
||||
capital += pnl
|
||||
equity.append(capital)
|
||||
|
||||
is_correct = (direction == "long" and actual_ret > 0) or (direction == "short" and actual_ret < 0)
|
||||
fold_trades += 1
|
||||
if is_correct:
|
||||
fold_correct += 1
|
||||
|
||||
all_trades.append({
|
||||
"fold": fold,
|
||||
"idx": i,
|
||||
"direction": direction,
|
||||
"prob": p_up,
|
||||
"actual_ret": actual_ret,
|
||||
"net_ret": net_ret,
|
||||
"pnl": pnl,
|
||||
"correct": is_correct,
|
||||
})
|
||||
|
||||
total_signals += fold_trades
|
||||
total_correct += fold_correct
|
||||
fold_acc = fold_correct / fold_trades * 100 if fold_trades > 0 else 0
|
||||
if fold % 3 == 0:
|
||||
print(f" Fold {fold}: trades={fold_trades} acc={fold_acc:.0f}% capital=€{capital:.0f}")
|
||||
|
||||
fold += 1
|
||||
train_start += step_size
|
||||
|
||||
# Results
|
||||
if not all_trades:
|
||||
return {"error": "no trades"}
|
||||
|
||||
trades_df = pd.DataFrame(all_trades)
|
||||
total_acc = total_correct / total_signals * 100 if total_signals > 0 else 0
|
||||
|
||||
test_candles = n - 200 - train_size
|
||||
test_days = test_candles / 24
|
||||
test_years = test_days / 365.25
|
||||
|
||||
ann_ret = ((capital / 1000) ** (1 / test_years) - 1) * 100 if test_years > 0 and capital > 0 else -100
|
||||
|
||||
# Max drawdown
|
||||
peak = equity[0]
|
||||
max_dd = 0
|
||||
for v in equity:
|
||||
if v > peak:
|
||||
peak = v
|
||||
dd = (peak - v) / peak if peak > 0 else 0
|
||||
if dd > max_dd:
|
||||
max_dd = dd
|
||||
|
||||
# Sharpe
|
||||
equity_arr = np.array(equity)
|
||||
rets = np.diff(equity_arr) / equity_arr[:-1]
|
||||
rets = rets[np.isfinite(rets)]
|
||||
sharpe = np.mean(rets) / np.std(rets) * np.sqrt(252 * 24) if np.std(rets) > 0 else 0
|
||||
|
||||
return {
|
||||
"total_trades": total_signals,
|
||||
"accuracy": total_acc,
|
||||
"total_return": (capital - 1000) / 1000 * 100,
|
||||
"annualized_return": ann_ret,
|
||||
"max_drawdown": max_dd * 100,
|
||||
"sharpe": sharpe,
|
||||
"final_capital": capital,
|
||||
"trades_per_year": total_signals / test_years if test_years > 0 else 0,
|
||||
"daily_pnl": (capital - 1000) / test_days if test_days > 0 else 0,
|
||||
"folds": fold,
|
||||
}
|
||||
|
||||
|
||||
# Run for both assets with parameter sweep
|
||||
for asset in ["BTC", "ETH"]:
|
||||
print(f"\n{'#'*60}")
|
||||
print(f" {asset} 1H — WALK-FORWARD")
|
||||
print(f"{'#'*60}")
|
||||
|
||||
df = load_data(asset, "1h")
|
||||
|
||||
for lookahead in [3, 6]:
|
||||
for threshold in [0.55, 0.60, 0.65, 0.70]:
|
||||
result = walk_forward_backtest(
|
||||
df,
|
||||
train_size=15000,
|
||||
step_size=3000,
|
||||
lookahead=lookahead,
|
||||
threshold=threshold,
|
||||
position_pct=0.3,
|
||||
)
|
||||
if "error" in result:
|
||||
continue
|
||||
|
||||
print(f"\n LA={lookahead} thr={threshold:.2f}: "
|
||||
f"trades={result['total_trades']:4d} "
|
||||
f"acc={result['accuracy']:.1f}% "
|
||||
f"ret={result['total_return']:+.1f}% "
|
||||
f"ann={result['annualized_return']:+.1f}% "
|
||||
f"dd={result['max_drawdown']:.1f}% "
|
||||
f"sharpe={result['sharpe']:.2f} "
|
||||
f"€/day={result['daily_pnl']:.2f}")
|
||||
@@ -0,0 +1,340 @@
|
||||
"""Strategia 10: High Precision (target >80% accuracy).
|
||||
Approccio: combina TUTTI gli indicatori disponibili, usa ensemble di 5 modelli,
|
||||
trade SOLO quando tutti concordano. Pochi trade ma molto precisi.
|
||||
Usa leva 3x per compensare bassa frequenza.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import GradientBoostingClassifier, RandomForestClassifier, ExtraTreesClassifier
|
||||
from sklearn.linear_model import LogisticRegression
|
||||
from sklearn.svm import SVC
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from src.data.downloader import load_data
|
||||
from src.fractal.patterns import encode_candles, extract_body_ratios, extract_shadow_ratios
|
||||
from src.fractal.indicators import hurst_exponent, volatility_ratio
|
||||
|
||||
LEVERAGE = 3
|
||||
FEE_PCT = 0.001
|
||||
INITIAL_CAPITAL = 1000
|
||||
|
||||
|
||||
def build_rich_features(df: pd.DataFrame, i: int) -> np.ndarray | None:
|
||||
if i < 200:
|
||||
return None
|
||||
|
||||
o = df["open"].values
|
||||
h = df["high"].values
|
||||
l = df["low"].values
|
||||
c = df["close"].values
|
||||
v = df["volume"].values
|
||||
|
||||
feats = []
|
||||
|
||||
for w in [6, 12, 24, 48, 96]:
|
||||
if i < w:
|
||||
feats.extend([0] * 18)
|
||||
continue
|
||||
|
||||
win_c = c[i - w : i]
|
||||
win_o = o[i - w : i]
|
||||
win_h = h[i - w : i]
|
||||
win_l = l[i - w : i]
|
||||
win_v = v[i - w : i]
|
||||
|
||||
mn, mx = win_l.min(), max(win_h.max(), win_c.max())
|
||||
rng = mx - mn if mx - mn > 0 else 1e-10
|
||||
|
||||
total = win_h - win_l
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
body = np.abs(win_c - win_o) / total
|
||||
direction = np.sign(win_c - win_o)
|
||||
|
||||
log_c = np.log(np.where(win_c == 0, 1e-10, win_c))
|
||||
rets = np.diff(log_c)
|
||||
|
||||
v_mean = np.mean(win_v)
|
||||
|
||||
feats.extend([
|
||||
np.mean(rets) if len(rets) > 0 else 0,
|
||||
np.std(rets) if len(rets) > 0 else 0,
|
||||
np.sum(rets) if len(rets) > 0 else 0,
|
||||
float(pd.Series(rets).skew()) if len(rets) > 2 else 0,
|
||||
float(pd.Series(rets).kurtosis()) if len(rets) > 3 else 0,
|
||||
np.mean(body),
|
||||
np.std(body),
|
||||
np.mean(direction),
|
||||
np.mean(direction[-min(3, w):]),
|
||||
(win_c[-1] - mn) / rng,
|
||||
win_v[-1] / v_mean if v_mean > 0 else 1,
|
||||
np.max(body) - np.min(body),
|
||||
np.corrcoef(rets[:-1], rets[1:])[0, 1] if len(rets) > 1 and np.std(rets) > 0 else 0,
|
||||
np.max(rets) if len(rets) > 0 else 0,
|
||||
np.min(rets) if len(rets) > 0 else 0,
|
||||
np.mean(np.abs(rets)) if len(rets) > 0 else 0,
|
||||
np.sum(direction == 1) / w,
|
||||
np.sum(direction == -1) / w,
|
||||
])
|
||||
|
||||
# Hurst on different windows
|
||||
for w in [48, 96]:
|
||||
ret_w = np.diff(np.log(np.where(c[max(0,i-w):i] == 0, 1e-10, c[max(0,i-w):i])))
|
||||
if len(ret_w) > 20:
|
||||
feats.append(hurst_exponent(ret_w, max_lag=min(len(ret_w)//4, 15)))
|
||||
else:
|
||||
feats.append(0.5)
|
||||
|
||||
# Volatility ratios
|
||||
feats.append(volatility_ratio(c[max(0,i-48):i], fast=6, slow=48))
|
||||
feats.append(volatility_ratio(c[max(0,i-96):i], fast=12, slow=96))
|
||||
|
||||
# ATR normalized
|
||||
tr = np.maximum(h[i-14:i] - l[i-14:i],
|
||||
np.maximum(np.abs(h[i-14:i] - np.roll(c[i-14:i], 1)),
|
||||
np.abs(l[i-14:i] - np.roll(c[i-14:i], 1))))
|
||||
atr = np.mean(tr[1:])
|
||||
feats.append(atr / c[i-1] if c[i-1] > 0 else 0)
|
||||
|
||||
# Position in range
|
||||
h48 = np.max(h[i-48:i])
|
||||
l48 = np.min(l[i-48:i])
|
||||
r48 = h48 - l48
|
||||
feats.append((c[i-1] - l48) / r48 if r48 > 0 else 0.5)
|
||||
|
||||
h96 = np.max(h[i-96:i])
|
||||
l96 = np.min(l[i-96:i])
|
||||
r96 = h96 - l96
|
||||
feats.append((c[i-1] - l96) / r96 if r96 > 0 else 0.5)
|
||||
|
||||
return np.nan_to_num(np.array(feats), nan=0, posinf=1e6, neginf=-1e6)
|
||||
|
||||
|
||||
def run_high_precision(asset: str, lookahead: int = 3):
|
||||
print(f"\n{'#'*60}")
|
||||
print(f" {asset} 1H — HIGH PRECISION (LA={lookahead})")
|
||||
print(f"{'#'*60}")
|
||||
|
||||
df = load_data(asset, "1h")
|
||||
close = df["close"].values
|
||||
n = len(df)
|
||||
|
||||
MIN_RETURN = 0.003
|
||||
|
||||
# Build dataset
|
||||
print(" Building features...")
|
||||
X_all, y_all, idx_all = [], [], []
|
||||
for i in range(200, n - lookahead, 1):
|
||||
f = build_rich_features(df, i)
|
||||
if f is None:
|
||||
continue
|
||||
ret = (close[i + lookahead - 1] - close[i - 1]) / close[i - 1]
|
||||
if abs(ret) < MIN_RETURN:
|
||||
continue
|
||||
X_all.append(f)
|
||||
y_all.append(1 if ret > 0 else 0)
|
||||
idx_all.append(i)
|
||||
|
||||
X = np.array(X_all)
|
||||
y = np.array(y_all)
|
||||
idx_arr = np.array(idx_all)
|
||||
|
||||
print(f" Samples: {len(X)}, Features: {X.shape[1]}, Up: {np.mean(y)*100:.1f}%")
|
||||
|
||||
# Walk-forward with 5-model ensemble
|
||||
TRAIN_SIZE = 15000
|
||||
STEP_SIZE = 3000
|
||||
|
||||
models_config = [
|
||||
("GB1", GradientBoostingClassifier(n_estimators=200, max_depth=4, min_samples_leaf=30, learning_rate=0.05, subsample=0.8, random_state=42)),
|
||||
("GB2", GradientBoostingClassifier(n_estimators=300, max_depth=5, min_samples_leaf=50, learning_rate=0.03, subsample=0.7, random_state=123)),
|
||||
("RF", RandomForestClassifier(n_estimators=300, max_depth=8, min_samples_leaf=30, class_weight="balanced", random_state=42, n_jobs=-1)),
|
||||
("ET", ExtraTreesClassifier(n_estimators=300, max_depth=8, min_samples_leaf=30, class_weight="balanced", random_state=42, n_jobs=-1)),
|
||||
("LR", LogisticRegression(max_iter=1000, C=0.1, random_state=42)),
|
||||
]
|
||||
|
||||
capital = float(INITIAL_CAPITAL)
|
||||
all_trades = []
|
||||
equity = [capital]
|
||||
|
||||
fold = 0
|
||||
start = 0
|
||||
|
||||
while start + TRAIN_SIZE + STEP_SIZE < len(X):
|
||||
train_end = start + TRAIN_SIZE
|
||||
test_end = min(train_end + STEP_SIZE, len(X))
|
||||
|
||||
X_tr, y_tr = X[start:train_end], y[start:train_end]
|
||||
X_te, y_te = X[train_end:test_end], y[train_end:test_end]
|
||||
idx_te = idx_arr[train_end:test_end]
|
||||
|
||||
scaler = StandardScaler()
|
||||
X_tr_s = scaler.fit_transform(X_tr)
|
||||
X_te_s = scaler.transform(X_te)
|
||||
|
||||
# Train all models
|
||||
trained = []
|
||||
for name, model in models_config:
|
||||
m = type(model)(**model.get_params())
|
||||
m.fit(X_tr_s, y_tr)
|
||||
trained.append((name, m))
|
||||
|
||||
# Test with consensus voting
|
||||
for j in range(len(X_te)):
|
||||
votes_up = 0
|
||||
votes_down = 0
|
||||
max_conf = 0
|
||||
|
||||
for name, m in trained:
|
||||
proba = m.predict_proba(X_te_s[j:j+1])[0]
|
||||
up_idx = list(m.classes_).index(1)
|
||||
p_up = proba[up_idx]
|
||||
|
||||
if p_up >= 0.60:
|
||||
votes_up += 1
|
||||
max_conf = max(max_conf, p_up)
|
||||
elif p_up <= 0.40:
|
||||
votes_down += 1
|
||||
max_conf = max(max_conf, 1 - p_up)
|
||||
|
||||
i = idx_te[j]
|
||||
actual_ret = (close[i + lookahead - 1] - close[i - 1]) / close[i - 1]
|
||||
|
||||
# Trade only with strong consensus
|
||||
min_votes = 4 # at least 4 out of 5 models agree
|
||||
direction = None
|
||||
if votes_up >= min_votes:
|
||||
direction = "long"
|
||||
elif votes_down >= min_votes:
|
||||
direction = "short"
|
||||
|
||||
if direction:
|
||||
if direction == "long":
|
||||
trade_ret = actual_ret
|
||||
else:
|
||||
trade_ret = -actual_ret
|
||||
|
||||
net_ret = trade_ret * LEVERAGE - FEE_PCT * 2 * LEVERAGE
|
||||
pos_size = 0.2 # 20% of capital per trade
|
||||
pnl = capital * pos_size * net_ret
|
||||
capital += pnl
|
||||
capital = max(capital, 0)
|
||||
equity.append(capital)
|
||||
|
||||
is_correct = (direction == "long" and actual_ret > 0) or (direction == "short" and actual_ret < 0)
|
||||
all_trades.append({
|
||||
"fold": fold,
|
||||
"idx": i,
|
||||
"direction": direction,
|
||||
"votes_up": votes_up,
|
||||
"votes_down": votes_down,
|
||||
"actual_ret": actual_ret,
|
||||
"net_ret": net_ret,
|
||||
"pnl": pnl,
|
||||
"correct": is_correct,
|
||||
})
|
||||
|
||||
fold += 1
|
||||
start += STEP_SIZE
|
||||
|
||||
if not all_trades:
|
||||
print(" No trades generated!")
|
||||
return
|
||||
|
||||
trades_df = pd.DataFrame(all_trades)
|
||||
n_correct = trades_df["correct"].sum()
|
||||
n_total = len(trades_df)
|
||||
accuracy = n_correct / n_total * 100
|
||||
|
||||
test_candles = idx_arr[-1] - idx_arr[TRAIN_SIZE]
|
||||
test_days = test_candles / 24
|
||||
test_years = test_days / 365.25
|
||||
|
||||
ann_ret = ((capital / INITIAL_CAPITAL) ** (1 / test_years) - 1) * 100 if test_years > 0 and capital > 0 else -100
|
||||
daily_pnl = (capital - INITIAL_CAPITAL) / test_days if test_days > 0 else 0
|
||||
|
||||
# Max DD
|
||||
peak = equity[0]
|
||||
max_dd = 0
|
||||
for v in equity:
|
||||
if v > peak:
|
||||
peak = v
|
||||
dd = (peak - v) / peak if peak > 0 else 0
|
||||
max_dd = max(max_dd, dd)
|
||||
|
||||
print(f"\n RISULTATI:")
|
||||
print(f" Trades: {n_total}")
|
||||
print(f" Accuracy: {accuracy:.1f}%")
|
||||
print(f" Return: {(capital-INITIAL_CAPITAL)/INITIAL_CAPITAL*100:+.1f}%")
|
||||
print(f" Annualized: {ann_ret:+.1f}%")
|
||||
print(f" Max Drawdown: {max_dd*100:.1f}%")
|
||||
print(f" Capital: €{capital:.0f}")
|
||||
print(f" Trades/year: {n_total/test_years:.0f}")
|
||||
print(f" €/day avg: €{daily_pnl:.2f}")
|
||||
|
||||
# Consensus threshold sweep
|
||||
print(f"\n --- CONSENSUS SWEEP ---")
|
||||
for min_v in [3, 4, 5]:
|
||||
for ind_thr in [0.55, 0.60, 0.65]:
|
||||
cap = float(INITIAL_CAPITAL)
|
||||
trades_count = 0
|
||||
correct_count = 0
|
||||
eq = [cap]
|
||||
|
||||
fold_s = 0
|
||||
start_s = 0
|
||||
while start_s + TRAIN_SIZE + STEP_SIZE < len(X):
|
||||
train_end_s = start_s + TRAIN_SIZE
|
||||
test_end_s = min(train_end_s + STEP_SIZE, len(X))
|
||||
|
||||
X_tr_s2 = scaler.fit_transform(X[start_s:train_end_s])
|
||||
X_te_s2 = scaler.transform(X[train_end_s:test_end_s])
|
||||
y_tr_s2 = y[start_s:train_end_s]
|
||||
idx_te_s2 = idx_arr[train_end_s:test_end_s]
|
||||
|
||||
trained_s = []
|
||||
for name, model in models_config:
|
||||
m2 = type(model)(**model.get_params())
|
||||
m2.fit(X_tr_s2, y_tr_s2)
|
||||
trained_s.append(m2)
|
||||
|
||||
for j in range(len(X_te_s2)):
|
||||
vu = sum(1 for m2 in trained_s
|
||||
if m2.predict_proba(X_te_s2[j:j+1])[0][list(m2.classes_).index(1)] >= ind_thr)
|
||||
vd = sum(1 for m2 in trained_s
|
||||
if m2.predict_proba(X_te_s2[j:j+1])[0][list(m2.classes_).index(1)] <= (1-ind_thr))
|
||||
|
||||
i_s = idx_te_s2[j]
|
||||
ar = (close[i_s + lookahead - 1] - close[i_s - 1]) / close[i_s - 1]
|
||||
|
||||
d = None
|
||||
if vu >= min_v:
|
||||
d = "long"
|
||||
elif vd >= min_v:
|
||||
d = "short"
|
||||
|
||||
if d:
|
||||
tr = ar if d == "long" else -ar
|
||||
nr = tr * LEVERAGE - FEE_PCT * 2 * LEVERAGE
|
||||
cap += cap * 0.2 * nr
|
||||
cap = max(cap, 0)
|
||||
eq.append(cap)
|
||||
trades_count += 1
|
||||
if (d == "long" and ar > 0) or (d == "short" and ar < 0):
|
||||
correct_count += 1
|
||||
|
||||
start_s += STEP_SIZE
|
||||
|
||||
if trades_count > 0:
|
||||
acc_s = correct_count / trades_count * 100
|
||||
ret_s = (cap - INITIAL_CAPITAL) / INITIAL_CAPITAL * 100
|
||||
ann_s = ((cap / INITIAL_CAPITAL) ** (1 / test_years) - 1) * 100 if test_years > 0 and cap > 0 else -100
|
||||
dpnl = (cap - INITIAL_CAPITAL) / test_days if test_days > 0 else 0
|
||||
print(f" min_votes={min_v} ind_thr={ind_thr:.2f}: trades={trades_count:4d} acc={acc_s:.1f}% ret={ret_s:+.1f}% ann={ann_s:+.1f}% €/day={dpnl:.2f}")
|
||||
|
||||
|
||||
for asset in ["BTC", "ETH"]:
|
||||
for la in [3, 6]:
|
||||
run_high_precision(asset, la)
|
||||
@@ -0,0 +1,209 @@
|
||||
"""Backtesting engine with fee support and performance metrics."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from enum import Enum
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
|
||||
class Side(Enum):
|
||||
LONG = 1
|
||||
SHORT = -1
|
||||
|
||||
|
||||
@dataclass
|
||||
class Trade:
|
||||
entry_idx: int
|
||||
exit_idx: int
|
||||
side: Side
|
||||
entry_price: float
|
||||
exit_price: float
|
||||
size: float
|
||||
fee_pct: float
|
||||
|
||||
@property
|
||||
def gross_pnl(self) -> float:
|
||||
if self.side == Side.LONG:
|
||||
return (self.exit_price - self.entry_price) * self.size
|
||||
return (self.entry_price - self.exit_price) * self.size
|
||||
|
||||
@property
|
||||
def fee(self) -> float:
|
||||
return self.fee_pct * (self.entry_price + self.exit_price) * self.size
|
||||
|
||||
@property
|
||||
def net_pnl(self) -> float:
|
||||
return self.gross_pnl - self.fee
|
||||
|
||||
@property
|
||||
def net_return(self) -> float:
|
||||
cost = self.entry_price * self.size + self.fee_pct * self.entry_price * self.size
|
||||
if cost == 0:
|
||||
return 0.0
|
||||
return self.net_pnl / cost
|
||||
|
||||
|
||||
@dataclass
|
||||
class BacktestResult:
|
||||
trades: list[Trade]
|
||||
initial_capital: float
|
||||
final_capital: float
|
||||
equity_curve: list[float]
|
||||
|
||||
@property
|
||||
def total_trades(self) -> int:
|
||||
return len(self.trades)
|
||||
|
||||
@property
|
||||
def win_rate(self) -> float:
|
||||
if not self.trades:
|
||||
return 0.0
|
||||
wins = sum(1 for t in self.trades if t.net_pnl > 0)
|
||||
return wins / len(self.trades)
|
||||
|
||||
@property
|
||||
def total_return(self) -> float:
|
||||
if self.initial_capital == 0:
|
||||
return 0.0
|
||||
return (self.final_capital - self.initial_capital) / self.initial_capital
|
||||
|
||||
@property
|
||||
def annualized_return(self) -> float:
|
||||
if not self.trades or self.total_return <= -1:
|
||||
return -1.0
|
||||
days = (self.trades[-1].exit_idx - self.trades[0].entry_idx) / 24
|
||||
if days <= 0:
|
||||
return 0.0
|
||||
years = days / 365.25
|
||||
if years == 0:
|
||||
return 0.0
|
||||
return (1 + self.total_return) ** (1 / years) - 1
|
||||
|
||||
@property
|
||||
def max_drawdown(self) -> float:
|
||||
if not self.equity_curve:
|
||||
return 0.0
|
||||
peak = self.equity_curve[0]
|
||||
max_dd = 0.0
|
||||
for val in self.equity_curve:
|
||||
if val > peak:
|
||||
peak = val
|
||||
dd = (peak - val) / peak if peak > 0 else 0
|
||||
if dd > max_dd:
|
||||
max_dd = dd
|
||||
return max_dd
|
||||
|
||||
@property
|
||||
def sharpe_ratio(self) -> float:
|
||||
if len(self.equity_curve) < 2:
|
||||
return 0.0
|
||||
eq = np.array(self.equity_curve)
|
||||
returns = np.diff(eq) / eq[:-1]
|
||||
returns = returns[np.isfinite(returns)]
|
||||
if len(returns) == 0 or np.std(returns) == 0:
|
||||
return 0.0
|
||||
return float(np.mean(returns) / np.std(returns) * np.sqrt(252 * 24))
|
||||
|
||||
@property
|
||||
def profit_factor(self) -> float:
|
||||
gross_wins = sum(t.net_pnl for t in self.trades if t.net_pnl > 0)
|
||||
gross_losses = abs(sum(t.net_pnl for t in self.trades if t.net_pnl < 0))
|
||||
if gross_losses == 0:
|
||||
return float("inf") if gross_wins > 0 else 0.0
|
||||
return gross_wins / gross_losses
|
||||
|
||||
def summary(self) -> dict:
|
||||
return {
|
||||
"total_trades": self.total_trades,
|
||||
"win_rate": round(self.win_rate * 100, 1),
|
||||
"total_return_pct": round(self.total_return * 100, 1),
|
||||
"annualized_return_pct": round(self.annualized_return * 100, 1),
|
||||
"max_drawdown_pct": round(self.max_drawdown * 100, 1),
|
||||
"sharpe_ratio": round(self.sharpe_ratio, 2),
|
||||
"profit_factor": round(self.profit_factor, 2),
|
||||
"initial_capital": self.initial_capital,
|
||||
"final_capital": round(self.final_capital, 2),
|
||||
}
|
||||
|
||||
|
||||
def run_backtest(
|
||||
df: pd.DataFrame,
|
||||
signals: pd.Series,
|
||||
initial_capital: float = 1000.0,
|
||||
fee_pct: float = 0.001,
|
||||
position_size_pct: float = 1.0,
|
||||
max_hold_candles: int = 24,
|
||||
) -> BacktestResult:
|
||||
"""Run backtest on signals.
|
||||
|
||||
signals: Series with same index as df.
|
||||
+1 = go long, -1 = go short, 0 = no signal
|
||||
"""
|
||||
capital = initial_capital
|
||||
trades: list[Trade] = []
|
||||
equity_curve: list[float] = [capital]
|
||||
in_position = False
|
||||
entry_idx = 0
|
||||
entry_price = 0.0
|
||||
current_side = Side.LONG
|
||||
size = 0.0
|
||||
|
||||
for i in range(len(df)):
|
||||
sig = signals.iloc[i] if i < len(signals) else 0
|
||||
|
||||
if in_position:
|
||||
hold_time = i - entry_idx
|
||||
exit_price = df["close"].iloc[i]
|
||||
should_exit = (
|
||||
hold_time >= max_hold_candles
|
||||
or (current_side == Side.LONG and sig == -1)
|
||||
or (current_side == Side.SHORT and sig == 1)
|
||||
)
|
||||
|
||||
if should_exit:
|
||||
trade = Trade(
|
||||
entry_idx=entry_idx,
|
||||
exit_idx=i,
|
||||
side=current_side,
|
||||
entry_price=entry_price,
|
||||
exit_price=exit_price,
|
||||
size=size,
|
||||
fee_pct=fee_pct,
|
||||
)
|
||||
capital += trade.net_pnl
|
||||
trades.append(trade)
|
||||
in_position = False
|
||||
|
||||
if not in_position and sig != 0 and capital > 0:
|
||||
entry_idx = i
|
||||
entry_price = df["close"].iloc[i]
|
||||
current_side = Side.LONG if sig > 0 else Side.SHORT
|
||||
alloc = capital * position_size_pct
|
||||
size = alloc / entry_price
|
||||
in_position = True
|
||||
|
||||
equity_curve.append(capital)
|
||||
|
||||
if in_position:
|
||||
exit_price = df["close"].iloc[-1]
|
||||
trade = Trade(
|
||||
entry_idx=entry_idx,
|
||||
exit_idx=len(df) - 1,
|
||||
side=current_side,
|
||||
entry_price=entry_price,
|
||||
exit_price=exit_price,
|
||||
size=size,
|
||||
fee_pct=fee_pct,
|
||||
)
|
||||
capital += trade.net_pnl
|
||||
trades.append(trade)
|
||||
equity_curve.append(capital)
|
||||
|
||||
return BacktestResult(
|
||||
trades=trades,
|
||||
initial_capital=initial_capital,
|
||||
final_capital=capital,
|
||||
equity_curve=equity_curve,
|
||||
)
|
||||
@@ -0,0 +1,256 @@
|
||||
"""Download historical OHLCV data. Primary: Cerbero MCP. Fallback: Binance/ccxt."""
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
import requests
|
||||
from tqdm import tqdm
|
||||
|
||||
DATA_DIR = Path(__file__).resolve().parents[2] / "data" / "raw"
|
||||
|
||||
CERBERO_URL = "https://cerbero-mcp.tielogic.xyz"
|
||||
CERBERO_TOKEN = "_hm0FkyC67P9OXJTy7R9SE2lfhGz_Wa6i89KqH_uXrk"
|
||||
CERBERO_HEADERS = {
|
||||
"Authorization": f"Bearer {CERBERO_TOKEN}",
|
||||
"X-Bot-Tag": "pythagoras-downloader",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
ASSETS = {
|
||||
"BTC": {
|
||||
"deribit": {"instrument": "BTC-PERPETUAL", "start": "2018-09-01"},
|
||||
"binance_symbol": "BTC/USDT",
|
||||
"binance_start": "2018-01-01",
|
||||
},
|
||||
"ETH": {
|
||||
"deribit": {"instrument": "ETH-PERPETUAL", "start": "2019-06-01"},
|
||||
"binance_symbol": "ETH/USDT",
|
||||
"binance_start": "2018-01-01",
|
||||
},
|
||||
}
|
||||
|
||||
TIMEFRAMES = ["1m", "5m", "15m", "1h"]
|
||||
|
||||
DERIBIT_RESOLUTION = {"1m": "1", "5m": "5", "15m": "15", "1h": "60"}
|
||||
|
||||
TF_SECONDS = {"1m": 60, "5m": 300, "15m": 900, "1h": 3600}
|
||||
|
||||
MAX_DAYS_PER_REQUEST = {"1m": 1, "5m": 5, "15m": 15, "1h": 30}
|
||||
|
||||
|
||||
def _parquet_path(asset: str, tf: str) -> Path:
|
||||
return DATA_DIR / f"{asset.lower()}_{tf}.parquet"
|
||||
|
||||
|
||||
def _fetch_deribit(instrument: str, resolution: str, start: str, end: str) -> list[dict]:
|
||||
resp = requests.post(
|
||||
f"{CERBERO_URL}/mcp-deribit/tools/get_historical",
|
||||
headers=CERBERO_HEADERS,
|
||||
json={
|
||||
"instrument": instrument,
|
||||
"start_date": start,
|
||||
"end_date": end,
|
||||
"resolution": resolution,
|
||||
},
|
||||
timeout=30,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
return data.get("candles", [])
|
||||
|
||||
|
||||
def _fetch_binance(symbol: str, tf: str, since_ms: int, limit: int = 1000) -> list[list]:
|
||||
import ccxt
|
||||
exchange = ccxt.binance({"enableRateLimit": True, "options": {"defaultType": "spot"}})
|
||||
return exchange.fetch_ohlcv(symbol, tf, since=since_ms, limit=limit)
|
||||
|
||||
|
||||
def _download_cerbero_range(
|
||||
instrument: str, resolution: str, tf: str, start_date: str, end_date: str
|
||||
) -> pd.DataFrame:
|
||||
all_candles: list[dict] = []
|
||||
max_days = MAX_DAYS_PER_REQUEST[tf]
|
||||
current = datetime.fromisoformat(start_date)
|
||||
end = datetime.fromisoformat(end_date)
|
||||
|
||||
pbar = tqdm(
|
||||
total=(end - current).days,
|
||||
desc=f" Cerbero {instrument} {tf}",
|
||||
unit="days",
|
||||
)
|
||||
|
||||
while current < end:
|
||||
chunk_end = min(current + timedelta(days=max_days), end)
|
||||
start_str = current.strftime("%Y-%m-%d")
|
||||
end_str = chunk_end.strftime("%Y-%m-%d")
|
||||
|
||||
for attempt in range(3):
|
||||
try:
|
||||
candles = _fetch_deribit(instrument, resolution, start_str, end_str)
|
||||
all_candles.extend(candles)
|
||||
break
|
||||
except Exception as e:
|
||||
if attempt == 2:
|
||||
print(f" SKIP {start_str}→{end_str}: {e}")
|
||||
time.sleep(2 ** attempt)
|
||||
|
||||
pbar.update(max_days)
|
||||
current = chunk_end
|
||||
|
||||
pbar.close()
|
||||
|
||||
if not all_candles:
|
||||
return pd.DataFrame()
|
||||
|
||||
df = pd.DataFrame(all_candles)
|
||||
df = df.rename(columns={"timestamp": "timestamp"})
|
||||
df["timestamp"] = df["timestamp"].astype("int64")
|
||||
return df.drop_duplicates(subset="timestamp").sort_values("timestamp").reset_index(drop=True)
|
||||
|
||||
|
||||
def _download_binance_range(
|
||||
symbol: str, tf: str, start_date: str, end_date: str
|
||||
) -> pd.DataFrame:
|
||||
import ccxt
|
||||
|
||||
exchange = ccxt.binance({
|
||||
"enableRateLimit": True,
|
||||
"timeout": 30000,
|
||||
"options": {"defaultType": "spot", "adjustForTimeDifference": True},
|
||||
})
|
||||
exchange.load_markets()
|
||||
|
||||
start_ms = int(datetime.fromisoformat(start_date).replace(tzinfo=timezone.utc).timestamp() * 1000)
|
||||
end_ms = int(datetime.fromisoformat(end_date).replace(tzinfo=timezone.utc).timestamp() * 1000)
|
||||
tf_ms = TF_SECONDS[tf] * 1000
|
||||
all_rows: list[list] = []
|
||||
|
||||
pbar = tqdm(desc=f" Binance {symbol} {tf}", unit=" candles")
|
||||
|
||||
since = start_ms
|
||||
while since < end_ms:
|
||||
for attempt in range(3):
|
||||
try:
|
||||
ohlcv = exchange.fetch_ohlcv(symbol, tf, since=since, limit=1000)
|
||||
break
|
||||
except ccxt.RateLimitExceeded:
|
||||
time.sleep(10)
|
||||
ohlcv = []
|
||||
except Exception as e:
|
||||
if attempt == 2:
|
||||
print(f" ERR: {e}")
|
||||
time.sleep(2 ** attempt)
|
||||
ohlcv = []
|
||||
|
||||
if not ohlcv:
|
||||
break
|
||||
|
||||
filtered = [c for c in ohlcv if c[0] < end_ms]
|
||||
all_rows.extend(filtered)
|
||||
pbar.update(len(filtered))
|
||||
since = ohlcv[-1][0] + tf_ms
|
||||
|
||||
if len(ohlcv) < 1000 or ohlcv[-1][0] >= end_ms:
|
||||
break
|
||||
|
||||
pbar.close()
|
||||
|
||||
if not all_rows:
|
||||
return pd.DataFrame()
|
||||
|
||||
cols = ["timestamp", "open", "high", "low", "close", "volume"]
|
||||
df = pd.DataFrame(all_rows, columns=cols)
|
||||
df["timestamp"] = df["timestamp"].astype("int64")
|
||||
return df.drop_duplicates(subset="timestamp").sort_values("timestamp").reset_index(drop=True)
|
||||
|
||||
|
||||
def download_asset(asset: str, tf: str) -> pd.DataFrame:
|
||||
path = _parquet_path(asset, tf)
|
||||
info = ASSETS[asset]
|
||||
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
|
||||
resolution = DERIBIT_RESOLUTION[tf]
|
||||
|
||||
parts: list[pd.DataFrame] = []
|
||||
|
||||
binance_start = info["binance_start"]
|
||||
deribit_start = info["deribit"]["start"]
|
||||
|
||||
if binance_start < deribit_start:
|
||||
print(f"\n Fase 1: Binance {binance_start} → {deribit_start}")
|
||||
df_binance = _download_binance_range(
|
||||
info["binance_symbol"], tf, binance_start, deribit_start
|
||||
)
|
||||
if not df_binance.empty:
|
||||
parts.append(df_binance)
|
||||
|
||||
print(f"\n Fase 2: Cerbero/Deribit {deribit_start} → {today}")
|
||||
df_deribit = _download_cerbero_range(
|
||||
info["deribit"]["instrument"], resolution, tf, deribit_start, today
|
||||
)
|
||||
if not df_deribit.empty:
|
||||
parts.append(df_deribit)
|
||||
|
||||
if not parts:
|
||||
print(f" VUOTO: {asset} {tf}")
|
||||
return pd.DataFrame()
|
||||
|
||||
df = pd.concat(parts).drop_duplicates(subset="timestamp").sort_values("timestamp").reset_index(drop=True)
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
df.to_parquet(path, index=False)
|
||||
|
||||
first = datetime.fromtimestamp(df["timestamp"].iloc[0] / 1000, tz=timezone.utc)
|
||||
last = datetime.fromtimestamp(df["timestamp"].iloc[-1] / 1000, tz=timezone.utc)
|
||||
print(f" ✓ {path.name}: {len(df)} candele [{first.date()} → {last.date()}]")
|
||||
|
||||
return df
|
||||
|
||||
|
||||
def download_all() -> dict[str, dict[str, pd.DataFrame]]:
|
||||
result: dict[str, dict[str, pd.DataFrame]] = {}
|
||||
for asset in ASSETS:
|
||||
print(f"\n{'='*60}")
|
||||
print(f" {asset}")
|
||||
print(f"{'='*60}")
|
||||
result[asset] = {}
|
||||
for tf in TIMEFRAMES:
|
||||
result[asset][tf] = download_asset(asset, tf)
|
||||
return result
|
||||
|
||||
|
||||
def load_data(asset: str = "BTC", tf: str = "1h") -> pd.DataFrame:
|
||||
path = _parquet_path(asset, tf)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Dati non trovati: {path}. Esegui download_all().")
|
||||
df = pd.read_parquet(path)
|
||||
df["datetime"] = pd.to_datetime(df["timestamp"], unit="ms", utc=True)
|
||||
return df
|
||||
|
||||
|
||||
def data_summary() -> pd.DataFrame:
|
||||
rows = []
|
||||
for asset in ASSETS:
|
||||
for tf in TIMEFRAMES:
|
||||
path = _parquet_path(asset, tf)
|
||||
if path.exists():
|
||||
df = pd.read_parquet(path)
|
||||
first = datetime.fromtimestamp(df["timestamp"].iloc[0] / 1000, tz=timezone.utc)
|
||||
last = datetime.fromtimestamp(df["timestamp"].iloc[-1] / 1000, tz=timezone.utc)
|
||||
rows.append({
|
||||
"asset": asset,
|
||||
"tf": tf,
|
||||
"candles": len(df),
|
||||
"from": first.date(),
|
||||
"to": last.date(),
|
||||
"size_mb": round(path.stat().st_size / 1024 / 1024, 1),
|
||||
})
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
download_all()
|
||||
print("\n" + "=" * 60)
|
||||
print(data_summary().to_string(index=False))
|
||||
@@ -0,0 +1,159 @@
|
||||
"""Fractal indicators: Hurst exponent, fractal dimension, self-similarity."""
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
from scipy.stats import linregress
|
||||
|
||||
|
||||
def hurst_exponent(series: np.ndarray, max_lag: int | None = None) -> float:
|
||||
"""Compute Hurst exponent via R/S analysis.
|
||||
H > 0.5: trending (persistent), H < 0.5: mean-reverting, H ≈ 0.5: random walk.
|
||||
"""
|
||||
n = len(series)
|
||||
if n < 20:
|
||||
return 0.5
|
||||
|
||||
if max_lag is None:
|
||||
max_lag = min(n // 4, 100)
|
||||
|
||||
lags = range(10, max_lag + 1)
|
||||
rs_values = []
|
||||
lag_values = []
|
||||
|
||||
for lag in lags:
|
||||
rs_list = []
|
||||
for start in range(0, n - lag, lag):
|
||||
chunk = series[start : start + lag]
|
||||
if len(chunk) < lag:
|
||||
continue
|
||||
mean = np.mean(chunk)
|
||||
deviations = np.cumsum(chunk - mean)
|
||||
r = np.max(deviations) - np.min(deviations)
|
||||
s = np.std(chunk, ddof=1)
|
||||
if s > 0:
|
||||
rs_list.append(r / s)
|
||||
|
||||
if rs_list:
|
||||
rs_values.append(np.mean(rs_list))
|
||||
lag_values.append(lag)
|
||||
|
||||
if len(lag_values) < 3:
|
||||
return 0.5
|
||||
|
||||
log_lags = np.log(lag_values)
|
||||
log_rs = np.log(rs_values)
|
||||
slope, _, _, _, _ = linregress(log_lags, log_rs)
|
||||
return float(np.clip(slope, 0, 1))
|
||||
|
||||
|
||||
def rolling_hurst(close: np.ndarray, window: int = 100, step: int = 1) -> np.ndarray:
|
||||
"""Compute rolling Hurst exponent."""
|
||||
n = len(close)
|
||||
result = np.full(n, 0.5)
|
||||
returns = np.diff(np.log(np.where(close == 0, 1e-10, close)))
|
||||
|
||||
for i in range(window, n, step):
|
||||
h = hurst_exponent(returns[i - window : i])
|
||||
result[i] = h
|
||||
for j in range(1, min(step, n - i)):
|
||||
result[i + j] = h
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def fractal_dimension_higuchi(series: np.ndarray, k_max: int = 10) -> float:
|
||||
"""Higuchi fractal dimension of a time series."""
|
||||
n = len(series)
|
||||
if n < k_max * 2:
|
||||
return 1.5
|
||||
|
||||
lk = []
|
||||
x = np.arange(1, k_max + 1)
|
||||
|
||||
for k in range(1, k_max + 1):
|
||||
lm_list = []
|
||||
for m in range(1, k + 1):
|
||||
indices = np.arange(m - 1, n, k)
|
||||
if len(indices) < 2:
|
||||
continue
|
||||
vals = series[indices]
|
||||
length = np.sum(np.abs(np.diff(vals)))
|
||||
norm = (n - 1) / (k * ((n - m) // k) * k)
|
||||
lm_list.append(length * norm)
|
||||
|
||||
if lm_list:
|
||||
lk.append(np.mean(lm_list))
|
||||
|
||||
if len(lk) < 3:
|
||||
return 1.5
|
||||
|
||||
log_k = np.log(1.0 / x[: len(lk)])
|
||||
log_lk = np.log(np.array(lk))
|
||||
slope, _, _, _, _ = linregress(log_k, log_lk)
|
||||
return float(np.clip(slope, 1.0, 2.0))
|
||||
|
||||
|
||||
def self_similarity_score(close: np.ndarray, window: int, scales: list[int] | None = None) -> float:
|
||||
"""Measure self-similarity across multiple time scales.
|
||||
Higher score = more fractal (self-similar) structure.
|
||||
"""
|
||||
if scales is None:
|
||||
scales = [2, 3, 4, 6]
|
||||
|
||||
if len(close) < window:
|
||||
return 0.0
|
||||
|
||||
base = close[-window:]
|
||||
base_returns = np.diff(np.log(np.where(base == 0, 1e-10, base)))
|
||||
if np.std(base_returns) == 0:
|
||||
return 0.0
|
||||
|
||||
similarities = []
|
||||
for scale in scales:
|
||||
scaled_window = window * scale
|
||||
if scaled_window > len(close):
|
||||
continue
|
||||
|
||||
scaled = close[-scaled_window:]
|
||||
step = scale
|
||||
downsampled = scaled[::step][:window]
|
||||
|
||||
if len(downsampled) != len(base):
|
||||
downsampled = np.interp(
|
||||
np.linspace(0, 1, window),
|
||||
np.linspace(0, 1, len(downsampled)),
|
||||
downsampled,
|
||||
)
|
||||
|
||||
ds_returns = np.diff(np.log(np.where(downsampled == 0, 1e-10, downsampled)))
|
||||
|
||||
if len(ds_returns) != len(base_returns):
|
||||
ds_returns = np.interp(
|
||||
np.linspace(0, 1, len(base_returns)),
|
||||
np.linspace(0, 1, len(ds_returns)),
|
||||
ds_returns,
|
||||
)
|
||||
|
||||
std_ds = np.std(ds_returns)
|
||||
if std_ds == 0:
|
||||
continue
|
||||
|
||||
corr = np.corrcoef(base_returns, ds_returns)[0, 1]
|
||||
if np.isfinite(corr):
|
||||
similarities.append(abs(corr))
|
||||
|
||||
if not similarities:
|
||||
return 0.0
|
||||
return float(np.mean(similarities))
|
||||
|
||||
|
||||
def volatility_ratio(close: np.ndarray, fast: int = 12, slow: int = 48) -> float:
|
||||
"""Ratio of short-term to long-term volatility."""
|
||||
returns = np.diff(np.log(np.where(close == 0, 1e-10, close)))
|
||||
if len(returns) < slow:
|
||||
return 1.0
|
||||
fast_vol = np.std(returns[-fast:])
|
||||
slow_vol = np.std(returns[-slow:])
|
||||
if slow_vol == 0:
|
||||
return 1.0
|
||||
return float(fast_vol / slow_vol)
|
||||
@@ -0,0 +1,121 @@
|
||||
"""Fractal pattern detection and encoding for candlestick sequences."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import IntEnum
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
|
||||
class CandleType(IntEnum):
|
||||
DOWN = -1
|
||||
DOJI = 0
|
||||
UP = 1
|
||||
|
||||
|
||||
@dataclass
|
||||
class FractalPattern:
|
||||
sequence: tuple[CandleType, ...]
|
||||
start_idx: int
|
||||
end_idx: int
|
||||
body_ratios: tuple[float, ...]
|
||||
shadow_ratios: tuple[float, ...]
|
||||
|
||||
@property
|
||||
def length(self) -> int:
|
||||
return len(self.sequence)
|
||||
|
||||
@property
|
||||
def code(self) -> str:
|
||||
m = {CandleType.DOWN: "D", CandleType.DOJI: "0", CandleType.UP: "U"}
|
||||
return "".join(m[c] for c in self.sequence)
|
||||
|
||||
def __hash__(self) -> int:
|
||||
return hash(self.sequence)
|
||||
|
||||
|
||||
def classify_candle(open_: float, close: float, high: float, low: float, doji_threshold: float = 0.1) -> CandleType:
|
||||
body = abs(close - open_)
|
||||
total_range = high - low
|
||||
if total_range == 0:
|
||||
return CandleType.DOJI
|
||||
ratio = body / total_range
|
||||
if ratio < doji_threshold:
|
||||
return CandleType.DOJI
|
||||
return CandleType.UP if close > open_ else CandleType.DOWN
|
||||
|
||||
|
||||
def encode_candles(df: pd.DataFrame, doji_threshold: float = 0.1) -> np.ndarray:
|
||||
types = np.zeros(len(df), dtype=np.int8)
|
||||
body = np.abs(df["close"].values - df["open"].values)
|
||||
total = df["high"].values - df["low"].values
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
ratio = body / total
|
||||
|
||||
types[ratio < doji_threshold] = CandleType.DOJI
|
||||
bullish = (ratio >= doji_threshold) & (df["close"].values > df["open"].values)
|
||||
bearish = (ratio >= doji_threshold) & (df["close"].values <= df["open"].values)
|
||||
types[bullish] = CandleType.UP
|
||||
types[bearish] = CandleType.DOWN
|
||||
return types
|
||||
|
||||
|
||||
def extract_body_ratios(df: pd.DataFrame) -> np.ndarray:
|
||||
body = np.abs(df["close"].values - df["open"].values)
|
||||
total = df["high"].values - df["low"].values
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
return body / total
|
||||
|
||||
|
||||
def extract_shadow_ratios(df: pd.DataFrame) -> np.ndarray:
|
||||
o, c, h, l = df["open"].values, df["close"].values, df["high"].values, df["low"].values
|
||||
upper_shadow = h - np.maximum(o, c)
|
||||
lower_shadow = np.minimum(o, c) - l
|
||||
total = h - l
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
return (upper_shadow - lower_shadow) / total
|
||||
|
||||
|
||||
def find_patterns(df: pd.DataFrame, min_len: int = 3, max_len: int = 6, doji_threshold: float = 0.1) -> list[FractalPattern]:
|
||||
candle_types = encode_candles(df, doji_threshold)
|
||||
body_ratios = extract_body_ratios(df)
|
||||
shadow_ratios = extract_shadow_ratios(df)
|
||||
patterns: list[FractalPattern] = []
|
||||
|
||||
for length in range(min_len, max_len + 1):
|
||||
for i in range(len(df) - length):
|
||||
seq = tuple(CandleType(t) for t in candle_types[i : i + length])
|
||||
br = tuple(body_ratios[i : i + length])
|
||||
sr = tuple(shadow_ratios[i : i + length])
|
||||
patterns.append(FractalPattern(
|
||||
sequence=seq,
|
||||
start_idx=i,
|
||||
end_idx=i + length,
|
||||
body_ratios=br,
|
||||
shadow_ratios=sr,
|
||||
))
|
||||
|
||||
return patterns
|
||||
|
||||
|
||||
def pattern_frequency(patterns: list[FractalPattern]) -> pd.DataFrame:
|
||||
from collections import Counter
|
||||
codes = [p.code for p in patterns]
|
||||
counts = Counter(codes)
|
||||
total = len(codes)
|
||||
rows = [
|
||||
{"pattern": code, "count": cnt, "freq": cnt / total, "length": len(code)}
|
||||
for code, cnt in counts.most_common()
|
||||
]
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
def normalize_pattern_window(df: pd.DataFrame, start: int, end: int) -> np.ndarray:
|
||||
"""Normalize OHLC window to [0,1] range for comparison."""
|
||||
window = df.iloc[start:end][["open", "high", "low", "close"]].values
|
||||
mn = window.min()
|
||||
mx = window.max()
|
||||
if mx - mn == 0:
|
||||
return np.zeros_like(window)
|
||||
return (window - mn) / (mx - mn)
|
||||
@@ -0,0 +1,80 @@
|
||||
"""Fractal similarity measures: DTW, Hausdorff, correlation-based."""
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
from scipy.spatial.distance import directed_hausdorff
|
||||
from scipy.signal import correlate
|
||||
|
||||
|
||||
def dtw_distance(s1: np.ndarray, s2: np.ndarray) -> float:
|
||||
"""Dynamic Time Warping distance between two 1D sequences."""
|
||||
n, m = len(s1), len(s2)
|
||||
dtw = np.full((n + 1, m + 1), np.inf)
|
||||
dtw[0, 0] = 0.0
|
||||
|
||||
for i in range(1, n + 1):
|
||||
for j in range(1, m + 1):
|
||||
cost = abs(s1[i - 1] - s2[j - 1])
|
||||
dtw[i, j] = cost + min(dtw[i - 1, j], dtw[i, j - 1], dtw[i - 1, j - 1])
|
||||
|
||||
return dtw[n, m]
|
||||
|
||||
|
||||
def hausdorff_distance(s1: np.ndarray, s2: np.ndarray) -> float:
|
||||
"""Hausdorff distance between two OHLC windows (shape Nx4)."""
|
||||
if s1.ndim == 1:
|
||||
s1 = s1.reshape(-1, 1)
|
||||
if s2.ndim == 1:
|
||||
s2 = s2.reshape(-1, 1)
|
||||
d1 = directed_hausdorff(s1, s2)[0]
|
||||
d2 = directed_hausdorff(s2, s1)[0]
|
||||
return max(d1, d2)
|
||||
|
||||
|
||||
def cross_correlation(s1: np.ndarray, s2: np.ndarray) -> float:
|
||||
"""Max normalized cross-correlation between two 1D sequences."""
|
||||
if len(s1) == 0 or len(s2) == 0:
|
||||
return 0.0
|
||||
s1_norm = (s1 - np.mean(s1))
|
||||
s2_norm = (s2 - np.mean(s2))
|
||||
std1, std2 = np.std(s1), np.std(s2)
|
||||
if std1 == 0 or std2 == 0:
|
||||
return 0.0
|
||||
corr = correlate(s1_norm, s2_norm, mode="full")
|
||||
corr /= (std1 * std2 * len(s1))
|
||||
return float(np.max(np.abs(corr)))
|
||||
|
||||
|
||||
def cosine_similarity(v1: np.ndarray, v2: np.ndarray) -> float:
|
||||
"""Cosine similarity between two feature vectors."""
|
||||
dot = np.dot(v1, v2)
|
||||
n1, n2 = np.linalg.norm(v1), np.linalg.norm(v2)
|
||||
if n1 == 0 or n2 == 0:
|
||||
return 0.0
|
||||
return float(dot / (n1 * n2))
|
||||
|
||||
|
||||
def pattern_feature_vector(ohlc_window: np.ndarray) -> np.ndarray:
|
||||
"""Extract compact feature vector from normalized OHLC window.
|
||||
|
||||
Features: body ratios, shadow ratios, close-to-close returns,
|
||||
volatility, trend.
|
||||
"""
|
||||
o, h, l, c = ohlc_window[:, 0], ohlc_window[:, 1], ohlc_window[:, 2], ohlc_window[:, 3]
|
||||
total = h - l
|
||||
total = np.where(total == 0, 1e-10, total)
|
||||
|
||||
body = np.abs(c - o) / total
|
||||
upper_shadow = (h - np.maximum(o, c)) / total
|
||||
lower_shadow = (np.minimum(o, c) - l) / total
|
||||
returns = np.diff(c) / np.where(c[:-1] == 0, 1e-10, c[:-1])
|
||||
|
||||
features = np.concatenate([
|
||||
body,
|
||||
upper_shadow,
|
||||
lower_shadow,
|
||||
returns,
|
||||
[np.std(returns) if len(returns) > 0 else 0],
|
||||
[c[-1] - c[0]],
|
||||
])
|
||||
return features
|
||||
Reference in New Issue
Block a user