Adriano Dal Pastro
14522262e6
Reset del progetto su fondamenta verificate dopo la scoperta che l'intera libreria "validata OOS" era artefatto di feed contaminato (print fantasma del feed Cerbero TESTNET + storico Binance/USDT). - Storico ricostruito da Deribit MAINNET (ccxt pubblico, tokenless) e CERTIFICATO (certify_feed.py): BTC/ETH puliti su TUTTA la storia (mediana 2-6 bps vs Coinbase USD), integrita' OHLC + coerenza resample (maxΔ 0.00) + cross-venue OK. Alt esclusi (illiquidi/divergenti: LTC/DOGE 50-82% barre flat; XRP/BNB non certificabili). - Verdetto sul feed pulito: FADE / PAIRS / XS01 / TSM01 morti (ogni portafoglio Sharpe -2.3..-3.0, DD ~40%); solo SH01 e frammenti HONEST con segnale residuo, da ri-validare in isolamento. - Cleanup "restart pulito": strategie, stack live (src/live, src/portfolio, runner/executor, yml, docker), ~100 script ricerca/gate, waste/games/ portfolios, dati non certificati + cache e 60+ diari -> archiviati in Old/ (preservati, non cancellati). Diario consolidato in un unico documento. - Skeleton ricerca tenuto: Strategy ABC + indicatori + src/fractal + src/backtest/engine + load_data; tool dati certificati (rebuild_history, certify_feed, audit_feed, multi_source_check). - Universo dati ATTIVO: solo BTC/ETH (5m/15m/1h); guardrail fisico (load_data su alt -> FileNotFoundError). Esecuzione DISABILITATA, conto flat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
84 lines
3.3 KiB
Python
84 lines
3.3 KiB
Python
"""Importa lo storico per-strike delle opzioni da cerbero-bite -> data/options/.
|
|
|
|
cerbero-bite (container Docker accanto) accumula in continuo gli snapshot della
|
|
catena opzioni Deribit (BTC+ETH) nella tabella `option_chain_snapshots` del suo
|
|
SQLite (`/app/data/state.sqlite`, volume `cerbero-bite_bite-data`, root-only).
|
|
Questo e' lo storico per-strike REALE (bid/ask/mid/IV/greche/OI/volume) che il
|
|
progetto credeva non-backtestabile (muro ARGO/GEX): qui esiste ed e' gratis.
|
|
|
|
Il volume non e' leggibile direttamente (root), quindi esportiamo via `docker
|
|
exec` (il container ha pandas+pyarrow) e copiamo i parquet in data/options/.
|
|
Analogo a regime_fetcher.py -> data/regime/. Rigenera con:
|
|
|
|
uv run python scripts/analysis/options_fetcher.py
|
|
|
|
NB: snapshot da 2026-05-01 in poi (cerbero-bite e' partito allora), cadenza ~12min.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
|
OUT = PROJECT_ROOT / "data" / "options"
|
|
BITE_COMPOSE = "/opt/docker/cerbero-bite/docker-compose.yml"
|
|
BITE_SERVICE = "cerbero-bite"
|
|
|
|
_EXPORT = r'''
|
|
import sqlite3, pandas as pd
|
|
con = sqlite3.connect("file:/app/data/state.sqlite?mode=ro", uri=True)
|
|
for asset in ("ETH", "BTC"):
|
|
df = pd.read_sql_query(
|
|
"SELECT * FROM option_chain_snapshots WHERE asset=?", con, params=(asset,))
|
|
df.to_parquet(f"/tmp/{asset.lower()}_chain.parquet")
|
|
print(f"{asset} {len(df)}")
|
|
try:
|
|
dv = pd.read_sql_query("SELECT * FROM dvol_history", con)
|
|
dv.to_parquet("/tmp/dvol_history.parquet"); print(f"DVOL {len(dv)}")
|
|
except Exception as e:
|
|
print("dvol skip", e)
|
|
# market_snapshots: pannello regime allineato (spot, dvol, VRP, funding, GEX dealer,
|
|
# gamma_flip, liquidation risk, oi_delta) -- feature reali pre-calcolate.
|
|
try:
|
|
ms = pd.read_sql_query("SELECT * FROM market_snapshots", con)
|
|
ms.to_parquet("/tmp/market_snapshots.parquet"); print(f"MARKET {len(ms)}")
|
|
except Exception as e:
|
|
print("market skip", e)
|
|
'''
|
|
|
|
|
|
def _container_id() -> str:
|
|
out = subprocess.run(["docker", "compose", "-f", BITE_COMPOSE, "ps", "-q", BITE_SERVICE],
|
|
capture_output=True, text=True, check=True)
|
|
cid = out.stdout.strip().splitlines()[0]
|
|
if not cid:
|
|
raise RuntimeError("container cerbero-bite non trovato (e' su?)")
|
|
return cid
|
|
|
|
|
|
def fetch() -> None:
|
|
OUT.mkdir(parents=True, exist_ok=True)
|
|
print("export dalla SQLite di cerbero-bite (docker exec)...")
|
|
r = subprocess.run(["docker", "compose", "-f", BITE_COMPOSE, "exec", "-T", BITE_SERVICE,
|
|
"python", "-c", _EXPORT], capture_output=True, text=True)
|
|
print(r.stdout.strip())
|
|
if r.returncode != 0:
|
|
print(r.stderr[-2000:], file=sys.stderr)
|
|
raise SystemExit("export fallito")
|
|
cid = _container_id()
|
|
for fn in ("eth_chain.parquet", "btc_chain.parquet", "dvol_history.parquet",
|
|
"market_snapshots.parquet"):
|
|
dst = OUT / fn
|
|
cp = subprocess.run(["docker", "cp", f"{cid}:/tmp/{fn}", str(dst)],
|
|
capture_output=True, text=True)
|
|
if cp.returncode == 0:
|
|
print(f" -> {dst.relative_to(PROJECT_ROOT)} ({dst.stat().st_size // 1024} KB)")
|
|
else:
|
|
print(f" (skip {fn}: {cp.stderr.strip()})")
|
|
print("OK. Loader: scripts/analysis/options_chain.py")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
fetch()
|