diff --git a/API_REFERENCE.md b/API_REFERENCE.md index a0111f7..b810d8e 100644 --- a/API_REFERENCE.md +++ b/API_REFERENCE.md @@ -292,3 +292,40 @@ curl -X POST http://localhost:9000/mcp-bybit/tools/get_ticker \ Per lo schema completo dei body di richiesta e risposta: . + +--- + +## 15. Discovery strumenti — schemi `get_instruments` / `get_markets` / `get_historical` + +Schemi dei body verificati sull'OpenAPI live (usati da `src/data/instruments.py`). + +### Lista strumenti +| Exchange | Tool | Body | Risposta (campi utili) | +|---|---|---|---| +| Deribit | `get_instruments` | `{currency:"any", kind:"future", offset:int, limit:100}` (paginato, `has_more`) | `instruments[].name` (es. `BTC-PERPETUAL`, `SOL_USDC-PERPETUAL`), `expiry`, `tick_size` | +| Bybit | `get_instruments` | `{category:"linear", symbol?}` | `instruments[]`: `symbol`, `status`, `base_coin`, `quote_coin` | +| Hyperliquid | `get_markets` | `{}` | lista `{asset, mark_price, funding_rate, open_interest, volume_24h, max_leverage}` | + +### Storico OHLCV (`get_historical`, chiave `candles` uniforme `{timestamp(ms),open,high,low,close,volume}`) +| Exchange | Body | +|---|---| +| Deribit | `{instrument, start_date:"YYYY-MM-DD", end_date, resolution}` — resolution `1/5/15/60/1D` | +| Bybit | `{symbol, category:"linear", interval:"1/5/15/60/D", start:int_ms, end:int_ms, limit}` | +| Hyperliquid | `{asset|instrument, start_date, end_date, resolution:"1m/5m/15m/1h/1d", limit}` | + +### Simboli Deribit +- BTC/ETH → perpetui **inverse**: `BTC-PERPETUAL`, `ETH-PERPETUAL` +- Altcoin → perpetui **lineari USDC**: `_USDC-PERPETUAL` (es. `SOL_USDC-PERPETUAL`) +- Trappola: `LTC-PERPETUAL`/`ADA-PERPETUAL` non esistono; `SOL-PERPETUAL` esiste ma è un contratto sbagliato (prezzo ~9.6 vs SOL reale ~82). + +### Validazione (lato progetto) +`src/data/instruments.py` valida ogni strumento sui dati storici realmente +raccoglibili — esistenza, congruenza OHLC, not-flat, liquidità (volume daily) e +**congruenza prezzo cross-exchange** (scostamento dalla mediana del base-coin ≤5%). +Solo gli exchange con feed affidabile sono inclusi: **Deribit** e **Hyperliquid** +(esclusi Alpaca/stocks e **Bybit**, il cui feed testnet è farlocco). Output in +`data/instruments_registry.json`; il downloader scarica **solo** strumenti validati. + +> **Testnet.** Il token osservatore punta a testnet (`"testnet": true` nei ticker): +> i prezzi possono divergere dal mainnet. La congruenza cross-exchange via mediana +> è il filtro che scarta i feed incongrui prima di usarli per backtest/trading. diff --git a/CLAUDE.md b/CLAUDE.md index cd0f235..97a486b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,7 +17,9 @@ Progetto di ricerca: riconoscimento pattern frattali per trading algoritmico su ## Struttura ``` -src/data/ → download e caricamento dati (downloader.py) +src/data/ → download e caricamento dati + downloader.py → download/caricamento parquet (gate: solo strumenti validati) + instruments.py → discovery + validazione strumenti per exchange, registry src/fractal/ → indicatori frattali (patterns.py, indicators.py, similarity.py) src/backtest/ → engine di backtesting (engine.py) src/strategies/ → classe base Strategy ABC + indicatori condivisi @@ -49,13 +51,15 @@ strategies.yml → config multi-strategy paper trader docs/diary/ → diario di ricerca giornaliero docs/specs/ → specifiche di design data/raw/ → file .parquet OHLCV (gitignored) +data/instruments_registry.json → allowlist strumenti validati (gate del downloader) ``` ## Comandi ```bash uv sync # installa dipendenze -uv run python -m src.data.downloader # scarica dati storici +uv run python -m src.data.downloader # scarica dati storici (solo strumenti validati) +uv run python -m src.data.instruments # (ri)costruisci il registry strumenti validati uv run python scripts/strategies/MR01_bollinger_fade.py # backtest una strategia (es. fade) uv run python scripts/strategies/PR01_pairs_reversion.py # backtest pairs market-neutral uv run python scripts/analysis/strategy_research.py # ricerca strategie fee-aware OOS @@ -83,6 +87,27 @@ df = load_data("ETH", "15m") # carica un asset/timeframe Fonte primaria: Cerbero MCP (endpoint `/mcp-deribit/tools/get_historical`). Token observer: nel file `secrets/observer.token` del progetto CerberoSuite. +### Strumenti & validazione (gate raccolta dati) + +`src/data/instruments.py` scopre e **valida** gli strumenti per ogni exchange +implementato — **Deribit** e **Hyperliquid** (esclusi Alpaca/stocks e **Bybit**, +il cui feed testnet è farlocco). Per ogni perpetuo enumera via `get_instruments` +/`get_markets` e verifica sui **dati storici realmente raccoglibili**: +esistenza, congruenza OHLC, not-flat (scarta contratti morti), liquidità (volume +daily) e **congruenza prezzo cross-exchange** (scostamento dalla mediana del +base-coin ≤ 5% → scarta outlier come `SOL-PERPETUAL`=9.6 vs SOL reale ~82). + +Output: `data/instruments_registry.json` (strumenti validi, timeframe, start-date). +**Gate:** `_download_cerbero_range` rifiuta gli strumenti non validati (override +`allow_unvalidated=True` solo per casi eccezionali). Rigenera con +`python -m src.data.instruments`. + +> **NB testnet.** Il token Cerbero punta a testnet; la congruenza cross-exchange +> è il filtro che distingue i feed realistici (Deribit, Hyperliquid) da quelli +> farlocchi (Bybit). Simboli Deribit: BTC/ETH = `-PERPETUAL` (inverse); +> alt = `_USDC-PERPETUAL` (lineari USDC). Registry attuale: Deribit 18/106, +> Hyperliquid 66/74 validi (major liquidi: BTC dal 2018, alt dal 2022). + ## Strategie attive > **LEZIONE CRITICA (2026-05-28).** L'intera famiglia squeeze-breakout (SQ01-04, diff --git a/README.md b/README.md index 3b84382..566cc7a 100644 --- a/README.md +++ b/README.md @@ -327,6 +327,28 @@ Formato: Apache Parquet (in `data/raw/`, gitignored). > `LTC-PERPETUAL`/`ADA-PERPETUAL` non esistono e `SOL-PERPETUAL` restituisce dati > errati — per gli altcoin usare sempre la forma `_USDC-PERPETUAL`. +### Discovery & validazione strumenti + +`src/data/instruments.py` scopre e **valida** gli strumenti disponibili sugli +exchange implementati — **Deribit** e **Hyperliquid** (esclusi Alpaca/stocks e +**Bybit**, feed testnet inaffidabile). Ogni perpetuo viene testato sui dati +storici realmente raccoglibili: esistenza, congruenza OHLC, contratto non-morto, +liquidità e **congruenza prezzo cross-exchange** (mediana per base-coin, tolleranza +5%) — così feed farlocchi e contratti sbagliati (es. `SOL-PERPETUAL`=9.6) vengono +scartati. Il risultato è `data/instruments_registry.json` (strumenti validi + +timeframe + data d'inizio). + +**Solo gli strumenti validati possono essere scaricati**: il downloader ha un gate +(`_download_cerbero_range`) che rifiuta quelli non nel registry. Rigenera con: + +```bash +uv run python -m src.data.instruments +``` + +Simboli Deribit: BTC/ETH = `-PERPETUAL` (inverse); altcoin = +`_USDC-PERPETUAL` (lineari USDC). Registry attuale (testnet): Deribit 18/106 +validi (major liquidi, BTC dal 2018), Hyperliquid 66/74. + ## Riferimenti - Serleto, L. & Malanga, C. — *Pythagoras Trading Prediction* (2024) diff --git a/data/instruments_registry.json b/data/instruments_registry.json new file mode 100644 index 0000000..253620f --- /dev/null +++ b/data/instruments_registry.json @@ -0,0 +1,2167 @@ +{ + "generated_at": "2026-05-29T07:48:46.431920+00:00", + "congruence_tol": 0.05, + "testnet": true, + "exchanges": { + "deribit": { + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ], + "instruments": { + "1000BONK_USDC-PERPETUAL": { + "base": "1000BONK", + "valid": false, + "reasons": [ + "flat_dead" + ], + "last_price": 0.00635, + "start_date": "2025-12-23", + "timeframes": [] + }, + "1000FLOKI_USDC-PERPETUAL": { + "base": "1000FLOKI", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.0471, + "start_date": "2026-03-16", + "timeframes": [] + }, + "1000MOG_USDC-PERPETUAL": { + "base": "1000MOG", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "1000PEPE_USDC-PERPETUAL": { + "base": "1000PEPE", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.005095, + "start_date": "2026-04-02", + "timeframes": [] + }, + "1000SHIB_USDC-PERPETUAL": { + "base": "1000SHIB", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.00861, + "start_date": "2026-01-13", + "timeframes": [] + }, + "1000TOSHI_USDC-PERPETUAL": { + "base": "1000TOSHI", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "AAVE_USDC-PERPETUAL": { + "base": "AAVE", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 86.03, + "start_date": "2026-04-09", + "timeframes": [] + }, + "ADA_USDC-PERPETUAL": { + "base": "ADA", + "valid": true, + "reasons": [], + "last_price": 0.2344, + "start_date": "2022-02-19", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AERO_USDC-PERPETUAL": { + "base": "AERO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "AIXBT_USDC-PERPETUAL": { + "base": "AIXBT", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ALGO_USDC-PERPETUAL": { + "base": "ALGO", + "valid": false, + "reasons": [ + "flat_dead" + ], + "last_price": 0.11584, + "start_date": "2022-05-09", + "timeframes": [] + }, + "APT_USDC-PERPETUAL": { + "base": "APT", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ARB_USDC-PERPETUAL": { + "base": "ARB", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.109, + "start_date": "2026-04-09", + "timeframes": [] + }, + "ATH_USDC-PERPETUAL": { + "base": "ATH", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.009058, + "start_date": "2026-01-28", + "timeframes": [] + }, + "ATOM_USDC-PERPETUAL": { + "base": "ATOM", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "AVAX_USDC-PERPETUAL": { + "base": "AVAX", + "valid": true, + "reasons": [], + "last_price": 8.894, + "start_date": "2022-04-26", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AXS_USDC-PERPETUAL": { + "base": "AXS", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "BCH_USDC-PERPETUAL": { + "base": "BCH", + "valid": true, + "reasons": [], + "last_price": 293.767, + "start_date": "2022-08-23", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BEAM_USDC-PERPETUAL": { + "base": "BEAM", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "BERA_USDC-PERPETUAL": { + "base": "BERA", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "BIO_USDC-PERPETUAL": { + "base": "BIO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "BNB_USDC-PERPETUAL": { + "base": "BNB", + "valid": true, + "reasons": [], + "last_price": 641.95, + "start_date": "2022-04-27", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BOME_USDC-PERPETUAL": { + "base": "BOME", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.000404, + "start_date": "2026-03-22", + "timeframes": [] + }, + "BRETT_USDC-PERPETUAL": { + "base": "BRETT", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "BTC-PERPETUAL": { + "base": "BTC", + "valid": true, + "reasons": [], + "last_price": 73346.5, + "start_date": "2018-08-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BTC_USDC-PERPETUAL": { + "base": "BTC", + "valid": true, + "reasons": [], + "last_price": 73313.5, + "start_date": "2022-02-18", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "CAKE_USDC-PERPETUAL": { + "base": "CAKE", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume", + "incongruent(px=2.118,med=1.736)" + ], + "last_price": 2.1178, + "start_date": "2026-04-22", + "timeframes": [] + }, + "CFX_USDC-PERPETUAL": { + "base": "CFX", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "CRV_USDC-PERPETUAL": { + "base": "CRV", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "DOGE_USDC-PERPETUAL": { + "base": "DOGE", + "valid": true, + "reasons": [], + "last_price": 0.09959, + "start_date": "2022-04-28", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "DOT_USDC-PERPETUAL": { + "base": "DOT", + "valid": true, + "reasons": [], + "last_price": 1.2041, + "start_date": "2022-05-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "DYDX_USDC-PERPETUAL": { + "base": "DYDX", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "EIGEN_USDC-PERPETUAL": { + "base": "EIGEN", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ENA_USDC-PERPETUAL": { + "base": "ENA", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ENS_USDC-PERPETUAL": { + "base": "ENS", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ETC_USDC-PERPETUAL": { + "base": "ETC", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ETH-PERPETUAL": { + "base": "ETH", + "valid": true, + "reasons": [], + "last_price": 2003.25, + "start_date": "2019-01-16", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ETH_USDC-PERPETUAL": { + "base": "ETH", + "valid": true, + "reasons": [], + "last_price": 2002.8, + "start_date": "2022-02-19", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ETHFI_USDC-PERPETUAL": { + "base": "ETHFI", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "FET_USDC-PERPETUAL": { + "base": "FET", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.2597, + "start_date": "2025-12-10", + "timeframes": [] + }, + "FIL_USDC-PERPETUAL": { + "base": "FIL", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "HBAR_USDC-PERPETUAL": { + "base": "HBAR", + "valid": false, + "reasons": [ + "flat_dead", + "incongruent(px=0.138,med=0.1146)" + ], + "last_price": 0.138, + "start_date": "2025-12-08", + "timeframes": [] + }, + "HYPE_USDC-PERPETUAL": { + "base": "HYPE", + "valid": false, + "reasons": [ + "flat_dead", + "incongruent(px=45,med=36.7)" + ], + "last_price": 45.0, + "start_date": "2025-12-01", + "timeframes": [] + }, + "ICP_USDC-PERPETUAL": { + "base": "ICP", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 2.592, + "start_date": "2026-04-09", + "timeframes": [] + }, + "IMX_USDC-PERPETUAL": { + "base": "IMX", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "INJ_USDC-PERPETUAL": { + "base": "INJ", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "IP_USDC-PERPETUAL": { + "base": "IP", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "JTO_USDC-PERPETUAL": { + "base": "JTO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "JUP_USDC-PERPETUAL": { + "base": "JUP", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "KAITO_USDC-PERPETUAL": { + "base": "KAITO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "KAS_USDC-PERPETUAL": { + "base": "KAS", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.03075, + "start_date": "2025-12-16", + "timeframes": [] + }, + "LDO_USDC-PERPETUAL": { + "base": "LDO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "LINK_USDC-PERPETUAL": { + "base": "LINK", + "valid": true, + "reasons": [], + "last_price": 8.876, + "start_date": "2022-07-29", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "LTC_USDC-PERPETUAL": { + "base": "LTC", + "valid": true, + "reasons": [], + "last_price": 51.61, + "start_date": "2022-07-22", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "MEME_USDC-PERPETUAL": { + "base": "MEME", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "MOODENG_USDC-PERPETUAL": { + "base": "MOODENG", + "valid": false, + "reasons": [ + "flat_dead", + "incongruent(px=0.091,med=0.06898)" + ], + "last_price": 0.091, + "start_date": "2025-12-05", + "timeframes": [] + }, + "MORPHO_USDC-PERPETUAL": { + "base": "MORPHO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "NEAR_USDC-PERPETUAL": { + "base": "NEAR", + "valid": false, + "reasons": [ + "flat_dead" + ], + "last_price": 2.4914, + "start_date": "2025-01-31", + "timeframes": [] + }, + "NEIRO_USDC-PERPETUAL": { + "base": "NEIRO", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.000149, + "start_date": "2025-12-22", + "timeframes": [] + }, + "ONDO_USDC-PERPETUAL": { + "base": "ONDO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "OP_USDC-PERPETUAL": { + "base": "OP", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.124, + "start_date": "2026-04-10", + "timeframes": [] + }, + "ORDI_USDC-PERPETUAL": { + "base": "ORDI", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "PAXG_USDC-PERPETUAL": { + "base": "PAXG", + "valid": true, + "reasons": [], + "last_price": 4517.8, + "start_date": "2024-10-08", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "PENDLE_USDC-PERPETUAL": { + "base": "PENDLE", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume", + "incongruent(px=1.966,med=1.707)" + ], + "last_price": 1.9655, + "start_date": "2026-05-15", + "timeframes": [] + }, + "PENGU_USDC-PERPETUAL": { + "base": "PENGU", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "PEOPLE_USDC-PERPETUAL": { + "base": "PEOPLE", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "PNUT_USDC-PERPETUAL": { + "base": "PNUT", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "POPCAT_USDC-PERPETUAL": { + "base": "POPCAT", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.0574, + "start_date": "2026-05-15", + "timeframes": [] + }, + "PUMP_USDC-PERPETUAL": { + "base": "PUMP", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "PYTH_USDC-PERPETUAL": { + "base": "PYTH", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "RAY_USDC-PERPETUAL": { + "base": "RAY", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "RENDER_USDC-PERPETUAL": { + "base": "RENDER", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "RUNE_USDC-PERPETUAL": { + "base": "RUNE", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "S_USDC-PERPETUAL": { + "base": "S", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "SAND_USDC-PERPETUAL": { + "base": "SAND", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "SEI_USDC-PERPETUAL": { + "base": "SEI", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "SHIB_USDC-PERPETUAL": { + "base": "SHIB", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.01, + "start_date": "2022-05-25", + "timeframes": [] + }, + "SOL-PERPETUAL": { + "base": "SOL", + "valid": false, + "reasons": [ + "no_volume", + "incongruent(px=9.6,med=81.94)" + ], + "last_price": 9.6, + "start_date": "2021-12-10", + "timeframes": [] + }, + "SOL_USDC-PERPETUAL": { + "base": "SOL", + "valid": true, + "reasons": [], + "last_price": 82.097, + "start_date": "2022-02-19", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "SPK_USDC-PERPETUAL": { + "base": "SPK", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "SPX_USDC-PERPETUAL": { + "base": "SPX", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "STETH_USDC-PERPETUAL": { + "base": "STETH", + "valid": false, + "reasons": [ + "flat_dead" + ], + "last_price": 2357.7, + "start_date": "2024-12-31", + "timeframes": [] + }, + "STRK_USDC-PERPETUAL": { + "base": "STRK", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "STX_USDC-PERPETUAL": { + "base": "STX", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "SUI_USDC-PERPETUAL": { + "base": "SUI", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "TAO_USDC-PERPETUAL": { + "base": "TAO", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume", + "incongruent(px=320.9,med=302.4)" + ], + "last_price": 320.8931, + "start_date": "2026-03-24", + "timeframes": [] + }, + "THETA_USDC-PERPETUAL": { + "base": "THETA", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "TIA_USDC-PERPETUAL": { + "base": "TIA", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "TON_USDC-PERPETUAL": { + "base": "TON", + "valid": false, + "reasons": [ + "flat_dead", + "incongruent(px=3,med=2.395)" + ], + "last_price": 3.0, + "start_date": "2025-08-28", + "timeframes": [] + }, + "TRB_USDC-PERPETUAL": { + "base": "TRB", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "TRUMP_USDC-PERPETUAL": { + "base": "TRUMP", + "valid": true, + "reasons": [], + "last_price": 1.8474, + "start_date": "2025-01-23", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TRX_USDC-PERPETUAL": { + "base": "TRX", + "valid": true, + "reasons": [], + "last_price": 0.34813, + "start_date": "2022-05-28", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TURBO_USDC-PERPETUAL": { + "base": "TURBO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "UNI_USDC-PERPETUAL": { + "base": "UNI", + "valid": true, + "reasons": [], + "last_price": 3.0063, + "start_date": "2022-10-03", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "VET_USDC-PERPETUAL": { + "base": "VET", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "VIRTUAL_USDC-PERPETUAL": { + "base": "VIRTUAL", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "W_USDC-PERPETUAL": { + "base": "W", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "WIF_USDC-PERPETUAL": { + "base": "WIF", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "WLD_USDC-PERPETUAL": { + "base": "WLD", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "XLM_USDC-PERPETUAL": { + "base": "XLM", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "XRP_USDC-PERPETUAL": { + "base": "XRP", + "valid": true, + "reasons": [], + "last_price": 1.3156, + "start_date": "2022-02-19", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "XTZ_USDC-PERPETUAL": { + "base": "XTZ", + "valid": false, + "reasons": [ + "flat_dead", + "no_volume" + ], + "last_price": 0.3463, + "start_date": "2026-03-30", + "timeframes": [] + }, + "ZEC_USDC-PERPETUAL": { + "base": "ZEC", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ZK_USDC-PERPETUAL": { + "base": "ZK", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ZORA_USDC-PERPETUAL": { + "base": "ZORA", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "ZRO_USDC-PERPETUAL": { + "base": "ZRO", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + } + } + }, + "hyperliquid": { + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ], + "instruments": { + "SOL": { + "base": "SOL", + "valid": true, + "reasons": [], + "last_price": 81.937, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "APT": { + "base": "APT", + "valid": true, + "reasons": [], + "last_price": 0.926, + "start_date": "2025-04-09", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ATOM": { + "base": "ATOM", + "valid": true, + "reasons": [], + "last_price": 2.0357, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BTC": { + "base": "BTC", + "valid": true, + "reasons": [], + "last_price": 73841.0, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ETH": { + "base": "ETH", + "valid": true, + "reasons": [], + "last_price": 1995.9, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BNB": { + "base": "BNB", + "valid": true, + "reasons": [], + "last_price": 636.8, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AVAX": { + "base": "AVAX", + "valid": true, + "reasons": [], + "last_price": 8.8806, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "DYDX": { + "base": "DYDX", + "valid": true, + "reasons": [], + "last_price": 0.17186, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "OP": { + "base": "OP", + "valid": true, + "reasons": [], + "last_price": 0.11875, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ARB": { + "base": "ARB", + "valid": true, + "reasons": [], + "last_price": 0.10479, + "start_date": "2025-04-09", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "WLD": { + "base": "WLD", + "valid": true, + "reasons": [], + "last_price": 0.30326, + "start_date": "2025-04-09", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AAVE": { + "base": "AAVE", + "valid": true, + "reasons": [], + "last_price": 81.88, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "LDO": { + "base": "LDO", + "valid": true, + "reasons": [], + "last_price": 0.32392, + "start_date": "2022-09-22", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "SUI": { + "base": "SUI", + "valid": true, + "reasons": [], + "last_price": 0.91577, + "start_date": "2023-05-03", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "INJ": { + "base": "INJ", + "valid": true, + "reasons": [], + "last_price": 6.2043, + "start_date": "2022-08-17", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "STX": { + "base": "STX", + "valid": true, + "reasons": [], + "last_price": 0.23235, + "start_date": "2025-04-12", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ZRO": { + "base": "ZRO", + "valid": true, + "reasons": [], + "last_price": 1.1361, + "start_date": "2024-06-20", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TRB": { + "base": "TRB", + "valid": true, + "reasons": [], + "last_price": 16.36, + "start_date": "2025-04-20", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "KAS": { + "base": "KAS", + "valid": true, + "reasons": [], + "last_price": 0.031029, + "start_date": "2025-04-11", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TIA": { + "base": "TIA", + "valid": true, + "reasons": [], + "last_price": 0.41279, + "start_date": "2023-10-31", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TON": { + "base": "TON", + "valid": false, + "reasons": [ + "incongruent(px=1.791,med=2.395)" + ], + "last_price": 1.7907, + "start_date": "2025-04-09", + "timeframes": [] + }, + "ADA": { + "base": "ADA", + "valid": true, + "reasons": [], + "last_price": 0.23586, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "PENDLE": { + "base": "PENDLE", + "valid": false, + "reasons": [ + "incongruent(px=1.448,med=1.707)" + ], + "last_price": 1.4478, + "start_date": "2025-04-12", + "timeframes": [] + }, + "FET": { + "base": "FET", + "valid": true, + "reasons": [], + "last_price": 0.24381, + "start_date": "2023-01-17", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "NEAR": { + "base": "NEAR", + "valid": true, + "reasons": [], + "last_price": 2.3636, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "MEME": { + "base": "MEME", + "valid": true, + "reasons": [], + "last_price": 0.000481, + "start_date": "2025-04-12", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ORDI": { + "base": "ORDI", + "valid": true, + "reasons": [], + "last_price": 3.4575, + "start_date": "2023-11-07", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "FIL": { + "base": "FIL", + "valid": true, + "reasons": [], + "last_price": 0.96404, + "start_date": "2025-04-10", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "PYTH": { + "base": "PYTH", + "valid": true, + "reasons": [], + "last_price": 0.041423, + "start_date": "2023-11-22", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "RUNE": { + "base": "RUNE", + "valid": true, + "reasons": [], + "last_price": 0.42624, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "IMX": { + "base": "IMX", + "valid": true, + "reasons": [], + "last_price": 0.15872, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "JUP": { + "base": "JUP", + "valid": true, + "reasons": [], + "last_price": 0.17676, + "start_date": "2025-04-11", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "JTO": { + "base": "JTO", + "valid": true, + "reasons": [], + "last_price": 0.50659, + "start_date": "2023-12-08", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "WIF": { + "base": "WIF", + "valid": true, + "reasons": [], + "last_price": 0.18725, + "start_date": "2024-01-18", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "CAKE": { + "base": "CAKE", + "valid": false, + "reasons": [ + "incongruent(px=1.354,med=1.736)" + ], + "last_price": 1.3544, + "start_date": "2023-11-02", + "timeframes": [] + }, + "PEOPLE": { + "base": "PEOPLE", + "valid": true, + "reasons": [], + "last_price": 0.006043, + "start_date": "2025-04-20", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ENS": { + "base": "ENS", + "valid": true, + "reasons": [], + "last_price": 5.8188, + "start_date": "2025-04-14", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ETC": { + "base": "ETC", + "valid": true, + "reasons": [], + "last_price": 8.2323, + "start_date": "2022-08-18", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ONDO": { + "base": "ONDO", + "valid": true, + "reasons": [], + "last_price": 0.36859, + "start_date": "2024-01-20", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "W": { + "base": "W", + "valid": true, + "reasons": [], + "last_price": 0.011639, + "start_date": "2024-04-03", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TAO": { + "base": "TAO", + "valid": false, + "reasons": [ + "incongruent(px=284,med=302.4)" + ], + "last_price": 283.97, + "start_date": "2024-04-11", + "timeframes": [] + }, + "HBAR": { + "base": "HBAR", + "valid": false, + "reasons": [ + "incongruent(px=0.09125,med=0.1146)" + ], + "last_price": 0.091245, + "start_date": "2022-07-13", + "timeframes": [] + }, + "POPCAT": { + "base": "POPCAT", + "valid": true, + "reasons": [], + "last_price": 0.052073, + "start_date": "2025-04-10", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "EIGEN": { + "base": "EIGEN", + "valid": true, + "reasons": [], + "last_price": 0.2184, + "start_date": "2025-04-15", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TURBO": { + "base": "TURBO", + "valid": true, + "reasons": [], + "last_price": 0.001084, + "start_date": "2025-04-26", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BRETT": { + "base": "BRETT", + "valid": true, + "reasons": [], + "last_price": 0.006357, + "start_date": "2025-04-12", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ZK": { + "base": "ZK", + "valid": true, + "reasons": [], + "last_price": 0.013627, + "start_date": "2025-04-15", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "RENDER": { + "base": "RENDER", + "valid": true, + "reasons": [], + "last_price": 2.0722, + "start_date": "2025-04-11", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "MOODENG": { + "base": "MOODENG", + "valid": false, + "reasons": [ + "incongruent(px=0.04695,med=0.06898)" + ], + "last_price": 0.046951, + "start_date": "2024-10-25", + "timeframes": [] + }, + "PNUT": { + "base": "PNUT", + "valid": true, + "reasons": [], + "last_price": 0.05302, + "start_date": "2025-04-22", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "XLM": { + "base": "XLM", + "valid": true, + "reasons": [], + "last_price": 0.20964, + "start_date": "2022-08-18", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "SAND": { + "base": "SAND", + "valid": true, + "reasons": [], + "last_price": 0.06742, + "start_date": "2022-07-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ALGO": { + "base": "ALGO", + "valid": true, + "reasons": [], + "last_price": 0.11699, + "start_date": "2025-04-10", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ICP": { + "base": "ICP", + "valid": true, + "reasons": [], + "last_price": 2.7324, + "start_date": "2025-04-20", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "VET": { + "base": "VET", + "valid": false, + "reasons": [ + "no_history", + "no_volume" + ], + "last_price": null, + "start_date": null, + "timeframes": [] + }, + "HYPE": { + "base": "HYPE", + "valid": false, + "reasons": [ + "incongruent(px=28.39,med=36.7)" + ], + "last_price": 28.395, + "start_date": "2025-04-07", + "timeframes": [] + }, + "VIRTUAL": { + "base": "VIRTUAL", + "valid": true, + "reasons": [], + "last_price": 0.72259, + "start_date": "2025-04-12", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "PENGU": { + "base": "PENGU", + "valid": true, + "reasons": [], + "last_price": 0.007615, + "start_date": "2025-04-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AIXBT": { + "base": "AIXBT", + "valid": true, + "reasons": [], + "last_price": 0.027521, + "start_date": "2025-04-26", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BIO": { + "base": "BIO", + "valid": true, + "reasons": [], + "last_price": 0.030108, + "start_date": "2025-04-25", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "SPX": { + "base": "SPX", + "valid": true, + "reasons": [], + "last_price": 0.32575, + "start_date": "2024-12-10", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "S": { + "base": "S", + "valid": true, + "reasons": [], + "last_price": 0.040244, + "start_date": "2025-04-22", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "MORPHO": { + "base": "MORPHO", + "valid": true, + "reasons": [], + "last_price": 2.0741, + "start_date": "2025-04-27", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "TRUMP": { + "base": "TRUMP", + "valid": true, + "reasons": [], + "last_price": 1.9095, + "start_date": "2025-04-10", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "BERA": { + "base": "BERA", + "valid": true, + "reasons": [], + "last_price": 0.34205, + "start_date": "2025-04-22", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "IP": { + "base": "IP", + "valid": true, + "reasons": [], + "last_price": 0.43244, + "start_date": "2025-02-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "KAITO": { + "base": "KAITO", + "valid": true, + "reasons": [], + "last_price": 0.45193, + "start_date": "2025-04-26", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "PAXG": { + "base": "PAXG", + "valid": true, + "reasons": [], + "last_price": 4506.9, + "start_date": "2025-04-08", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ZORA": { + "base": "ZORA", + "valid": true, + "reasons": [], + "last_price": 0.009803, + "start_date": "2025-04-26", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "DOGE": { + "base": "DOGE", + "valid": true, + "reasons": [], + "last_price": 0.098882, + "start_date": "2025-05-13", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "PUMP": { + "base": "PUMP", + "valid": true, + "reasons": [], + "last_price": 0.001694, + "start_date": "2025-07-10", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "ZEC": { + "base": "ZEC", + "valid": true, + "reasons": [], + "last_price": 138.65, + "start_date": "2025-10-03", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AERO": { + "base": "AERO", + "valid": true, + "reasons": [], + "last_price": 0.41157, + "start_date": "2025-11-12", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + }, + "AXS": { + "base": "AXS", + "valid": true, + "reasons": [], + "last_price": 1.169, + "start_date": "2026-01-19", + "timeframes": [ + "1m", + "5m", + "15m", + "1h" + ] + } + } + } + } +} \ No newline at end of file diff --git a/src/data/downloader.py b/src/data/downloader.py index 0014962..ab35ffb 100644 --- a/src/data/downloader.py +++ b/src/data/downloader.py @@ -69,8 +69,19 @@ def _fetch_binance(symbol: str, tf: str, since_ms: int, limit: int = 1000) -> li def _download_cerbero_range( - instrument: str, resolution: str, tf: str, start_date: str, end_date: str + instrument: str, resolution: str, tf: str, start_date: str, end_date: str, + allow_unvalidated: bool = False, ) -> pd.DataFrame: + # Gate: si raccolgono dati SOLO per strumenti validati nel registry. + # Esegui `python -m src.data.instruments` per (ri)costruirlo. + if not allow_unvalidated: + from src.data.instruments import is_validated + if not is_validated(instrument, tf, "deribit"): + raise ValueError( + f"Strumento non validato: {instrument} @ {tf}. " + f"Costruisci il registry (python -m src.data.instruments) o passa " + f"allow_unvalidated=True per forzare." + ) all_candles: list[dict] = [] max_days = MAX_DAYS_PER_REQUEST[tf] current = datetime.fromisoformat(start_date) diff --git a/src/data/instruments.py b/src/data/instruments.py new file mode 100644 index 0000000..24190ca --- /dev/null +++ b/src/data/instruments.py @@ -0,0 +1,278 @@ +"""Discovery + validazione strumenti per gli exchange implementati (via Cerbero MCP). + +Per ogni exchange (Deribit, Hyperliquid — esclusi Alpaca/stocks e Bybit, il cui +feed testnet e' farlocco) enumera i perpetui, ne verifica i dati e produce un +registry di strumenti VALIDATI. +Solo gli strumenti nel registry possono essere usati per la raccolta dati +(vedi gate in src/data/downloader.py). + +Controlli di validazione (uno strumento e' valido solo se TUTTI passano): + - exists : la storia daily ritorna candele + - ohlc_sane : high>=low, high>=max(o,c), low<=min(o,c), prezzi>0 + - not_flat : non e' un contratto morto (quasi tutte le barre O==H==L==C) + - liquid : volume_24h>0 dal ticker + - congruent : il prezzo concorda (entro tolleranza) con la MEDIANA dello + stesso base-coin su tutti gli exchange. Scarta i feed testnet + farlocchi (es. Bybit BTC=300k) e i contratti sbagliati + (es. Deribit SOL-PERPETUAL=9.6 vs SOL reale ~82). + +NB: il token Cerbero punta a TESTNET; la congruenza cross-exchange e' il filtro +che distingue i feed realistici (Deribit, Hyperliquid) da quelli farlocchi. +""" +from __future__ import annotations + +import json +import statistics +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path + +import pandas as pd + +from src.live.cerbero_client import CerberoClient + +REGISTRY_PATH = Path(__file__).resolve().parents[2] / "data" / "instruments_registry.json" + +# I nostri timestep -> codice risoluzione per ciascun exchange +TF_CODES = { + "deribit": {"1m": "1", "5m": "5", "15m": "15", "1h": "60", "1d": "1D"}, + "hyperliquid": {"1m": "1m", "5m": "5m", "15m": "15m", "1h": "1h", "1d": "1d"}, +} +CONGRUENCE_TOL = 0.05 # 5% di scostamento dalla mediana del base-coin + + +def _today() -> str: + return datetime.now(timezone.utc).strftime("%Y-%m-%d") + + +@dataclass +class Quote: + base: str + symbol: str + last: float | None = None + volume_24h: float | None = None + open_interest: float | None = None + + +# --------------------------- adapters --------------------------- +class ExchangeAdapter: + name = "base" + + def __init__(self, client: CerberoClient): + self.c = client + + def _post(self, tool: str, payload: dict) -> dict: + return self.c._post(f"/mcp-{self.name}/tools/{tool}", payload) + + def list_symbols(self) -> list[Quote]: + """Lista perpetui (economica). I prezzi possono essere None (vedi ticker).""" + raise NotImplementedError + + def ticker(self, q: Quote) -> None: + """Riempie last/volume/OI sul Quote (per-simbolo). No-op se gia' pieni.""" + + def candles(self, symbol: str, tf: str, start: str, end: str) -> pd.DataFrame: + raise NotImplementedError + + +class DeribitAdapter(ExchangeAdapter): + name = "deribit" + + def list_symbols(self) -> list[Quote]: + perps, offset = [], 0 + while True: + r = self._post("get_instruments", {"currency": "any", "kind": "future", + "offset": offset, "limit": 100}) + insts = r.get("instruments", []) + perps += [i["name"] for i in insts if i.get("name", "").endswith("-PERPETUAL")] + if not r.get("has_more") or not insts: + break + offset += len(insts) + if offset > 2000: + break + out = [] + for name in perps: + base = name.split("-PERPETUAL")[0].replace("_USDC", "").replace("_USD", "") + out.append(Quote(base, name)) + return out + + def ticker(self, q: Quote) -> None: + t = self._post("get_ticker", {"instrument": q.symbol}) + q.last, q.volume_24h, q.open_interest = t.get("last_price"), t.get("volume_24h"), t.get("open_interest") + + def candles(self, symbol, tf, start, end) -> pd.DataFrame: + r = self._post("get_historical", {"instrument": symbol, "start_date": start, + "end_date": end, "resolution": TF_CODES["deribit"][tf]}) + return _to_df(r.get("candles", [])) + + +class HyperliquidAdapter(ExchangeAdapter): + name = "hyperliquid" + + def list_symbols(self) -> list[Quote]: + r = self._post("get_markets", {}) + markets = r if isinstance(r, list) else r.get("markets", []) + return [Quote(m["asset"], m["asset"], m.get("mark_price"), + m.get("volume_24h"), m.get("open_interest")) for m in markets] + + # prezzi gia' presenti da get_markets -> ticker no-op + + def candles(self, symbol, tf, start, end) -> pd.DataFrame: + r = self._post("get_historical", {"asset": symbol, "start_date": start, "end_date": end, + "resolution": TF_CODES["hyperliquid"][tf], "limit": 5000}) + return _to_df(r.get("candles", [])) + + +ADAPTERS = {"deribit": DeribitAdapter, "hyperliquid": HyperliquidAdapter} + + +def _to_df(candles: list[dict]) -> pd.DataFrame: + if not candles: + return pd.DataFrame() + df = pd.DataFrame(candles) + df["timestamp"] = df["timestamp"].astype("int64") + return df.sort_values("timestamp").reset_index(drop=True) + + +# --------------------------- validazione --------------------------- +def _ohlc_sane(df: pd.DataFrame) -> bool: + if df.empty: + return False + o, h, l, c = df["open"], df["high"], df["low"], df["close"] + ok = (h >= l) & (h >= o) & (h >= c) & (l <= o) & (l <= c) & (c > 0) & (l > 0) + return bool(ok.mean() > 0.99) + + +def _not_flat(df: pd.DataFrame) -> bool: + if df.empty: + return False + flat = (df["open"] == df["high"]) & (df["high"] == df["low"]) & (df["low"] == df["close"]) + return bool(flat.mean() < 0.90) + + +def build_registry(exchanges: list[str] | None = None, + tf_check: tuple[str, ...] = ("1m", "5m", "15m", "1h"), + start_scan_from: str = "2017-01-01", + save: bool = True) -> dict: + exchanges = exchanges or ["deribit", "hyperliquid"] # NO alpaca, NO bybit (testnet farlocco) + client = CerberoClient() + adapters = {ex: ADAPTERS[ex](client) for ex in exchanges} + + # 1) lista economica per ogni exchange + listed: dict[str, list[Quote]] = {} + for ex, ad in adapters.items(): + try: + listed[ex] = ad.list_symbols() + print(f" [{ex}] {len(listed[ex])} strumenti elencati") + except Exception as e: + print(f" [{ex}] discovery FALLITA: {type(e).__name__}: {e}") + listed[ex] = [] + + # 2) universo = base-coin presenti su Deribit (il nostro venue). Bybit/HL + # vengono validati solo sull'overlap (cross-check), non su 500+ simboli. + deribit_bases = {q.base for q in listed.get("deribit", [])} + selected: dict[str, list[Quote]] = {} + for ex, qs in listed.items(): + selected[ex] = qs if ex == "deribit" else [q for q in qs if q.base in deribit_bases] + + # 3) timeframe disponibili per exchange (testati su BTC recente) + ref = {"deribit": "BTC-PERPETUAL", "hyperliquid": "BTC"} + tf_by_ex: dict[str, list[str]] = {} + for ex, ad in adapters.items(): + oks = [] + for tf in tf_check: + try: + if not ad.candles(ref[ex], tf, _today(), _today()).empty: + oks.append(tf) + except Exception: + pass + tf_by_ex[ex] = oks + print(f" [{ex}] timeframe ok: {oks}") + + # 4) UNA fetch daily per strumento: e' il dato che davvero raccoglieremmo. + # Da qui ricaviamo esistenza, OHLC, not-flat, start-date, prezzo-per-congruenza + # (ultima close STORICA, non il ticker) e liquidita' (volume daily recente). + scan: dict[tuple[str, str], dict] = {} + for ex, ad in adapters.items(): + for q in selected[ex]: + rec = {"reasons": [], "last_close": None, "start_date": None, "vol": 0.0} + try: + d = ad.candles(q.symbol, "1d", start_scan_from, _today()) + if d.empty: + rec["reasons"].append("no_history") + else: + if not _ohlc_sane(d): + rec["reasons"].append("ohlc_insane") + if not _not_flat(d): + rec["reasons"].append("flat_dead") + rec["last_close"] = float(d["close"].iloc[-1]) + rec["vol"] = float(d["volume"].tail(7).mean()) + rec["start_date"] = str(pd.to_datetime(d["timestamp"].iloc[0], unit="ms", utc=True).date()) + except Exception as e: + rec["reasons"].append(f"history_err:{type(e).__name__}") + scan[(ex, q.symbol)] = rec + + # 5) mediana per base-coin dall'ULTIMA CLOSE STORICA (riferimento congruenza) + by_base: dict[str, list[float]] = {} + for (ex, sym), rec in scan.items(): + base = next(q.base for q in selected[ex] if q.symbol == sym) + if rec["last_close"] and rec["last_close"] > 0: + by_base.setdefault(base, []).append(rec["last_close"]) + median_px = {b: statistics.median(v) for b, v in by_base.items()} + + # 6) finalizza validazione + registry: dict = {"generated_at": datetime.now(timezone.utc).isoformat(), + "congruence_tol": CONGRUENCE_TOL, "testnet": True, "exchanges": {}} + for ex, ad in adapters.items(): + registry["exchanges"][ex] = {"timeframes": tf_by_ex[ex], "instruments": {}} + for q in selected[ex]: + rec = scan[(ex, q.symbol)] + reasons = list(rec["reasons"]) + px, med, n_src = rec["last_close"], median_px.get(q.base), len(by_base.get(q.base, [])) + if not (rec["vol"] and rec["vol"] > 0): + reasons.append("no_volume") + if px is None or px <= 0: + if "no_history" not in reasons: + reasons.append("no_price") + elif med and n_src >= 2 and abs(px - med) / med > CONGRUENCE_TOL: + reasons.append(f"incongruent(px={px:.4g},med={med:.4g})") + valid = len(reasons) == 0 + registry["exchanges"][ex]["instruments"][q.symbol] = { + "base": q.base, "valid": valid, "reasons": reasons, + "last_price": px, "start_date": rec["start_date"], + "timeframes": tf_by_ex[ex] if valid else [], + } + if save: + REGISTRY_PATH.write_text(json.dumps(registry, indent=2)) + print(f" registry salvato in {REGISTRY_PATH}") + return registry + + +# --------------------------- gate per il downloader --------------------------- +def load_registry() -> dict: + return json.loads(REGISTRY_PATH.read_text()) if REGISTRY_PATH.exists() else {} + + +def is_validated(symbol: str, tf: str, exchange: str = "deribit") -> bool: + """True solo se lo strumento e' nel registry come valido per quel timeframe.""" + inst = load_registry().get("exchanges", {}).get(exchange, {}).get("instruments", {}).get(symbol) + return bool(inst and inst.get("valid") and tf in inst.get("timeframes", [])) + + +if __name__ == "__main__": + reg = build_registry() + print("\n" + "=" * 96) + print(" REGISTRY STRUMENTI VALIDATI") + print("=" * 96) + for ex, exd in reg["exchanges"].items(): + insts = exd["instruments"] + valid = {s: i for s, i in insts.items() if i["valid"]} + print(f"\n {ex.upper()} | tf={exd['timeframes']} | validi {len(valid)}/{len(insts)}") + for s, i in sorted(valid.items(), key=lambda kv: kv[1]["base"]): + print(f" {s:30s} {i['base']:10s} px={i['last_price']:<12.6g} dal {i['start_date']}") + bad = {s: i for s, i in insts.items() if not i["valid"]} + if bad: + shown = list(bad.items())[:6] + print(f" -- scartati {len(bad)} (primi {len(shown)}):") + for s, i in shown: + print(f" {s:30s} {','.join(i['reasons'])[:64]}")