research(blind): 52 agenti ciechi su curve anonime BTC/ETH — orchestratore valuta PnL/maxDD, niente di nuovo regge
Flotta di 52 subagenti "esperti di segnali" su storico BTC/ETH ANONIMIZZATO (Series A/B rebased a 100, calendario sintetico, split 70/30) — non sanno cosa siano. Ognuno scrive un signal(df)->position causale (script o ML), tunato solo sul train. Orchestratore valuta su PnL e maxDD nel test held-out. Harness cieco leak-free (riusabile): - make_blind.py: export anonimo + overlay; blindlib.py: evaluator con shift della posizione + GUARDIA DI CAUSALITA' online (squalifica ogni look-ahead, ML incluso); blind_eval.py CLI; score_all.py giudice OOS; verify_top.py (corr-al-trend, fee-stress, jackknife). - 52/52 passano la guardia (zero leak su tutta la flotta). Esito OOS (benchmark buy&hold: -7% PnL, 68% DD): - top = macd (+21%, DD 11%, Sh 0.84), accel, vol_of_vol, regime_switch, rf, obv — tutti trend/vol-regime. Sharpe OOS ~0.84 decade dal train ~1.4. Mean-rev e ML in fondo. - 3 scettici indipendenti: REFUTED. regime-luck (top-5 bar = 67-102% del PnL); trend-redundancy (HAC alpha t=+0.9..+1.5, nessuno >1.96 — TSMOM travestito); overfit (accel/vov knife-edge). Verdetto: ri-conferma CIECA e indipendente del soffitto direzionale ~1.3. macd = classe-TP01, forward-monitor non deploy. Diario 2026-06-21-blind-signal-fleet.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Adriano Dal Pastro
@@ -0,0 +1,188 @@
|
||||
"""blindlib — the ONLY module a blind-signal agent imports.
|
||||
|
||||
It hands you anonymized OVERLAID price curves ("Series A", "Series B") and an
|
||||
HONEST, leak-free evaluator. You never touch the real-data loaders, you never learn
|
||||
the tickers. Your job: write a CAUSAL `signal(df) -> position[]` that anticipates the
|
||||
move, tune it on the TRAIN view, and report PnL + max drawdown.
|
||||
|
||||
THE CONTRACT (read carefully — the orchestrator enforces it automatically):
|
||||
* `signal(df)` returns a float array len(df). position[i] in [-1, +1] is the
|
||||
fraction of equity you want to hold during the NEXT bar (sign = long/short,
|
||||
0 = flat). The evaluator SHIFTS it for you (held during bar i+1), so you can
|
||||
NEVER leak by multiplying a weight by the same bar's return.
|
||||
* It must be ONLINE / CAUSAL: position[i] may use ONLY rows 0..i of df. No
|
||||
`.shift(-k)`, no centered windows, no fitting a model on the whole df then
|
||||
predicting the whole df (at test time that df CONTAINS the held-out future).
|
||||
-> Verified by `causality_ok()`: we call signal on a truncated prefix and require
|
||||
the tail to match signal on the full array. A leaky signal is DISQUALIFIED.
|
||||
* Fees are real (Deribit 0.10% round-trip = 0.0005/side) and charged on turnover.
|
||||
|
||||
The metrics that decide validity (orchestrator ranks on these):
|
||||
* pnl = total net return over the period (final/initial - 1) <- "PNL"
|
||||
* maxdd = worst peak-to-trough drawdown of the equity curve <- "DD max"
|
||||
(sharpe / cagr / turnover reported for context.)
|
||||
|
||||
Toolkit: causal indicators are re-exported from the project's vetted altlib so you
|
||||
don't reinvent (or mis-implement) them. All are causal (value at i uses data <= i).
|
||||
|
||||
Typical agent usage:
|
||||
import blindlib as bl
|
||||
df = bl.load("A", "train") # anonymized training curve for Series A
|
||||
def signal(df):
|
||||
c = df["close"].values
|
||||
mom = c / bl.sma(c, 50) - 1.0 # causal
|
||||
return np.tanh(3.0 * mom) # position in [-1,1]
|
||||
print(bl.evaluate(signal, "A", "train")) # {pnl, maxdd, sharpe, ...}
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
_BLIND_DIR = Path("/opt/docker/PythagorasGoal/data/blind")
|
||||
sys.path.insert(0, "/opt/docker/PythagorasGoal/scripts/research/alt")
|
||||
|
||||
# Re-export causal indicators + the vol-targeting helper + the net->metrics core.
|
||||
# (These are pure math; they reveal nothing about the underlying asset.)
|
||||
from altlib import ( # noqa: E402
|
||||
simple_returns, log_returns, ema, sma, rolling_std, zscore, rsi, atr,
|
||||
realized_vol, donchian, bbands, vol_target, bars_per_day, bars_per_year,
|
||||
_metrics_from_net,
|
||||
)
|
||||
|
||||
FEE_SIDE = 0.0005 # 0.05%/side = 0.10% round-trip (Deribit taker)
|
||||
SERIES = ("A", "B")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DATA — anonymized loaders. "train" = agent-visible. "full"/"test" = orchestrator.
|
||||
# ---------------------------------------------------------------------------
|
||||
def _meta() -> dict:
|
||||
return json.loads((_BLIND_DIR / "blind_meta.json").read_text())
|
||||
|
||||
|
||||
def load(series: str, split: str = "train") -> pd.DataFrame:
|
||||
"""Anonymized OHLCV curve. split: 'train' (first 70%, what you tune on) |
|
||||
'full' (whole series) | 'test' (held-out tail only — for inspection; you should
|
||||
NOT tune on it). datetime is synthetic daily."""
|
||||
series = series.upper()
|
||||
if series not in SERIES:
|
||||
raise ValueError(f"Unknown series {series}; pick from {SERIES}")
|
||||
if split == "train":
|
||||
df = pd.read_parquet(_BLIND_DIR / f"blind_{series}_train.parquet")
|
||||
else:
|
||||
df = pd.read_parquet(_BLIND_DIR / f"blind_{series}_full.parquet")
|
||||
if split == "test":
|
||||
cut = int(len(df) * _meta()["split_frac"])
|
||||
df = df.iloc[cut:].reset_index(drop=True)
|
||||
return df.reset_index(drop=True)
|
||||
|
||||
|
||||
def split_cut(series: str) -> int:
|
||||
df = pd.read_parquet(_BLIND_DIR / f"blind_{series.upper()}_full.parquet")
|
||||
return int(len(df) * _meta()["split_frac"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# EVALUATION — leak-free (position shifted), fee on turnover, PnL + maxDD.
|
||||
# ---------------------------------------------------------------------------
|
||||
def eval_target(df: pd.DataFrame, target: np.ndarray, fee_side: float = FEE_SIDE,
|
||||
metric_mask: np.ndarray | None = None) -> dict:
|
||||
"""Backtest a per-bar position series on df. target[i] decided at close[i] is
|
||||
HELD during bar i+1 (shift done here). Fee on |Δposition|. If metric_mask is
|
||||
given, metrics are computed only on those bars (used for OOS = test slice)."""
|
||||
c = df["close"].values.astype(float)
|
||||
target = np.nan_to_num(np.asarray(target, float), nan=0.0)
|
||||
target = np.clip(target, -1.0, 1.0)
|
||||
r = simple_returns(c)
|
||||
pos = np.zeros(len(target))
|
||||
pos[1:] = target[:-1] # held during bar t = decided at t-1
|
||||
gross = pos * r
|
||||
turn = np.abs(np.diff(pos, prepend=0.0))
|
||||
net = gross - fee_side * turn
|
||||
net[0] = 0.0
|
||||
idx = pd.DatetimeIndex(pd.to_datetime(df["datetime"], utc=True))
|
||||
if metric_mask is not None:
|
||||
net_m, idx_m = net[metric_mask], idx[metric_mask]
|
||||
else:
|
||||
net_m, idx_m = net, idx
|
||||
m = _metrics_from_net(net_m, idx_m)
|
||||
bpy_d = bars_per_day(df) * 365.25
|
||||
tin = float(np.mean(pos[metric_mask] != 0)) if metric_mask is not None else float(np.mean(pos != 0))
|
||||
turn_m = turn[metric_mask].sum() if metric_mask is not None else turn.sum()
|
||||
span = max(len(net_m) / bpy_d, 1e-9)
|
||||
return dict(pnl=round(m["ret"], 4), maxdd=round(m["maxdd"], 4),
|
||||
sharpe=round(m["sharpe"], 3), cagr=round(m["cagr"], 4),
|
||||
n_bars=int(len(net_m)), time_in_market=round(tin, 3),
|
||||
turnover_per_year=round(float(turn_m / span), 1),
|
||||
net=net, idx=idx)
|
||||
|
||||
|
||||
def evaluate(signal_fn, series: str, split: str = "train",
|
||||
fee_side: float = FEE_SIDE) -> dict:
|
||||
"""Run signal_fn on the chosen view and return {pnl, maxdd, sharpe, ...}.
|
||||
train: signal sees only train rows, metrics over train.
|
||||
test : signal sees the FULL series (proper warmup) but metrics ONLY on the
|
||||
held-out tail -> the honest out-of-sample PnL/DD. (orchestrator use)
|
||||
full : signal + metrics over the whole series.
|
||||
"""
|
||||
if split == "train":
|
||||
df = load(series, "train")
|
||||
tgt = np.asarray(signal_fn(df), float)
|
||||
rep = eval_target(df, tgt, fee_side)
|
||||
else:
|
||||
df = load(series, "full")
|
||||
tgt = np.asarray(signal_fn(df), float)
|
||||
mask = None
|
||||
if split == "test":
|
||||
cut = split_cut(series)
|
||||
mask = np.zeros(len(df), bool); mask[cut:] = True
|
||||
rep = eval_target(df, tgt, fee_side, metric_mask=mask)
|
||||
rep.pop("net", None); rep.pop("idx", None)
|
||||
return rep
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CAUSALITY GUARD — disqualifies look-ahead. Online-consistency: signal on a
|
||||
# prefix must agree (on its tail) with signal on the full array. A function that
|
||||
# uses future rows, centered windows, or fits globally on the input will diverge.
|
||||
# ---------------------------------------------------------------------------
|
||||
def causality_ok(signal_fn, series: str = "A", split: str = "full",
|
||||
tail: int = 60, tol: float = 1e-4) -> dict:
|
||||
"""Returns {ok, max_diff, frac_bad, checked_at}. We truncate the input at two
|
||||
late cut points and require signal(df[:cut]) to match signal(df)[:cut] over the
|
||||
last `tail` bars before each cut (the bars a deployable signal would have emitted
|
||||
in real time)."""
|
||||
df = load(series, split)
|
||||
full = np.nan_to_num(np.asarray(signal_fn(df), float), nan=0.0)
|
||||
n = len(df)
|
||||
cuts = [int(n * 0.80), int(n * 0.92)]
|
||||
max_diff = 0.0; frac_bad = 0.0; checked = []
|
||||
for cut in cuts:
|
||||
if cut <= tail + 5 or cut >= n:
|
||||
continue
|
||||
sub = np.nan_to_num(np.asarray(signal_fn(df.iloc[:cut].reset_index(drop=True)), float), nan=0.0)
|
||||
if len(sub) != cut:
|
||||
return dict(ok=False, reason=f"signal returned len {len(sub)} != {cut} on prefix",
|
||||
max_diff=9.99, frac_bad=1.0, checked_at=cut)
|
||||
a = sub[cut - tail:cut]
|
||||
b = full[cut - tail:cut]
|
||||
d = np.abs(a - b)
|
||||
max_diff = max(max_diff, float(np.max(d)) if len(d) else 0.0)
|
||||
frac_bad = max(frac_bad, float(np.mean(d > tol)) if len(d) else 0.0)
|
||||
checked.append(cut)
|
||||
ok = (max_diff <= max(tol * 10, 1e-3)) and (frac_bad <= 0.02)
|
||||
return dict(ok=bool(ok), max_diff=round(max_diff, 6), frac_bad=round(frac_bad, 4),
|
||||
checked_at=checked)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"load", "split_cut", "evaluate", "eval_target", "causality_ok", "FEE_SIDE",
|
||||
"SERIES", "simple_returns", "log_returns", "ema", "sma", "rolling_std",
|
||||
"zscore", "rsi", "atr", "realized_vol", "donchian", "bbands", "vol_target",
|
||||
"bars_per_day", "bars_per_year",
|
||||
]
|
||||
Reference in New Issue
Block a user