"""Agent 43 — Kalman local-level+slope online filter (family=cycle, slug=kalman). The angle (assigned): a Kalman / local-linear-trend filter run fully ONLINE on the log-price. The hidden state is [level, slope] with a constant-velocity transition level_t = level_{t-1} + slope_{t-1} + w_l (w_l ~ N(0, Q_LEVEL)) slope_t = slope_{t-1} + w_s (w_s ~ N(0, Q_SLOPE)) obs_t = level_t + v (v ~ N(0, OBS_VAR)) We run the textbook predict/update recursion bar by bar using ONLY data <= i, then take the position from the SIGN/MAGNITUDE of the *filtered slope*: an up-sloping latent trend -> long, a flattening/down-sloping one -> de-risk toward flat. The filter is the cycle/trend extractor; its derivative (the slope state) is the anticipation signal — it bends down BEFORE price has fully rolled over, because the slope state carries momentum and decays as observations come in below the predicted level. Design choices that matter (all tuned on split='train', combined A&B): * Filter on LOG price -> the slope is a per-bar geometric growth rate, comparable across the two differently-scaled curves (A ~8x, B ~24x over the train window). * The signal-to-noise ratio is the only real knob. We split process noise into a level term Q_LEVEL and a much smaller slope term Q_SLOPE: the level tracks fast, the slope stays a smooth, persistent trend that turns gradually (few whipsaws). * Direction = the filtered slope normalized by its OWN trailing dispersion (a causal z-score) squashed through tanh -> a graded -1..+1 conviction, not a hard flip. The z makes the signal scale-free and self-calibrating across regimes. * LONG-FLAT (no short): both curves trend persistently up; on split='train' a symmetric short bleeds (it shorts dips). The Kalman edge here is to be fully long when the latent slope is up and step OUT (toward flat) when it turns — that is what cuts the drawdown vs buy&hold without paying the short-side drag. (Sweep: short_w 0.0 -> sharpe_min 1.42; 0.5 -> 1.17; 1.0 -> 0.87.) * Vol-target on top so the two curves are risk-comparable and DD stays bounded. Sharpe is invariant to TARGET_VOL (it scales PnL and DD together); TARGET_VOL is chosen to land DD ~24% with strong PnL. WHY IT WINS THE BRIEF: long-only buy&hold on train is PnL 6.7/23.0 at DD ~0.77/0.79 (sharpe 0.89/1.16). The Kalman-slope signal delivers PnL ~2.0/2.5 at DD ~0.24 with sharpe ~1.42 on BOTH curves — comparable/positive PnL at ~3x smaller drawdown, by anticipating the rollovers via the filtered slope. CAUSAL/ONLINE: the Kalman recursion is the canonical online filter — state at i is a function of states/observations 0..i only. The slope z uses a trailing window; vol_target uses trailing realized vol. No .shift(-k), no centered window, no global fit. Verified by causality_ok (max_diff 0.0). Tuning plateau (train, combined): the chosen cell is INTERIOR on every axis. Q_LEVEL in [1e-2..1e-1], Q_SLOPE=1e-3 -> sharpe_min 1.39..1.46 SLOPE_Z_WIN in [60..75], TANH_K in [0.9..1.5] -> sharpe_min 1.42..1.44 Chosen: Q_LEVEL=3e-2, Q_SLOPE=1e-3, SLOPE_Z_WIN=60, TANH_K=1.2, TARGET_VOL=0.26, VOL_WIN_DAYS=60, LEV_CAP=1.5, short_w=0 -> train combined: pnl_mean ~2.25, maxdd_worst ~0.24, sharpe_min ~1.42. """ import numpy as np import pandas as pd import blindlib as bl # --- Kalman knobs (signal-to-noise; process_var = Q_* * OBS_VAR) --- OBS_VAR = 1.0 # measurement noise variance (scale-free reference) Q_LEVEL = 3e-2 # process noise on the level (tracks the price fast) Q_SLOPE = 1e-3 # process noise on the slope (smaller -> smooth, persistent trend) # --- signal shaping --- SLOPE_Z_WIN = 60 # trailing window to normalize the filtered slope into a z TANH_K = 1.2 # squash gain on the slope-z -> conviction in [-1,1] SHORT_W = 0.0 # de-weight the short side; 0 = LONG-FLAT (curves trend up) # --- sizing --- TARGET_VOL = 0.26 VOL_WIN_DAYS = 60 LEV_CAP = 1.5 def _kalman_slope(logp: np.ndarray) -> np.ndarray: """Online local-linear-trend Kalman filter on a log-price series. State x = [level, slope] with a constant-velocity transition. Returns the filtered slope at each bar. Causal: slope[i] uses observations 0..i only.""" n = len(logp) slope_out = np.zeros(n) if n == 0: return slope_out F = np.array([[1.0, 1.0], [0.0, 1.0]]) # level += slope ; slope persists H = np.array([[1.0, 0.0]]) # we observe the level (log-price) Q = np.array([[Q_LEVEL, 0.0], [0.0, Q_SLOPE]]) * OBS_VAR R = OBS_VAR x = np.array([logp[0], 0.0]) # level = first obs, slope = 0 P = np.eye(2) # mildly diffuse prior slope_out[0] = 0.0 for i in range(1, n): # predict x = F @ x P = F @ P @ F.T + Q # update with observation logp[i] innov = logp[i] - (H @ x)[0] # innovation S = (H @ P @ H.T)[0, 0] + R # innovation variance K = (P @ H.T).ravel() / S # Kalman gain (2,) x = x + K * innov P = P - np.outer(K, H @ P) slope_out[i] = x[1] return slope_out def _causal_z(x: np.ndarray, win: int) -> np.ndarray: """Trailing z-score over a backward window (causal: uses x[<=i] only).""" s = pd.Series(x) mp = max(5, win // 4) m = s.rolling(win, min_periods=mp).mean() sd = s.rolling(win, min_periods=mp).std(ddof=0) z = (s - m) / sd.replace(0.0, np.nan) return z.fillna(0.0).values def signal(df): c = df["close"].values.astype(float) logp = np.log(np.maximum(c, 1e-9)) slope = _kalman_slope(logp) # filtered local trend (derivative) z = _causal_z(slope, SLOPE_Z_WIN) # self-calibrating conviction direction = np.tanh(TANH_K * z) # -1..+1 # long-flat (short de-weighted by SHORT_W; 0 -> never short) raw = np.where(direction >= 0.0, direction, direction * SHORT_W) pos = bl.vol_target(raw, df, target_vol=TARGET_VOL, vol_win_days=VOL_WIN_DAYS, leverage_cap=LEV_CAP) return np.clip(np.nan_to_num(pos, nan=0.0), -1.0, 1.0)