"""Agent 36 — RandomForest direction model (family=ml, slug=rf). THE ANGLE (assigned): a RandomForestClassifier on a causal technical feature vector, refit on an EXPANDING walk-forward window every ~25 bars. The forest VOTES on "will the forward multi-bar move be up?"; the fraction of trees voting up (an out-of-bag-ish ensemble consensus) is mapped to a position in [-1, +1]. RF is the BAGGED-TREE cousin of the linear logit / tiny MLP: it can pick up threshold-y, non-monotone feature interactions (e.g. "momentum up AND vol low") that a linear model cannot, while the bagging averages out the variance of individual trees on a thin edge. WHY A CLASSIFIER (sign, not magnitude): per-bar return magnitude on these curves is dominated by noise; only the SIGN of a multi-bar forward move has any persistence. The forest targets that Bernoulli up/down label; the vote fraction is a natural conviction (0.5 = no edge -> flat; far from 0.5 = take the side). Shallow trees + a min-leaf floor + many trees keep it from memorizing noise. CAUSALITY (the whole game): * Features at row i use ONLY data up to and including bar i (rows <= i): lagged log-returns, multi-horizon trailing momentum, trailing realized vol, RSI, distance-from-MA. * The LABEL for row j is the sign of the cumulative return over bar j -> j+FWD_H, which needs close[j+FWD_H]. Sitting at decision-row i we train ONLY on rows whose label is already realized: j + FWD_H <= i => j <= i - FWD_H. Row i's own label is NEVER used. * The forest is refit on the EXPANDING window of those realized (X, y) pairs at most every REFIT_EVERY (~25) bars; frozen in between. position[i] = frozen forest vote at row i, mapped to a direction, then vol-targeted. Deterministic (fixed random_state, capped depth) so signal(prefix) == signal(full)[:cut] -> passes the causality guard. TUNING (split='train' only, combined A & B): shallow trees (MAX_DEPTH) + a big MIN_LEAF so the weak lag->sign edge isn't memorized; FWD_H in the forecastable band (next-bar sign is a coin-flip, the multi-bar sign persists); a deadband on the centered vote to avoid fee churn; an asymmetric short scale (both curves drift UP, so the forest's real value is STEPPING ASIDE from declines, not fighting the drift with full shorts); then vol-target (cap 1.0) so the DRAWDOWN, not the raw forecast, is what we control. HONEST READ: forward-sign forecastability here is weak and a RandomForest does not manufacture it. The realistic, defensible win is a vol-controlled, low-drawdown book that de-risks/flips into declines — comparable PnL to long-only at a FRACTION of the ~70-80% buy&hold drawdown. The de-risking is the alpha, not a strong classifier. A thin/negative result is the honest result for this angle. """ import warnings import numpy as np import blindlib as bl warnings.filterwarnings("ignore") try: from sklearn.ensemble import RandomForestClassifier _HAVE_SK = True except Exception: # pragma: no cover - sklearn expected present _HAVE_SK = False # ---- tuned on split='train' only (interior of broad plateaus; see scans) ---- N_TREES = 120 # many shallow trees -> bagging averages the thin-edge variance MAX_DEPTH = 4 # SHALLOW (edge is tiny -> resist memorizing noise) MIN_LEAF = 40 # big leaf floor: each split must keep a real sample -> smooth votes MAX_FEATURES = "sqrt" # decorrelate trees (classic RF default) WARMUP = 220 # realized (X, y) pairs required before the first fit REFIT_EVERY = 30 # expanding-window refit cadence (~25 assigned; 30 keeps us in budget) LAGS = (1, 2, 3, 5) # lagged log-return features MOM_WINS = (10, 20, 40) # multi-horizon trailing-momentum features VOL_WIN = 20 # trailing realized-vol feature window RSI_WIN = 14 # RSI feature window MA_WIN = 50 # distance-from-MA feature window FWD_H = 20 # label HORIZON: sign of cumulative return over next FWD_H bars. Next-bar # sign is a coin-flip; the longer multi-bar sign is the persistent, # classifiable quantity. Train scan: shmin rises monotone with H to ~20 # then fades (H30 overfits) -> H=20 (plateau 18-25). # --- vote -> position MAPPING (long-sizing under a causal trend gate) --- # The forest VOTE (fraction of trees voting up) sizes the LONG; it never shorts. Train # ablation was decisive: (1) shorting the up-drift strictly worsens shmin/DD on both curves # (vote on declines is unreliable); (2) a causal trend GATE that blocks longs below a trailing # SMA cuts the worst drawdown (B 0.30->0.12) AND lifts PnL — it stops the book holding long # THROUGH the big declines, exactly where the forest's vote is least trustworthy. So the # deployable book is: long-only, gated by trend, with the FOREST sizing the exposure inside the # uptrend (step partly aside when its vote is weak). HONEST: the gate+vol-target do most of the # de-risking; the vote's marginal lift is real but modest (floor=0.35 keeps it material without # letting it dominate). This is the defensible RF result, not a strong stand-alone classifier. TREND_GATE_WIN = 50 # block longs when close < trailing SMA(this) -> de-risk declines VOTE_GAIN = 2.0 # sharpen the centered vote (v-0.5) before squashing to [0,1] LONG_FLOOR = 0.35 # min long size when gated-in & vote barely up (vote swings 0.35..1.0) TARGET_VOL = 0.20 # vol-target the directional book VOL_WIN_DAYS = 30 LEV_CAP = 1.5 # modest leverage headroom in calm regimes (cap rarely binds) def _build_features(c): """Causal feature matrix X (len(c) rows). Row i uses ONLY data <= i.""" n = len(c) lr = np.zeros(n) lr[1:] = np.log(c[1:] / c[:-1]) # lr[i] = return of bar ending at i (causal) csum = np.cumsum(lr) cs2 = np.cumsum(lr * lr) cols = [] # lagged returns: value at i is the return k bars ago (all <= i) for k in LAGS: f = np.zeros(n) if k < n: f[k:] = lr[: n - k] cols.append(f) # multi-horizon trailing momentum: cumulative log-return over last w bars (<= i) for w in MOM_WINS: mom = np.zeros(n) mom[w:] = csum[w:] - csum[:-w] cols.append(mom) # trailing realized vol (std of last VOL_WIN returns, <= i) vol = np.zeros(n) for i in range(VOL_WIN, n): m = (csum[i] - csum[i - VOL_WIN]) / VOL_WIN v = (cs2[i] - cs2[i - VOL_WIN]) / VOL_WIN - m * m vol[i] = np.sqrt(max(v, 0.0)) cols.append(vol) # RSI (causal, from blindlib), centered to ~[-0.5, 0.5] rsi = np.nan_to_num(bl.rsi(c, RSI_WIN), nan=50.0) / 100.0 - 0.5 cols.append(rsi) # distance from a trailing MA (causal): log(close / sma) ma = np.nan_to_num(bl.sma(c, MA_WIN), nan=c[0]) ma[ma <= 0] = 1e-9 dist = np.log(np.maximum(c, 1e-9) / ma) dist[:MA_WIN] = 0.0 cols.append(dist) X = np.column_stack(cols) return X, lr, csum def _fit(Xtr, ytr): """RandomForest fit. Returns model or None if labels are single-class (no fit yet).""" if not _HAVE_SK or len(np.unique(ytr)) < 2: return None m = RandomForestClassifier( n_estimators=N_TREES, max_depth=MAX_DEPTH, min_samples_leaf=MIN_LEAF, max_features=MAX_FEATURES, bootstrap=True, random_state=0, n_jobs=1, ) m.fit(Xtr, ytr) return m def _up_index(model): """Column index of the 'up' (label 1.0) class in predict_proba, or None.""" classes = list(model.classes_) return classes.index(1.0) if 1.0 in classes else None def signal(df): c = df["close"].values.astype(float) n = len(c) X, lr, csum = _build_features(c) # label[j] = 1 if cumulative return over bar j -> j+FWD_H is up, else 0. # realized (known) only as of close[j+FWD_H]. fwd = np.zeros(n) fwd[: n - FWD_H] = csum[FWD_H:] - csum[: n - FWD_H] label = (fwd > 0).astype(float) first = max(max(LAGS), max(MOM_WINS), VOL_WIN, RSI_WIN, MA_WIN) # first fully-featured row vote = np.full(n, 0.5) model = None # Walk forward in REFIT_EVERY-bar BLOCKS. The forest is frozen within a block, so we refit # once at the block start (on labels realized as of that bar) and BATCH-predict the whole # block in a single predict_proba call. This is identical, bar-for-bar, to a per-bar loop # that refits at multiples of REFIT_EVERY (the model is constant across the block) but # ~REFIT_EVERY x fewer forest evaluations -> fits the <30s budget. Still strictly causal: # every prediction at row i uses a model fit only on labels realized at or before i. i = 0 while i < n: blk_end = min(i + REFIT_EVERY, n) last_train = i - FWD_H # labels <= last_train are realized as of close[i] ntrain = last_train - first + 1 if ntrain >= WARMUP: Xtr = X[first : last_train + 1] ytr = label[first : last_train + 1] fit = _fit(Xtr, ytr) if fit is not None: model = fit if model is not None: j = _up_index(model) if j is not None: proba = model.predict_proba(X[i:blk_end]) vote[i:blk_end] = proba[:, j] i = blk_end # vote -> LONG-SIZING direction in [0, 1]. Center the vote at 0.5, sharpen with tanh, then # map the up-half to [LONG_FLOOR, 1]; a vote <= 0.5 (no up-conviction) -> flat. The forest # thus sizes how MUCH long to hold, never short. sharp = np.tanh(VOTE_GAIN * (vote - 0.5)) / np.tanh(VOTE_GAIN * 0.5) # ~[-1, 1] up = np.clip(sharp, 0.0, 1.0) # only up-conviction long_size = np.where(up > 0.0, LONG_FLOOR + (1.0 - LONG_FLOOR) * up, 0.0) # causal trend GATE: block longs when price is below its trailing SMA (de-risk declines — # where the vote is least reliable and the curves take their worst draws). sma() at i uses # only rows <= i, so the whole pipeline stays online. ma = np.nan_to_num(bl.sma(c, TREND_GATE_WIN), nan=c[0]) in_trend = c >= ma direction = np.where(in_trend, long_size, 0.0) direction = np.nan_to_num(direction, nan=0.0) pos = bl.vol_target(direction, df, target_vol=TARGET_VOL, vol_win_days=VOL_WIN_DAYS, leverage_cap=LEV_CAP) return np.clip(np.nan_to_num(pos, nan=0.0), -1.0, 1.0)