PythagorasGoal/scripts/research/blind/agents/agent_41_entropy.py

"""Agent 41 — Entropy/randomness gate (family=stat, slug=entropy).

The angle (assigned): estimate the PREDICTABILITY of the recent path and only take
the trend when the path is STRUCTURED (low entropy / non-random). When the recent
path is statistically random the trend is noise -> scale exposure down toward flat.

How the gate is built (and why NOT permutation entropy)
-------------------------------------------------------
Permutation entropy (Bandt-Pompe) of DAILY returns is near-saturated (~0.98 of max)
on these curves; when I measured it, its "low-entropy" regime actually had a NEGATIVE
edge for trend-following (-0.07/-0.03 hit-rate on A/B). The discriminating, well-ranged
"is the path random?" statistic here is the KAUFMAN EFFICIENCY RATIO over a window W:

    ER[i] = |logC[i] - logC[i-W]|  /  sum_{i-W<k<=i} |Δ logC[k]|        in [0,1]

ER is exactly an INVERSE path-entropy: ER->1 means every step pushed the same way (a
clean, low-entropy directional move -> the trend is predictable); ER->0 means the
steps cancelled out (a high-entropy random walk / chop -> the trend is noise). It is
the canonical randomness gate for trend systems (KAMA is built on it). I blend a short
and a medium window so the gate reacts to fast chop yet respects the macro structure.

Measured on train (per-bar): trend-following PnL is markedly higher in the high-ER
(low-entropy) half than the low-ER half on BOTH curves -> the gate does what the angle
promises: concentrate trend exposure in the predictable, structured legs and stand
down in the random chop (which are also the chaotic crash legs that drive drawdown).

Honest finding: ungated multi-horizon TSMOM has a slightly HIGHER Sharpe on these two
relentlessly up-trending curves (gating away "random" stretches removes some good
trend too). The entropy gate's real, robust contribution is DRAWDOWN: it cuts the
worst train DD from ~0.207 (ungated) to ~0.162 while keeping the Sharpe within ~6%
(1.37 -> 1.29). So this is a risk-reducing overlay, not a Sharpe-maximiser — reported
honestly. To get that DD cut without throwing away return I gate ONLY the bottom of
the ER distribution (genuinely random regimes) and keep half size there, rather than
linearly fading the whole range (which over-suppressed and lost ~0.3 of Sharpe).

Pipeline
--------
1. Direction: causal multi-horizon TSMOM sign blend (the trend we *might* take).
2. Entropy gate g in [FLOOR,1]: soft ramp on the LOW end of the ER distribution only.
   ER below an expanding Q_LO quantile -> FLOOR; ER above an expanding Q_MID quantile
   -> 1.0; linear in between. Quantiles are EXPANDING (history <= i) so "random vs
   structured" is judged vs this series' own past, never the future.
3. Size = direction * gate, then a causal vol-target so A & B are risk-comparable.

CAUSAL: ER at i uses only logC in (i-W, i]; gate quantiles are EXPANDING (history
<= i); vol_target uses a trailing window. No look-ahead, no centered windows, no
global fit. Verified by causality_ok (max_diff 0.0).

Tuning (train only, combined A&B). Coarse->fine sweep over ER windows, the gate
quantiles, the floor, and SHORT_W settled on a WIDE interior plateau:
  ER_WINS=(30,90), Q_LO=0.10, Q_MID=0.50, FLOOR=0.50, SHORT_W=0.25
  -> train combined: pnl_mean ~2.63, maxdd_worst ~0.162, sharpe_min ~1.29.
All 1-step neighbours (window, qlo/qmid, floor in [0.45..0.55], short_w in [0..0.4])
sit in the same plateau (sh_min 1.26..1.32, dd 0.16..0.19) -> robust, not a spike.
"""
import numpy as np
import blindlib as bl

# --- trend direction (multi-horizon TSMOM sign blend) ---
HORIZONS = (45, 130, 240)   # ~1.5/4.5/8 months of daily bars
SHORT_W = 0.25              # de-weight short side (curves trend up); 0 -> long-flat

# --- entropy / randomness gate (efficiency ratio = inverse path entropy) ---
ER_WINS = (30, 90)         # blended short+medium ER windows
Q_LO = 0.10                # expanding-quantile of ER below which gate = FLOOR
Q_MID = 0.50               # expanding-quantile of ER above which gate = 1.0
FLOOR = 0.50               # exposure kept in the most-random (high-entropy) regime
WARMUP = 120               # bars before the gate is trusted (else FLOOR)
HIST_MIN = 60              # min ER history before quantiles are meaningful

# --- sizing ---
TARGET_VOL = 0.30
VOL_WIN_DAYS = 45
LEV_CAP = 1.5


def _tsmom_sign(c, h):
    """Sign of the past-h-bar return, causal. 0 before warmup (i < h)."""
    out = np.zeros(len(c))
    if h < len(c):
        out[h:] = np.sign(c[h:] / c[:-h] - 1.0)
    return out


def _efficiency_ratio(logc, win):
    """Causal Kaufman efficiency ratio over `win` bars: |net move| / sum|steps|.
    er[i] uses logc in (i-win, i] only. ER in [0,1]: 1 = clean directional (low
    entropy), 0 = random chop (high entropy)."""
    n = len(logc)
    er = np.zeros(n)
    abs_step = np.zeros(n)
    abs_step[1:] = np.abs(np.diff(logc))
    csum = np.cumsum(abs_step)
    for i in range(win, n):
        change = abs(logc[i] - logc[i - win])
        vol = csum[i] - csum[i - win]
        er[i] = change / vol if vol > 1e-12 else 0.0
    return er


def _expanding_gate(er):
    """Map ER -> [FLOOR, 1] with a soft ramp on the LOW end of the ER distribution.
    ER below expanding-quantile Q_LO -> FLOOR (random regime, stand down); ER above
    expanding-quantile Q_MID -> 1.0 (structured regime, full trend); linear between.
    Fully causal: only ER history (values <= i) feeds the quantiles."""
    n = len(er)
    gate = np.full(n, FLOOR)
    hist = []
    for i in range(n):
        v = er[i]
        if i >= WARMUP and len(hist) >= HIST_MIN and np.isfinite(v):
            arr = np.asarray(hist)
            lo = np.quantile(arr, Q_LO)
            mid = np.quantile(arr, Q_MID)
            if v >= mid:
                gate[i] = 1.0
            elif mid > lo:
                g = FLOOR + (1.0 - FLOOR) * (v - lo) / (mid - lo)
                gate[i] = float(np.clip(g, FLOOR, 1.0))
            else:
                gate[i] = 1.0
        if np.isfinite(v) and v > 0:
            hist.append(v)
    return gate


def signal(df):
    c = df["close"].values.astype(float)
    logc = np.log(c)

    # 1) trend direction: multi-horizon TSMOM sign blend, asymmetric long-short
    sig = np.zeros(len(c))
    for h in HORIZONS:
        sig += _tsmom_sign(c, h)
    sig /= len(HORIZONS)
    raw = np.where(sig >= 0.0, sig, sig * SHORT_W)

    # 2) entropy/randomness gate from blended efficiency ratios (inverse path entropy)
    gate = np.zeros(len(c))
    for w in ER_WINS:
        gate += _expanding_gate(_efficiency_ratio(logc, w))
    gate /= len(ER_WINS)

    # 3) gated direction, causal vol-target so A & B are risk-comparable
    gated = raw * gate
    pos = bl.vol_target(gated, df, target_vol=TARGET_VOL,
                        vol_win_days=VOL_WIN_DAYS, leverage_cap=LEV_CAP)
    return np.clip(np.nan_to_num(pos, nan=0.0), -1.0, 1.0)