Validation discipline, not a performance claim
Read this as a worked example of honest evaluation, not as evidence that this strategy makes money. The 91-day sample below is now the trailer, not the movie: far too small to conclude anything on its own, it stays here to show how to see that, rather than hide it. The movie — six years, eight symbols — follows the caveats.
Setup: 91 days of real NEARUSDT 1-minute data (mid-April → mid-July 2026), aggregated to 60-minute bars, with Binance taker fees (0.10%/side) and slippage (0.02%/side) charged on every fill. Walk-forward re-optimises a deliberately small grid on rolling 30-day windows and reports only the unseen 10-day windows that follow.
| Configuration | OOS return | Sharpe | Max DD | Trades |
|---|---|---|---|---|
| v1 scalper @ 4m (the old bot) | −82.4% | — | −82.5% | 714 |
| v1 scalper @ 60m (same strategy, slower) | −9.4% | — | −17.0% | 39 |
| v2 @ 60m — walk-forward OOS | +10.7% | +2.84 | −7.5% | 38 |
| v2 @ 60m — 2× cost stress | +13.3% | +3.54 | −5.9% | 36 |
| v2 @ 60m — ETHUSDT (held-out symbol) | +5.8% | +2.15 | −7.5% | 26 |
| v2 @ 60m — SOLUSDT (held-out symbol) | +0.3% | +0.18 | −5.7% | 35 |
Read the +10.7% row together with the caveats below — on this sample it is one draw from a wide distribution, not evidence of skill. Every figure is reproducible in two commands (see Quickstart), including the ones that undercut the strategy.
The numbers above are statistically inconclusive, on purpose
- The trade count is inflated. 38 "trades" are the legs of a top-3 parameter ensemble over the same windows, so they are heavily correlated — the number of independent decisions is closer to ~13. A Sharpe of 2.84 estimated on ~13 effective, serially-dependent observations has a standard error wide enough that the 95% confidence interval comfortably contains zero.
- The 2× cost stress going up is a warning, not a win. Costs cannot improve any individual trade; the rise from +10.7% to +13.3% means the in-sample optimiser simply selected different parameters under stressed costs — a direct measurement of selection noise, reported instead of buried.
- One market phase. 91 days is a single trending regime for NEAR. ETH (+5.8%) and especially SOL (+0.3% — i.e. nothing) on the same window are the honest counter-evidence, deliberately left in.
- Buy & hold beat it on raw return (+38.2% over the same windows) with far larger drawdowns; the strategy was invested only ~30% of the time.
Six years later: the multi-year verdict
The caveats above named what a real claim would need: multi-year data, bootstrapped confidence intervals and a deflated Sharpe ratio. That run is done — NEAR, ETH, SOL, BTC, BCH, DOGE, ADA and LINK 1-minute klines from 2021-01-01 to 2026-09-30 (SHA-256-verified Binance dumps), walk-forward on two window sizes, 10,000-resample block-bootstrap confidence intervals, independent decisions counted, and a deflated Sharpe charged against the trials ledger. All eighteen symbol × window blocks: not statistically distinguishable from noise.
| Walk-forward OOS, 2021–2026 | Sharpe, 90d/30d windows [95% CI] | Sharpe, 30d/10d windows [95% CI] | Verdict |
|---|---|---|---|
| NEARUSDT (production) | +0.35 [−0.50, +1.17] | +0.53 [−0.27, +1.28] | noise |
| ADAUSDT | +0.28 [−0.61, +1.11] | +0.61 [−0.22, +1.43] | noise |
| BCHUSDT | −0.22 [−1.17, +0.63] | −0.21 [−1.10, +0.60] | noise · a loss (−34% / −35%) |
| DOGEUSDT | +0.68 [−0.28, +1.54] | +0.89 [−0.02, +1.71] | noise · closest call (30d/10d)† |
| ETHUSDT | +0.27 [−0.62, +1.13] | +0.42 [−0.47, +1.29] | noise |
| LINKUSDT | +0.47 [−0.35, +1.24] | +0.24 [−0.57, +1.02] | noise |
| SOLUSDT | +0.47 [−0.37, +1.26] | +0.75 [−0.12, +1.58] | noise |
| Pooled (7 symbols) | +0.52 [−0.36, +1.36] | +0.70 [−0.16, +1.53] | noise |
| BTCUSDT (context stream) | −0.20 [−1.20, +0.75] | +0.36 [−0.56, +1.24] | noise |
Every Sharpe and annualised-return confidence interval includes zero. Independent decisions per block: 288–464; engine max drawdowns −22% to −61% (the pooled blocks −22% / −25%; per symbol −29% to −61%, BCH worst). BCH lost money on both window sizes: total return −34% / −35%, max drawdown −60% / −61%. † DOGE 30d/10d is the closest call: a deflated-Sharpe probability of 0.949 against the 0.95 gate, and the one per-decision expectancy CI that excludes zero — but its Sharpe CI [−0.02, +1.71] still includes zero. Full report: reports/validate.md.
The obvious next question — could better parameters
rescue it? — got a pre-registered answer: 125 configurations on 2021–2024
(NEAR/ETH/SOL), with the grid, the selection rule and the trials-ledger charge committed
before any configuration was scored. It selected the corner cell
theta_in=0.15, trail_atr=4.0, er_min=0.1: in-sample Sharpe +1.13 [95% CI +0.15,
+2.07], but a deflated-Sharpe probability of 0.69 against the required 0.95 after charging
for 219 configurations — not established. The corner is the tell: the search converged
toward the loosest filters, i.e. toward more time in the market, and on the same bars it
trails plain buy-and-hold.
| 2021–2024, pooled equal weight | Selected cell | Buy & hold |
|---|---|---|
| Sharpe | +1.13 | +1.32 |
| Total return | +150% | +2472% |
| Max drawdown | −30% | −89% |
| Exposure · correlation with buy & hold | 39% · +0.58 | 100% · — |
Post-hoc descriptive context, not used for selection. Lower Sharpe than buy-and-hold on every symbol, with a much shallower drawdown at ~39% exposure: a partially invested version of the same market exposure, not timing skill. Full report: reports/plateau_hunt.md.
The holdout (2025-01-01 → 2026-09-30) was deliberately not fired: a selection that fails its own pre-registered gate does not get to spend it. It stays sealed for a future strategy family — as a viewed holdout, not a pristine one, since those months are part of the validation run above.
The v1 scalper is decisively dead (−82% out-of-sample, a robust finding). v2 is not distinguishable from noise — first on 91 days, then on six years across eight symbols, with the bootstrapped confidence intervals and deflated Sharpe the 91-day sample lacked. A pre-registered search for better parameters converged toward market exposure and trails buy-and-hold.
The deliverable here is the machinery — and the discipline to reach that conclusion honestly — not the +10.7%.
python -m near_bot walkforward --plot.