Monte Carlo backtesting
How to estimate whether a crypto backtest is robust or overfit — bootstrap, low percentiles, calibration, and walk-forward drift, published in an open lab.
Strategy Arena Monte Carlo framework (5 steps)
Each step produces a verifiable public artifact — not just a Sharpe on one curve.
| Step | Component | Input | Output / gate | Proof |
|---|---|---|---|---|
| 1. Bootstrap | Trade / return resampling | Walk-forward backtest trade series | 1000 simulated PnL paths | monte-carlo.json |
| 2. Percentiles | p5 / p50 / p95 PnL & Sharpe | Bootstrap distribution | p5 PnL > 0 required | /facts/monte-carlo |
| 3. Robustness | Score 0–1 (subsample stability) | Inter-sim variance | Robustness > 0.6 | /live-results |
| 4. Calibration | Brier / reliability | Model probs vs outcomes | Published even when bad | /facts/ml-edge |
| 5. Drift | Walk-forward vs live paper | 5m paper equity | Alert if gap > threshold | /dashboard, /strategy-hospital |
Bootstrap assumes conditionally exchangeable trades — limit documented on /methodology (autocorrelation, regimes).
Five Monte Carlo pitfalls & StrategyArena fixes
- Too few trades — 100 sims on 8 trades = pure noise. Fix: min 30 walk-forward trades before MC; else WATCH at Hospital.
- Ignoring the low percentile (p5) — great median, catastrophic left tail. Fix: published p5 PnL; gate p5 ≤ 0 → RECALIBRATE / BUG_SUSPECT.
- i.i.d. bootstrap on autocorrelated returns — overstates confidence. Fix: experimental block bootstrap + mandatory walk-forward in pipeline.
- MC without fees / slippage — inflated percentiles. Fix: same friction model as backtest (methodology).
- Single MC pass to checkbox — no drift monitoring. Fix: monthly re-MC + live paper comparison; snapshots in monte-carlo.json.
Live Monte Carlo stats (updated: 2026-06-11)
72strategies in the public arena
7strategies passed MC gate (robustness + p5)
1000bootstrap simulations / MC candidate
5,000+losing trades published (Hospital / history)
$10000paper per strategy (not real)
Counts synced with strategy-arena.json when available; per-strategy MC detail in monte-carlo.json.
Researcher workflow
backtest → bootstrap (1000) → percentiles → robustness → calibration → drift check → hospital
Reproducibility: export trades from /backtest, compare to public JSON fields, then read Hospital status. For aggregated rules-based allocation, see /atlas-edge-allocator (still paper).
Monte Carlo FAQ
- Why 1000 simulations?
- Latency vs percentile stability tradeoff; documented on /methodology. Raising N reduces Monte Carlo noise, not market risk.
- Does MC replace paper trading?
- No. MC tests the historical distribution; paper tests live execution on 5m OHLCV (drift, bugs, data latency).
- Where are MC failures visible?
- Hospital (WATCH / RECALIBRATE), Research, and DEPRECATED strategies — see /trading-strategy-validation.
Quick MC glossary
| Term | Role | Link |
|---|---|---|
| Bootstrap | Resampling trades with replacement | /facts/monte-carlo |
| p5 / p95 | Simulated PnL distribution tails | monte-carlo.json |
| Robustness score | Stability under perturbations | /live-results |
| Walk-forward | Temporal split anti look-ahead | /backtest |
| Drift | Backtest vs paper gap | /dashboard |
Explicit limits
- Crypto / perps: fat tails — classical bootstrap may understate extremes.
- Regime shifts: a historical MC pass does not guarantee the future regime.
- Educational content; no profit promise.
TradingView
Proof ready? Open the chart review path.
Strategy Arena finds and validates the idea. TradingView is where builders review charts, Pine and alerts after proof.
Official partner path. Use it before signup or upgrade if your TradingView region shows the referral coupon.
Survivor
ArenaScript
Pine v5
TradingView