Methodology: the Lab's judge, then the arena

How does the Lab's judge decide?

The judge cuts the history in time order, chooses on the training part only, then opens the sealed period. A rule "survives" only if it gains on that period, beats buy-and-hold on the same bars, has at least 10 trades and passes the overfitting controls below. Otherwise the answer is mirage, inconclusive or not enough trades. The judge describes past data: it is neither a forecast nor advice.

Last verified: 2 November 2026, against the code of version 0.1.0-254.

How much of the history is sealed?

The last 20% in the Lab interface and the online judge. The last 30% by default when an AI goes through MCP, adjustable from 20% to 50%.

Which controls decide the verdict?

An expected control that could not be computed also blocks "survives".

Is a single rule judged the same way?

No. A script judged as it is, with no sweep, as in the online judge, has no selection to check: PBO and CPCV do not apply. The verdict rests on activity, the sealed period, the Deflated Sharpe and the SPA.

What changes in certified mode?

Free mode is the default: a period already opened, fewer than 30 independent events and cumulated trials are reported without blocking the verdict, and the report states its mode. Certified mode, the only one used for the public badge, applies those rules: a period already opened by the same rule family can no longer give "survives", and the threshold of a fixed rule becomes 10% / [n(n+1)] for the n-th rule.

What informs without deciding?

• Monte Carlo: 10,000 draws bootstrapping the account's exposure episodes on the sealed period.
• StepM by Romano and Wolf: which settings of the universe beat buy-and-hold on training once all are counted.
• Leak check: decisions of the sealed period are replayed on data cut at the decision's bar.

Walk-forward is a separate proof mode (anchored folds), requested explicitly; it is not added to an ordinary proof.

What about labels and limits?

• The Lab interface and the Survival Audit report show KEEP, REJECT and INCONCLUSIVE for survives, mirage and inconclusive.
• The seal stops an AI from reading the candles, not from remembering market history learned in training.
• A script outside supported Pine v5 is refused with the reason, never tested in an approximate form.
• The judge has had defects, published: SPA false positive fixed (11 October 2026).

The judge in practice: what it answered to old rules, and how to test a Pine script out of sample, tools compared.

Which references does it build on?

• Bailey, D. H., Borwein, J., López de Prado, M., Zhu, Q. J. (2016). « The Probability of Backtest Overfitting ». The Journal of Computational Finance. doi.org/10.21314/JCF.2016.322
• Bailey, D. H., López de Prado, M. (2014). « The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality ». The Journal of Portfolio Management, 40(5), 94-107. doi.org/10.3905/jpm.2014.40.5.094
• White, H. (2000). « A Reality Check for Data Snooping ». Econometrica, 68(5), 1097-1126. doi.org/10.1111/1468-0262.00152
• Hansen, P. R. (2005). « A Test for Superior Predictive Ability ». Journal of Business & Economic Statistics, 23(4), 365-380. doi.org/10.1198/073500105000000063
• Politis, D. N., Romano, J. P. (1994). « The Stationary Bootstrap ». Journal of the American Statistical Association, 89(428), 1303-1313. doi.org/10.1080/01621459.1994.10476870
• Politis, D. N., White, H. (2004). « Automatic Block-Length Selection for the Dependent Bootstrap ». Econometric Reviews, 23(1), 53-70. doi.org/10.1081/ETC-120028836
• Patton, A., Politis, D. N., White, H. (2009). « Correction to “Automatic Block-Length Selection for the Dependent Bootstrap” ». Econometric Reviews, 28(4), 372-375. doi.org/10.1080/07474930802459016
• Romano, J. P., Wolf, M. (2005). « Stepwise Multiple Testing as Formalized Data Snooping ». Econometrica, 73(4), 1237-1282. doi.org/10.1111/j.1468-0262.2005.00615.x
• López de Prado, M. (2018). « Advances in Financial Machine Learning (ch. 12, CPCV) ». Wiley, ISBN 978-1-119-48208-6. www.wiley-vch.de/en/areas-interest/finance-economics-law/advances-in-financial-machine-learning-978-1-119-48208-6

Part two: the public arena and its ML layers

The rest of this page covers the arena in simulation, not the Lab's judge. Its figures are dated observations.

The 4 active monsters + 1 archived finding

Brier 0.25: what it means

The methodology we use to validate arena strategies

• Fees included: 0.20% round-trip.
• Single-split validations are treated as weak until they survive MC CV.
• Example: Wyckoff Evolved had OOS Sharpe 1.85 on a single split, then MC mean Sharpe 0.73 on PUMP, -0.04 on INJ, -0.36 on FLOKI. We rejected it.

Strategies validated by Monte Carlo

Data leaks we fixed

Measured consequence: Leviathan NN's Brier moved from 0.244 with leakage to 0.2589 without leakage. We publish the leak-free number.

Why some "AI strategies" are not real ML

• Leviathan 9-Layer Ensemble Brain used to be 8 heuristic layers and storytelling. After the graft, it is a 9-Layer Ensemble: 8 heuristics + 1 PyTorch MLP.
• The old Chimera total was an exaggerated count from a live-accumulated brain JSON. We now display 50 patterns kept by the 2026-05-14 mining run: Benjamini-Hochberg filter (5% FDR), 5 walk-forward folds and 1,000 permutations. The 17 names shown by the scanner (SERPENT_COIL, STEEL_WALL…) are descriptive labels of the market state, not validated patterns.
• ML Arena V3 used to be isolated from the main monsters. The models were migrated in-place into backend/: chimera_cnn.py, leviathan_nn.py, hydra_lstm.py.

What we are not claiming

Newsjacker editorial process

Newsjacker connects one financial, crypto or AI news source to one precise Strategy Arena finding each day. It is assisted editorial infrastructure, not a marketing-content machine: original source link required, internal link to a measured finding required, caveat required, and owner review queue by default.

Distributed Research Network

Strategy Arena is moving toward a quantitative citizen-science layer: local personal backtests, public hardware benchmarks, then cooperative themed raids. The important constraint: every raid must start from a pre-registered hypothesis, with quorum, replication and publication of positive and negative results.

Known limits

• Brier ~0.25 on 5m crypto: practical ceiling, not standalone directional edge.
• Monte Carlo CV does not guarantee future live performance.
• Metals feeds: MC re-validation after Yahoo GC=F/SI=F migration.
• Heterogeneous AI layers (heuristics + ML) explicitly labeled.
• /facts/*.json snapshots = VPS state at generation, not HFT feed.

A test sealed on future data

On 29 September 2026 we sealed one rule, 25 markets and the pass mark before the data existed. Decision on 1 October 2027, or 2028 at the latest; the result will be published whatever it is. See the forward test

See also: the AI researcher leaderboard · plug your AI into the Lab: counted attempts, sealed period, one proof

This page validates claims on: