Test a Pine Strategy Out of Sample: Judge, Lab, PineForge, Backtrex, TradingView
To test a Pine strategy out of sample you need two things: a period the strategy has never seen, and a count of every trial made before opening it. The online judge does this for one script as it is, with no install. The Lab, free for Windows and Linux, does it for a search across variants. In TradingView, the cut is made by hand with a date switch.
Published on 2 November 2026. Third-party pages read the same day. Strategy Arena makes the judge and the Lab: this comparison comes from an interested party, and every cell about a third-party tool points to its source. Research on past data, not investment advice.
What an out-of-sample period changes
A result obtained on the bars used for tuning is a training score. Two cases published on this site:
- A champion found by a genetic algorithm on Bitcoin: +172.7% on its training period, −37% on data it had not seen (post-mortem, 16 August 2026).
- Turtle 20/10 on EUR/USD: +31.9% in training, −42.1% over 2013-2026 (article, 13 October 2026).
The period only works once. If you change a setting after seeing it, it becomes a training period.
What counting trials changes
The more variants you try on the same data, the better the best one looks, even with nothing real behind it. The measurement published on 12 October 2026 covers 350 automated research sessions on pure noise (details):
| Measurement on pure noise | Result |
|---|---|
| Sessions where the loop believes it improves | 88 to 98% |
| Deflated Sharpe above 95%, trials not counted | 64 to 86% |
| Deflated Sharpe above 95%, all trials counted | 0 to 1% |
| "Survives" verdict with counting and a sealed period | 0 of 350 (95% interval: 0 to 1.1%) |
The same measurement shows the cost: on a clear planted signal, the loop confirms it in only 8% of sessions (interval 4.1 to 15%), against 50% for the judge applied directly to the starting script. This protocol lets few false positives through, and it also misses real signals.
How to do it in the online judge
- Open the online judge and paste your Pine v5, or load the EMA 9/21 example.
- Choose the data: BTCUSDT in 1 d (full history), 4 h (3 years) or 1 h (1 year).
- The calculation runs in your browser. The code goes to the server only if you make it public on a card.
- Read the verdict: survives, mirage, inconclusive or not enough trades.
The last 20% is sealed. A rule can survive only with at least 10 trades on that period, a gain above buy-and-hold on the same bars, a deflated Sharpe of at least 95% and a significant SPA test (full method).
Limits. One market, BTCUSDT. One script, with no variant search: if you tried fifty before pasting the best one, the judge does not know. Pine outside the supported subset is refused with the reason, never tested in an approximate form.
How to do it in the Lab
- Install the Lab: Windows 10/11 x64 or Linux, NVIDIA GPU optional. The app, the MCP connection, backtests, sweeps, Pine export and the sealed proof are free.
- Import your Pine v5, or plug in your AI over MCP.
- Search on the training part. The end of the history stays sealed: 20% in the interface, 30% by default through MCP, adjustable from 20 to 50%. Every variant evaluated is counted.
- Run the proof. The verdict uses the trial count, and adds PBO and CPCV since there was a selection.
Free mode, the default, reports a period already opened or fewer than 30 independent events without blocking. Certified mode, the only one used for the public badge, applies them.
Limits. The Lab's code is not public; only the MCP connector is (MIT). Windows installers are not signed. The seal stops an AI from reading the candles, not from remembering market history. The comparison with TradingView is only checked on BINANCE:BTCUSDT 1D: on 1 November 2026, 916 / 916, 156 / 156 and 10 / 10 trades on the same dates for levels computed from position_avg_price, with the Pine supplied. Without the Pine supplied, the default replay can differ up to the sign of the result, and the Lab says so (notes for version 254).
Comparison: where it runs, which Pine, what price
| Tool | Where it runs | Pine accepted | MCP server | Displayed price | Open code |
|---|---|---|---|---|---|
| Online judge | Browser | v5, subset | No | Free | No (the Lab's engine) |
| Strategy Arena Lab 254 | PC, Windows and Linux | v5, subset | Yes, local | Testing features free | Connector only (MIT) |
| PineForge MCP | Local Docker, or hosted service | v6, subset | Yes | $0 for personal use; hosted capped at 100 calls per week per IP; commercial licence on request | Server and engine public; transpiler under its own licence |
| Backtrex | Web app | v5 detected on import | Not stated | €119 or €299 incl. tax, once; first backtest without a card | Not stated |
| backtester-mcp | Local, Python | Not stated: strategies are Python functions | Yes, 13 tools | Free | Yes, Apache 2.0 |
| TradingView Strategy Tester | Web, desktop, mobile | v6 in the documentation | Yes, in beta, from Essential; the published tool list has no backtest tool | Basic €0, 5,000 bars; paid plans from €12.95/month billed annually | Not stated |
Comparison: what protects against a false result
| Tool | Out-of-sample period | Trials and selection bias | Comparison with TradingView trades | Data |
|---|---|---|---|---|
| Online judge | Sealed, 20% | Deflated Sharpe, SPA; one script, no search | No | Binance BTCUSDT, frozen |
| Strategy Arena Lab 254 | Sealed on disk before computing; walk-forward as a separate mode | Trials counted; deflated Sharpe, PBO, SPA, CPCV | Yes, checked on BINANCE:BTCUSDT 1D, see the limits |
On your machine |
| PineForge MCP | Not stated | Parameter grid; counting and correction not stated | Yes, its speciality: 7,989 graded probes, 7,983 excellent, 6 strong, baseline dated 31 October 2026 | Public Binance or your CSVs; hosted: crypto, last 365 days |
| Backtrex | Not stated | Not stated | "Less than 2% divergence. Guaranteed."; measurement method not stated | Dukascopy, 17 assets |
| backtester-mcp | Walk-forward; seal not stated | Deflated Sharpe, perturbation PBO, bootstrap interval | Not stated | Your CSV or Parquet files, any price series |
| TradingView Strategy Tester | Manual; Deep Backtesting lets you select a date range | Sharpe and Sortino shown; counting and correction not stated | It is the reference | The platform's own |
"Not stated" means absent from the pages read, not absent from the product.
What these tools do better
- PineForge MCP accepts Pine v6, publishes an open corpus of 312 reference strategies with their trade lists, and gives its parity a figure and a written method. On parity, its published evidence is far wider than ours. That figure comes from the publisher itself, with its date and its thresholds.
- Backtrex needs no code, covers forex, indices, crypto and commodities, and claims 61 indicators.
- backtester-mcp installs with
pip, has all its code open and accepts any price series. It also simulates three execution-cost scenarios. Its only release on PyPI is 0.1.0, dated 14 April 2026. - TradingView remains where Pine is written and run, with the strategy report on the chart, real-time tracking and Pine v6.
FAQ
How can I test a Pine strategy out of sample without installing anything?
Paste a Pine v5 script into Strategy Arena's online judge. It replays the script in your browser on frozen BTCUSDT data, seals the last 20% and returns a verdict. It judges the script as it is and searches no variant.
Why count trials if the test period is already sealed?
The sealed period protects the final verdict. The trial count protects the statistical measure: the best of a thousand trials looks good by construction. On pure noise, without counting, the deflated Sharpe exceeded 95% in 64 to 86% of sessions; with counting, in 0 to 1%.
Does the Lab replace TradingView's Strategy Tester?
No. Pine is written in TradingView, which stays the reference for its own results. The Lab is an independent test: sealed period, counted trials, verdict. It accepts supported Pine v5 and refuses the rest.
Which tool accepts Pine v6?
TradingView and PineForge. The Lab and the online judge accept supported Pine v5. Backtrex says it detects Pine v5 on import; v6 is not stated on the pages read.
Sources
All read on 2 November 2026.
- PineForge: GitHub repository, parity methodology, pricing.
- Backtrex: home page, pricing, comparison published by Backtrex.
- backtester-mcp: PyPI package (read through its JSON API, as the web page asks for a browser check), GitHub repository.
- TradingView: pricing, Pine documentation on strategies, Deep Backtesting, MCP server.
- Strategy Arena: judge methodology, MCP guide, Lab 254, what the judge answered to old rules.
Strategy Arena takes part in the TradingView partner program. No link in this article is a partner link. No bot on this site places real orders. Research on past data, not investment advice. Past results do not guarantee future results.
⚠️ Disclaimer — This article is for informational and educational purposes only. It does not constitute investment advice or a buy/sell recommendation. Past performance does not guarantee future results. Strategy Arena is an educational simulator with virtual capital. Always do your own research before making investment decisions.