Skip to main content
Edition of 2026-11-17 · Lab 0.1.0-266 · measured from the Lab’s logs

How AI models search for a strategy: four Lab sessions, measured

On 17 November 2026, four AI models drove the Strategy Arena Lab 0.1.0-266 live, a free desktop app controlled over MCP. Same base brief for all: look for a strategy that holds on a sealed period, declare the hypothesis before any trial, keep a log. This page publishes what the Lab recorded, call by call: how many calls, which tools, how many trials, how many sealed periods opened, which verdicts. Research on past data, never advice.

Plug your AI into the Lab and run the same experiment

Free desktop app for Windows and Linux. Its local MCP server works with Claude, Codex, Cursor, LM Studio or opencode: give your AI the same brief and compare its log with these ones. Research on past data; nothing is linked to a real account.

Get the free Lab → No AI at hand? Judge a Pine script in the browser →

Read this before the numbers

What the Lab gives an AI that searches

A dated hypothesis before the first trial

All four sessions declared their idea with discovery before running any search. The Lab timestamps that declaration and then counts every setting tried: the AI cannot rewrite its starting point.

Dropping an idea without burning the sealed period

The diagnosis (fees, entry timing, sub-periods, concentration of gains) and the power threshold shown by the board are often enough to drop an idea on training data alone. Two sessions stopped all their leads without opening a seal; another set about a dozen aside the same way.

A judge that remembers who looked

A US sector rotation beat its reference on the test period, and the Lab still answered “inconclusive”: that period had already been opened 5 times before. The Lab counts openings and does not let a result seen several times become a proof (report SAP-281cfd6ddd13).

A three-word verdict and a replayable report

Holds, mirage or inconclusive, with reasons and sealed-period figures, in an SAP-… report that can be read and replayed.

The sessions, in chronological order

Model announced as Astra 6.2 · client Codex

BTC 4h, a rebound idea after a low RSI

21:42–21:47 UTC

25MCP calls
5.4 minduration
4.6calls per minute
8.0 %failed calls (2)
8different tools
yeshypothesis before the first trial (1 declared)
64settings evaluated by search
1searches stopped without opening their seal
0sealed periods opened
—verdicts

Search style observed. Short and frugal. One hypothesis, a 64-trial cap set in advance, a search of about one second, then a careful reading of the diagnosis: the edge vanished from 10 basis points of fees and entry timing was nothing unusual. Decision: stop on training data, without opening the sealed period, using two stop criteria announced before the trial.

Outcome. No verdict requested: idea stopped before the seal (hypothesis SAH-64faecbe2f04).

Tools used

discovery 6 · list_sandbox_datasets 5 · board 4 · read_local_audit_events 3 · diagnose 3 · move 2 · start_here 1 · read_proof_report 1

What the Lab learned from this session.

Model announced as Grok 4.7 · client Grok

ETH 4h, a trend rule (close above its moving average)

21:59–22:04 UTC

20MCP calls
5.0 minduration
4.0calls per minute
0.0 %failed calls (0)
7different tools
yeshypothesis before the first trial (1 declared)
362settings evaluated by search
0searches stopped without opening their seal
1sealed periods opened
1 inconclusiveverdicts

Search style observed. Pick untouched ground, then a single opening. Before any trial, the session looked at the state of several markets and set aside those that already had counted trials or an opened seal. It declared three structures, ran one search, then opened a single sealed period and stopped on the verdict.

Outcome. Inconclusive (SAP-d1b53a7bfb31): on the sealed period, +85.1% over 126 trades vs +113.7% for buy-and-hold, with a maximum drawdown of 41% vs 65%.

Tools used

board 7 · list_sandbox_datasets 5 · move 3 · discovery 2 · start_here 1 · gym 1 · promote 1

What the Lab learned from this session.

Model announced as Gemini Hydrogen · client Cursor

Broad: portfolios (rotations, trend baskets), indices, currencies, crypto, commodities

22:13–22:36 UTC

435MCP calls
23.1 minduration
18.8calls per minute
6.4 %failed calls (28)
26different tools
yeshypothesis before the first trial (10 declared)
7630settings evaluated by search · 56 single backtests
12searches stopped without opening their seal
12sealed periods opened
10 mirage, 2 inconclusiveverdicts

11 of 22 searches with trials ran without a recorded hypothesis, mostly right after a refused declaration.

Search style observed. Broad exploration, in bursts. The session looked at more than a hundred games across dozens of markets, often several calls within the same second, with about twenty different tools. It stopped about a dozen searches without opening their seal (oil, silver, BTC 4h, BNB 4h…), two of them after reading the diagnosis, and opened sealed periods for its portfolios and several scripts. The brief suggested at most 3 openings; nothing in the Lab reminded it of that budget.

Outcome. No “holds”. The US sector rotation (SAP-281cfd6ddd13) beat its reference on the test period, but stays “inconclusive” because that period had already been opened 5 times. The other verdicts are “mirage” (for example SAP-0831a01997ad, SAP-48b84150fec4) or “inconclusive” (SAP-40d9730c6722).

Tools used

board 185 · run_sandbox_backtest 58 · compose_portfolio 54 · get_sandbox_capabilities 26 · discovery 25 · move 25 · list_sandbox_datasets 22 · diagnose 9 · promote 4 · search 4 · prove_sandbox_script 4 · market_catalog 3 · query_strategy_base 2 · suggest_lab_capability 2 · start_here 1 · get_lab_state 1 · get_lab_plan 1 · list_sandbox_experiments 1 · list_local_lab_reports 1 · inspect_local_strategy_progression 1 · read_locked_ai_search_trial_register 1 · list_strategy_atoms 1 · get_lab_capabilities 1 · read_proof_report 1 · explore_lab 1 · suggest_lab_experiments 1

What the Lab learned from this session.

Model announced as Muse Wonder · client Cursor

Daily crypto (ADA, XRP…) after a wide look at the board

22:46–22:48 UTC

Different brief: a budget of 100 calls, no parallel calls.

147MCP calls
2.3 minduration
63.0calls per minute
3.4 %failed calls (5)
14different tools
yeshypothesis before the first trial (2 declared)
2640settings evaluated by search
2searches stopped without opening their seal
0sealed periods opened
—verdicts

Search style observed. Scouting first, then two short leads. Board reads (`board`) dominate, interleaved with the searches: two hypotheses declared on daily ADA and XRP, searches of more than a thousand settings each, one reading of the diagnosis, then a stop. The board gave it the size of edge the sealed period can detect on these markets (a Sharpe of at least 2.24): with that yardstick, opening the seal made no sense, and no sealed period was opened.

Outcome. No verdict requested: two leads stopped before the seal (hypotheses SAH-1df8cb394ca4 and SAH-7bcd0a4664e3).

Tools used

board 114 · get_sandbox_capabilities 8 · discovery 6 · list_sandbox_datasets 4 · move 4 · get_lab_state 2 · query_strategy_base 2 · explore_lab 1 · get_lab_capabilities 1 · start_here 1 · get_lab_plan 1 · suggest_lab_experiments 1 · diagnose 1 · validate_sandbox_strategy 1

What the Lab learned from this session.

1 call in the log carried no agent label and is not counted in any session.

Campaign results

One line per proof judged by the Lab during these sessions, built only from the Lab’s proof reports. Results only: no price or return series is published. Sealed period = the last part of the history, kept out of the search until the verdict.

campagnes JSON · CSV · CC BY-NC-SA 4.0 (https://creativecommons.org/licenses/by-nc-sa/4.0/)

ReportModelMarketRuleSealed periodRule / referenceMax DDPBODSRNVerdict
SAP-d1b53a7bfb31Grok 4.7ETH 4hclose_above_sma signal2023-05-17 → 2025-10-31+85.1 % / +113.7 %41.4 %51.3 %87.0 %28inconclusive
below the reference on the sealed period
SAP-0e29b611b467Gemini HydrogenAUDUSD + GBPUSD + USDCAD + USDCHF + USDJPY 1dtrend basket2010-01-05 → 2026-10-02+3.8 % / +4.1 %11.4 %49.4 %49.5 %12mirage
deflated Sharpe below 50%
SAP-11a68fc4db69Gemini HydrogenSOL + ETH + BNB + XRP 1hrotation (momentum, top N)2025-11-15 → 2026-10-09-50.3 % / -24.8 %63.4 %63.1 %26.0 %525mirage
PBO of at least 50% (unstable choice between settings)
SAP-ebbf5752c2f5Gemini HydrogenLTC 1dclose_above_sma timeout_short2023-06-21 → 2025-10-31-68.4 % / +11.9 %76.2 %87.4 %16.6 %80mirage
PBO of at least 50% (unstable choice between settings)
SAP-163858f57a9eGemini HydrogenXLM 1dclose_above_sma timeout_short2023-08-11 → 2025-10-31+71.2 % / +117.9 %61.0 %59.6 %3.8 %678mirage
PBO of at least 50% (unstable choice between settings)
SAP-48b84150fec4Gemini HydrogenETH 1dETH Trend ADX2023-05-17 → 2025-10-31+2.9 % / +110.8 %63.0 %93.1 %73.8 %36mirage
PBO of at least 50% (unstable choice between settings)
SAP-673c1ef71252Gemini HydrogenAPT 1dAPT BB MeanRev LS2025-11-16 → 2026-10-09-31.2 % / -71.3 %44.7 %37.3 %48.5 %15mirage
deflated Sharpe below 50%

Proofs left out of the file. Their verdict is shown, not their figures, because the licence of the source data could not be confirmed:

Searches stopped without opening their sealed period.

ModelMarketSettings evaluatedDiagnosis read
Astra 6.2BTC 4h64yes: 3 alert(s), entry timing at percentile 60
Gemini HydrogenSILVER 4h355no
Gemini HydrogenBTCUSDT 1h380no
Gemini HydrogenOIL 4h348no
Gemini HydrogenBNB 4h709no
Gemini Hydrogennot stated in the log1353yes: 1 alert(s), entry timing at percentile 79
Gemini HydrogenOIL 1d212yes: 1 alert(s), entry timing at percentile 94
Gemini HydrogenETH 1d217no
Gemini Hydrogennot stated in the log1975no
Gemini HydrogenBTC 4h288no
Gemini HydrogenSILVER 1d221no
Gemini Hydrogennot stated in the log335no
Gemini Hydrogennot stated in the log1179no
Muse WonderADA 1d1514yes: 1 alert(s), entry timing at percentile 98
Muse WonderXRP 1d1126no
Columns of the file (schema)

Descriptions are in French in the file; column names are stable from one edition to the next.

The protocol, to redo at every Lab version

The common brief.

  1. Start with start_here and follow its path, without reading the code.
  2. Look for a strategy that holds on the sealed period, with your own plan: a single market or a portfolio.
  3. Declare the hypothesis before any trial (discovery), then move, diagnose and promote as needed.
  4. Budget: about one hour, or at most 3 sealed periods opened.
  5. Keep a log and write a report: exact verdicts, hesitations, whether each opening was justified, what would have saved time. No investment advice.

What is measured.

When. At every new published version of the Lab, with the same brief. Each edition is added to this page; earlier ones are not rewritten.

Figures are recomputed from the Lab’s logs by a published script: mesurer_pilotage.py. Same Lab, other public follow-ups: the registry of frozen rules and the AI researcher leaderboard.

Machine-readable version for AI assistants: mesures.json.

Plug your AI into the Lab and run the same experiment

Free desktop app for Windows and Linux. Its local MCP server works with Claude, Codex, Cursor, LM Studio or opencode: give your AI the same brief and compare its log with these ones. Research on past data; nothing is linked to a real account.

Get the free Lab → No AI at hand? Judge a Pine script in the browser →

To compare AI models on other uses: ScoreIA.

Research on past data, never advice. One session per model, different conditions: no ranking. Measured on 2026-11-17 22:52 UTC.