AI Backtester
A desktop app that turns plain-English strategy ideas into sandboxed backtests — AI writes the code, deterministic engines do the math.
My own product — the same engine sold on the products page. Source private; architecture and flows shown.
Describe a trading strategy in plain English — or paste TradingView Pine Script — and AI Backtester turns it into a running backtest: a multi-agent AI pipeline parses the idea into a structured spec, generates the strategy code, executes it against historical market data and returns metrics, interactive charts and a written analysis. The same conversation handles data questions, parameter optimization, and requests like "download BTC/USDT 1h from Binance".
Running AI-generated code safely is the core engineering problem. Every generated script passes an AST-level safety gate — import whitelist, blocked builtins, no file access — before executing in an isolated subprocess with hard timeouts. Failures don't surface to the user: the error is fed back to the coder agent, which repairs its own code across up to five retries. And the numbers are never the AI's opinion — all metrics come from a deterministic analytics engine, with sanitizers between every layer.
The product is three systems: a desktop app (Tauri shell, React frontend), a local backend streaming results over server-sent events through sixteen agent modules, and a cloud licensing server that proxies every LLM call — the API key never ships to users — meters token usage server-side, binds licenses to hardware and handles subscriptions through Dodo Payments.
Validation goes beyond a single equity curve: 1,000-run Monte Carlo simulation with percentile bands, walk-forward analysis, monthly-returns heatmaps and side-by-side strategy comparison, over data from seven crypto exchanges, stocks and forex, or the user's own files.
System architecture
The modules that make it up, and how data flows between them.
Desktop shell · streaming chat UI
16 agents · parse → code → review
LLM proxy · metering · licenses
AST-checked isolated subprocess
Deterministic metrics (never the LLM)
Hover a module to trace what it feeds.
Request flow
One strategy idea, end to end — from English to audited metrics.
- 1
Intent routing
Strategy workBacktests, strategy builds and optimizations head into the AI pipeline.
Data analysisConversational questions about loaded data get their own analyst path.
Data download"Download X from Y" requests are parsed with pure regex — zero LLM tokens spent.
- 2
Spec extraction
The parser agent turns the description into a structured spec — indicators, entry and exit signals, cost model — so generation works from data, not prose.
- 3
Code generation
The coder agent writes the strategy as pandas Python against the spec.
- 4
AST safety gate
The generated code is parsed and checked against an import whitelist and blocked-builtin rules — anything suspicious never executes.
- 5
Sandboxed run
Execution happens in an isolated subprocess with a hard timeout; output is captured and sanitized.
- 6
Self-repair loop
Clean resultTrades and equity data flow on to analytics.
Error or zero tradesThe failure is fed back to the coder agent, which fixes its own code — up to five repair rounds before giving up gracefully.
- 7
Deterministic analytics
Returns, drawdown, win rate, profit factor, expectancy and rolling Sharpe are computed in plain Python and sanitized — the LLM never touches a number.
- 8
AI analysis
The analyst agent streams a written read of the results — what worked, what's fragile, what to try next.
- 9
Validation on demand
1,000-run Monte Carlo with percentile bands, walk-forward windows and side-by-side comparisons stress-test anything promising.
- 10
Everything persists
Strategies, generated code and results auto-save locally — every experiment is reproducible.
Engineering highlights
The details that keep it alive with real money on the line.
Sandboxing untrusted AI code
AST whitelist validation plus subprocess isolation plus hard timeouts — the answer to "what if the AI writes something dangerous" is layered, not hopeful.
Self-healing generation
Errors and zero-trade results loop back to the coder agent with full context — most failures repair themselves without the user noticing.
Deterministic math
Every metric is computed by code, never estimated by the model, with sanitizers defending against malformed AI output at each boundary.
Zero-client-secret design
All LLM calls route through the licensing server — users never hold the API key — with server-side token metering and hardware-bound licenses.
Token-free fast paths
Common requests like data downloads are handled by regex parsing — instant, free and deterministic.
Tested core
A pytest suite covers the safety validator, the sandbox runner and the data models — the security-critical paths have the tests.
Under the hood
The stack, and the system in numbers.