17 Aug 2026
Practical guide to improve your AI agents for alpha generation
With real examples from our deployed strategy
a practical walkthrough
End-to-end walkthrough on how to leverage fintel to evaluate and improve your own agents, using examples from our own strategy — F1 — that actually impacted our final returns.
How we run evals with fintel
Create a strategy pack
A folder that encapsulates your strategy: a mission (how the agent should think and trade), an output schema (what it emits), data (what it can access, point-in-time controlled), a universe, a schedule, and the KPIs (IR, Sharpe or other performance metrics you care about).
Hook your agent
Write one adapter — decide(environment) → AgentResponse — to hook your agent so fintel can feed the right data to the right sub-agents at the right time. It also enables monitoring of how your agent behaves, and surfaces tool errors, during runtime.
Run evals
During runtime, fintel fans out parallel runs (to control for agent stochasticity), trials (multiple tickers on each date), and cells (where the action happens). Within the environment, the agents can only access point-in-time controlled data and guidance. The platform then catches and saves the decisions and traces to designated folders.
Score and compare
fintel then evaluates the performance based on your specified KPIs.
Iterate one knob at a time
Generate iterative strategy packs or agents for separate evals by repeating steps 1–4. Compare across them to identify the optimal strategy or agent.
Examples
Three real iterations from our own runs — each one knob on the grid, scored on the same KPI, then kept or cut.
Finding the right model
what we wanted to explore
Most presume that a smarter model = better performance. We did too, until we systematically ran evals on our agents across different models, freezing our strategy. The most notable example is Grok 4.5 vs 4.3, where 4.3 consistently outperformed 4.5 despite a significantly lower intelligence score. We found the culprit in the end — hallucination rate — which tracked nicely with our results. Turns out, when AI is making investment decisions, getting things (especially numbers) less wrong is more important than fancy words.
How to read. Factor-neutralized residual IC for the same scaffold on two Grok checkpoints. Higher is better. Intelligence Index and Hallucination rate published by Artificial Analysis.
factor-neutralized residual IC
higher is better · h1Never assume a smarter model = better performance. Moreover, model and harness interact non-monotonically — Claude is not the same in OpenClaw vs Claude Code. The only way to truly be sure is to run ablation evals across other models. You could potentially save a lot of token costs on frontier models too!
Finding the right harness
what we wanted to explore
We held the model fixed and compared two harnesses: a tool-calling agent that picks its own tools each turn, and a minimal harness that is fed a point-in-time dossier in a single turn. We caught another surprising finding: sometimes, even across the harness axis, less is more. This was again attributed to hallucination/error rates — in this case the tool-calling capabilities and multi-turn reasoning amplify them; it is like giving bad drivers a manual sports car.
How to read. Factor-neutralized residual IC, same model, same data — only the harness differs.
factor-neutralized residual IC
higher is better · h1It is all about net capability gain — what a harness adds in capability minus what it loses to hallucination and its amplification. On the same model the minimal harness beats the tool-calling agent by more than 2× on our KPI, because open tool loops compound mistakes. In the end F1 went with a hybrid — feeding where the data is knowable ex-ante, tooling only where discovery and investigation are actually needed.
Fine-tuning your AI-native strategy
what we wanted to explore
Could a strategic prompt improve performance for our agents? We also ran evals, freezing our agents while tweaking the strategy — this time adding a small section called "Alpha View" that steers them away from sentiment-driven headlines. On the broad book, headline performance is similar while volatility and drawdowns fall; at the high-conviction threshold the alpha view pulls ahead on both return and stability — exactly how a fundamental-driven, less sentiment-reactive strategy should behave.
How to read. Net NAV (top) and underwater / drawdown from running peak (bottom) for the score-weighted long book, aligned on the overlapping decision window. Left: score-weighted long > 0.0; right: > 0.3. The stability gap widens at the higher-conviction threshold, since those names attract more sentiment-driven news (e.g. NVDA).
score-weighted long > 0.0
net of cost · aligned windowscore-weighted long > 0.3
net of cost · aligned windowStrategy iterations should be evaluated too; a "better" prompt can also cut both ways. Your eval KPIs should also capture dimensions beyond headline metrics — in particular, how variable the agents are in their behaviour and performance.
Evaluate, iterate, improve
There is no magic key to spin up a money-making AI trader on day 1. The findings here are the results of dozens of runs, millions of tokens, and years of research experience. But fintel offers the infrastructure and plumbing so that you can improve your AI-native strategy, one step at a time.