How Effective Are LLM Trading Agents?

AI is evolving rapidly across many areas of life, and trading is no exception. Large Language Models (LLMs) have increasingly been used as tools to assist financial analysts with research, data synthesis, and decision-making.

More recently, researchers and practitioners have moved beyond analyst assistance and begun exploring LLMs as autonomous end-to-end trading agents. As of this writing, even brokerage platforms such as Robinhood have introduced services that allow investors to trade using AI agents.

But how effective are these systems?

Reference [1] reviews recently proposed and developed end-to-end LLM trading agents and evaluates their performance and limitations. The authors pointed out,

The two-year arc of end-to-end LLM trading agents has been productive on its own terms: it forced concrete questions about how language models behave when asked to produce orderable decisions, and the resulting catalogue of architectures has real research value. The trouble starts when this exploratory progress is reported as deployment progress—headline Sharpe numbers from short, in-cutoff windows carried into abstracts, talks, and follow-up surveys as if they had already answered the deployment question.

Our argument has been narrow but specific. Public evidence cannot yet distinguish robust predictive ability from temporal contamination, unmodeled friction, short-sample noise, narrative fitting, and parametric prior; and even were every evaluation step cleaned up, three structural gaps would remain-language confidence is not tradable probability, narrative ability is not numerical execution, and model priors can become undisclosed implicit factor exposures. P1–P6 and the modular alternative are not a new framework but the minimum bookkeeping that lets a reader tell which kind of evidence a paper is offering—historical backtest, prototype, deployable claim, or autonomous trading ability—and lets reviewers hold each level of claim to its matching evidence floor.

So basically, the paper concludes that reported alpha from trading agents should not be interpreted as evidence of deployable trading capability. It shows evidence that reported performance can collapse once evaluation moves beyond the model’s knowledge cutoff.

The authors argue that current LLM trading results are confounded by three major issues:

• Temporal contamination (training-data leakage),

• Unmodeled trading frictions,

• Short-sample and multiple-testing effects.

They propose six minimum reporting protocols and recommend a modular architecture instead of end-to-end LLM trading,

• LLMs extract information from news, filings, and transcripts,

• Independent quantitative models perform forecasting,

• Separate modules handle calibration, risk management, sizing, and execution.

This paper provides a transparent, scientific assessment of the current state of LLM-based trading agents, separating evidence-based findings from the hype often found in popular media.

Let us know what you think in the comments below.

References

[1] Ye, Y., Han, J., Hu, A., Bu, J., Chen, Y., Wen, L., Mandic, D., Sun, D. D., Yinghui, X., & Xu, Z. (2026). The Alpha Illusion: Reported Alpha from LLM Trading Agents Should Not Be Treated as Deployment Evidence. arXiv:2605.16895v1, 16 May 2026.