Top 3 LLM+RL Advances in Equity Trading (2025)

            

           In 2025, a handful of practical studies emerged on combining Large Language Models (LLMs) with reinforcement learning (RL) for systematic equity trading. These papers stand out for their robust methodologies, rigorous backtesting, and focus on real-world applicability. Below we review three top papers — each backed by credible authors (including experienced quant researchers or fintech collaborations) — summarizing their contributions, methodology, results, and practical strengths/limitations. Recent works help R&D teams and firms focused on market data analysis derive new alpha, manage risk in trading and creating best portfolios for systematic and discretionary trading.

1. FinRL-DeepSeek: LLM-Infused Risk-Sensitive RL (Benhenda, 2025)

• Methodology & Contribution: Introduces a hybrid trading agent that augments deep RL with LLM-derived signals from financial news for both trade decisions and risk management. It extends a risk-aware RL algorithm (CVaR-Proximal Policy Optimization, CPPO) by feeding in an LLM’s news-based stock recommendations and risk assessment score each day. This goes beyond simple sentiment analysis by using carefully prompted LLMs (e.g. DeepSeek V3, Qwen-2.5, Llama 3.3) to extract nuanced risk/return insights from news. The result is an RL agent that adapts its actions based on news-informed risk signals, aiming to manage downside risk in volatile markets.
• Backtesting & Results: The study backtests the agent on the Nasdaq-100 index over decades of data (1999–2023 news from the FNSPID dataset). Code and data are open-sourced for transparency. In tests, the LLM-enhanced RL (especially the risk-sensitive CPPO variant) showed improved performance over baseline RL and even outperformed the Nasdaq-100 benchmark in certain runs. Notably, the CPPO+DeepSeek agent excelled in bear markets (e.g. post-2021 downturn), where it outperformed a vanilla RL agent that struggled with high volatility. This suggests the news-informed risk signals helped the agent avoid losses during market stress. (By contrast, a standard PPO agent without risk adjustment was too volatile.) The author reports that the RL agent with LLM inputs achieved higher risk-adjusted returns (e.g. better Information Ratio and tail-risk metrics) when calibrated appropriately.
• Credibility & Practical Strengths: This paper is by Mostapha Benhenda (affiliated with LAGA, a math/CS research lab) – an active contributor to AI-for-Finance projects. The approach is practical: it uses readily available news data and existing large models via API, rather than hypothetical data. The inclusion of risk management (CVaR) makes it appealing for real trading, as it targets drawdowns and not just returns. The fact that code and agents are released adds credibility and allows quants to replicate or build on the work. It also formed the basis of a 2025 FinRL trading contest, underscoring industry interest in the approach.
• Limitations: While promising, the results indicate that LLM signals must be integrated carefully. In the study, simply injecting strong LLM-based biases into a plain PPO agent actually hurt performance (the agent overreacted to news noise). Only when combined with the CVaR-PPO risk-aware framework did the LLM signals consistently add value. This highlights that real-world use would require careful calibration of LLM influence (the paper shows performance sensitivity to the “infusion strength” of LLM signals. Also, the author notes that results vary between bull and bear regimes, so this approach may need further tuning or regime detection to be reliably profitable across cycles. Overall, FinRL-DeepSeek demonstrates a credible and actionable integration of news-reading LLMs into an RL trading strategy, with an emphasis on risk-adjusted performance.

2. FLAG-Trader: Fusion LLM Agent with Gradient RL (Xiong et al., 2025)

• Methodology & Contribution: FLAG-Trader is a large collaborative effort (12 authors from academia and a fintech lab) that proposes a unified architecture fusing an LLM with deep RL for trading decisions. Here, a pre-trained language model is partially fine-tuned on financial data (to imbue it with domain knowledge) and then used as the policy network in an RL agent. In other words, the LLM itself learns to output trading actions (buy/sell/hold) in response to state prompts, and its parameters are further optimized via policy-gradient RL (PPO) using trading rewards. This approach leverages the LLM’s broad knowledge (financial reasoning, context understanding) while training it to achieve specific trading goals, effectively marrying pattern recognition with goal-driven learning. The team uses parameter-efficient fine-tuning so that even a relatively small model can adapt to market specifics without massive compute costs.
• Backtesting & Results: The authors conducted extensive empirical tests to validate that this LLM+RL fusion improves trading performance. They evaluated multiple model sizes and types (from a tiny 135M-parameter custom model up to 70B Llama and even OpenAI’s GPT-4 as baselines) on historical market data, measuring metrics like cumulative return (CR), Sharpe ratio (SR), volatility, and max drawdown. A key finding is that their RL-fine-tuned small model (135M) actually outperformed much larger off-the-shelf models (like GPT-4) on trading metrics such as total return and Sharpe. For example, after training, the 135M agent achieved higher risk-adjusted returns than even a 175B GPT-3.5 when the latter was used naively for trading signals. This highlights the benefit of domain-specific RL fine-tuning – the LLM agent learned to make coherent multi-step trading decisions, beating models that, while powerful in general knowledge, weren’t specialized to the task. The paper reports that FLAG-Trader’s agent surpassed a buy-and-hold baseline and other benchmarks in their tests, and even showed improved generalization to other financial tasks (like question-answering) as a byproduct of the training. These results were demonstrated on standard market environments (using the FinRL framework) to ensure reproducibility and rigor.
• Credibility & Team: The research was carried out by a cross-disciplinary team from institutions like Harvard, Columbia, Stevens Institute, University of Manchester, NVIDIA, and TheFinAI (a fintech AI research group). Notably, Dr. Xiao-Yang Liu — known for the FinRL library and prior RL trading research — is a co-author, lending credence to the implementation quality. This blend of academic and industry perspectives helped ensure the approach wasn’t just theoretically sound but also practically oriented. The “extensive empirical evidence” and comparisons to many baseline models demonstrate a high level of rigor.
• Practical Strengths: FLAG-Trader stands out for showing a viable path to use LLMs directly as trading agents, which could potentially shorten development time for new strategies (since the LLM can incorporate knowledge from financial texts, analyst reports, etc. out of the box). The fact that a smaller, open-source modelfine-tuned with their RL method can outperform larger proprietary models is encouraging for real-world use – it implies one can achieve strong performance without relying on black-box models or exorbitant computing resources. This makes the approach attractive for firms that want control over the model. Moreover, by focusing on risk-adjusted metrics (Sharpe) and testing against a buy-hold benchmark, the paper addresses what matters to quant traders (not just raw profits, but consistency).
• Limitations: On the flip side, integrating an LLM into an RL loop is computationally intensive. The authors acknowledge challenges with training stability and the non-stationarity of markets; the LLM policy may need continual retraining as market regimes change. There’s also the question of latency — large models can be slow, though the success with a 135M model suggests a practical trade-off can be found. Another consideration is interpretability: while the LLM can explain its actions in plain text (a nice bonus), its decision-making after RL fine-tuning might become am opaque mix of learned patterns. The paper notes that future work should explore techniques like continual learning to keep the LLM agent adaptive in live markets. In summary, FLAG-Trader provides a robust, tested framework that a quant team could experiment with, especially if they have access to domain-specific LLMs – it balances cutting-edge AI with awareness of trading metrics, making it one of the most practically credible LLM+RL studies of 2025.

3. LLM-Guided Stock Trading via RL (Stock-Evol-Instruct) (Riahi-Samani et al., 2025)

• Methodology & Contribution: This work (submitted to ICLR 2025) takes a holistic, pipeline-driven approach to integrating LLMs with reinforcement learning for stock trading. The authors propose an architecture that uses multiple LLMs in various roles to inform an RL trading agent. They introduce an algorithm called “Stock-Evol-Instruct”, which generates dynamic trading instructions or prompts for the RL agent based on market data and news. In practice, they experimented with six different LLMs (both general models and finance-specific ones) and developed a procedure where the LLMs analyze daily stock news and indicators, then produce trading guidance that the RL agent uses to adjust its strategy. The RL component is implemented with Deep Q-Network variants (DQN/DDQN) to make daily trading decisions, while the LLM-generated insights serve as an additional input or modulator for the agent’s state/reward (a form of LLM-informed policy fine-tuning). Uniquely, Stock-Evol-Instruct continuously updates the instruction prompts using recent market trends and the agent’s performance, creating a feedback loop between the LLM analysis and the RL policy learning. They also fine-tuned two smaller open-source language models (Mistral-7B and LLaMA-3B) on the generated instruction data, effectively turning them into specialized trading agents able to act directly on market observations. This two-stage setup (LLMs guiding an RL agent, and in turn improving the LLMs) is a novel contribution, aiming to make the RL more sample-efficient by leveraging rich textual data that pure price-based models might miss.
• Backtesting & Performance: The authors evaluated their system on real daily stock data for two instruments– the iShares Silver Trust (SLV, a silver ETF) and JPMorgan Chase & Co. stock – over a multi-year period. The choice of two very different assets (one commodity-based, one banking equity) was to test generality. Results indicate that the LLM-guided approach beat several conventional trading models and baselines. Notably, the RL agent augmented with LLM insights achieved higher prediction accuracy for trading signals and improved profitability compared to an RL agent without LLM guidance. For example, they report higher F1-scores in predicting profitable trades (over 81% in the case of the LLaMA-based agent on JPM) and better cumulative returns than baseline strategies. The LLM-informed agent outperformed a standard DQN trader and even a purely LLM-driven strategy, highlighting the synergy of combining them. The paper’s abstract notes a “significant potential to outperform conventional trading models”– indeed, in backtests the integrated approach had better Sharpe ratios and cumulative returns than a classical price-only RL and a buy-and-hold benchmark. These experiments, while limited to two assets, provide a proof-of-concept that blending news understanding (via LLMs) with technical decision-making (via RL) can yield a more powerful trading strategy.
• Authors & Credibility: The team (Ali Riahi Samani, Fatemeh Darvishvand, and Feng Chen) comes from a data science research background and submitted this work to a top-tier AI conference (ICLR), indicating it underwent rigorous peer review. While not directly from a bank or hedge fund, the authors demonstrate solid understanding of both NLP and trading domains. The paper references real-world entities (like using JPM stock data and citing industry data sources), showing an intent to solve practical problems rather than toy examples. The empirical approach – using actual market data and incorporating fundamental news – reflects a quant mindset. Additionally, by integrating numerous prior studies (86 references, as noted in an SSRN review), the authors built on a broad base of knowledge. This gives the work credibility despite the academic origin, as it clearly targets applicable techniques.
• Practical Strengths: One strength of this approach is its comprehensiveness: it doesn’t rely on a single model or data stream but rather combines technical signals with fundamental context via natural language. For a quant team, this is appealing because it resembles how a human analyst might operate (reading news and charts together) – here that process is automated. The introduction of a human-in-the-loop style instruction generation (Stock-Evol-Instruct) is also practically interesting: it can be seen as an offline training step that produces high-quality scenarios for the RL agent to learn from, potentially improving stability. The paper also demonstrated that even relatively small models, when fine-tuned properly on financial text, can add significant value to an RL trader. This suggests a path to deployment that doesn’t require heavyweight infrastructure. Moreover, the use of distinct LLMs for different subtasks (they mention experimenting with closed-source vs open-source LLMs, presumably ChatGPT, BloombergGPT, etc.) means the framework is flexible – a firm could plug in whatever language model (or ensemble of models) they trust to get the best insights and then use RL to translate those insights into trades.
• Limitations: Despite its innovation, this study is somewhat complex and not fully proven at scale. The backtests were on two assets; it remains to be seen if the approach scales to a broad portfolio or intraday trading (the paper focused on daily decisions). The heavy use of multiple LLMs and a custom instruction generator means the strategy could be hard to reproduce without significant NLP expertise. There is also the challenge of timeliness and overfitting: news-based signals can decay quickly, and an RL agent trained on past news might not generalize if the market regime or news tone shifts (a risk the authors acknowledge). Additionally, the need to fine-tune LLMs on finance data and then integrate them adds latency – in live trading, one might prefer the LLM analysis to be faster or even streaming. From an implementation standpoint, maintaining two learning loops (one for the LLM instructions and one for the trading agent) is resource intensive. The authors propose it as a frontier framework, and a quant team considering it would likely simplify or streamline parts of it. In summary, this paper demonstrates a powerful idea – that LLMs can actively coach an RL trader – and provides evidence of enhanced performance on real stocks. Its practical viability will depend on careful engineering, but it offers a roadmap for those looking to combine textual data and reinforcement learning in a systematic strategy.

Conclusion: 

In 2025 the fusion of Large Language Models with reinforcement learning moved from theory to credible practice in systematic equity trading, where three clear currents now dominate: first, hybrid frameworks that feed RL agents rich context from financial news and analyst reports, bridging structured price data with unstructured text; second, the success of compact, domain-fine-tuned LLMs—often outperforming huge general models once their policies are optimized for risk-aware objectives; and third, a shift from chasing raw alpha to maximizing risk-adjusted metrics such as Sharpe, CVaR and drawdown resilience. Early studies show that LLM-infused RL agents can outpace vanilla RL systems and even passive benchmarks, especially in volatile regimes, proving the commercial viability of this approach. The beneficiaries span quantitative hedge funds, prop desks, AI research groups, and—critically—vendors that aggregate, cleanse, and annotate market news or alternative data, because high-quality, machine-ready information is the fuel that makes these next-generation trading engines run.