Discussion of quantitative methods in finance tends to centre on models: which architecture, which training approach, which features. That is the interesting part and rarely the limiting one.
The limiting factor is almost always the data. A model trained on a dataset containing information that was not available at the time it purports to describe will produce excellent historical results and nothing useful afterwards, regardless of how well specified it is.
This has been understood in quantitative finance for decades. What has changed recently is where the contamination can hide.
The Traditional Data Problems
Any AI investment strategy using historical market data inherits a well-catalogued set of issues, all of which inflate backtested performance in the same direction.
The main ones:
- Survivorship bias, where delisted and failed companies are absent from the dataset
- Look-ahead bias, where information enters the record before it was publicly known
- Restatement contamination, where financial figures reflect later corrections rather than what was originally reported
- Index reconstitution, where constituents are backfilled as though they had always been included
- Backfilled fundamentals, where data is stamped with the period it describes rather than the date it was published
The standard remedy is point-in-time data, meaning records stamped with when information actually became available rather than what period it refers to. This is expensive to maintain and is the main reason institutional data costs what it does.
When the Model Itself Carries the Bias
The newer problem is that these protections all address the input data, and a language model brings its own history with it.
Research examining this describes how look-ahead bias in large language models arises when information that would not have been available at the time of prediction is included in the training data and inflates prediction performance.
The mechanism is straightforward once stated. A model trained on text through a given year has already encountered every earnings surprise, policy decision and market episode in that window. Prompted with an earlier date, it can recall what happened next rather than reason about it, and the distinction between recall and reasoning is invisible in the output.
Where Memorisation Is Strongest
The same research found the effect varies in a predictable way, which makes it manageable rather than fatal.
Bias was pronounced for lower-frequency data and for index-level series, where a smaller number of well-documented values are easier to memorise. Higher-frequency, individual-stock data showed minimal memorisation effects, and smaller models consistently showed less of it, reflecting limited parameter capacity.
That pattern points somewhere useful. The tasks most vulnerable are the ones involving widely reported aggregate figures, and the tasks least vulnerable involve granular data that appears rarely in training text.
How Widely This Is Checked
The more concerning finding concerns how often any of this is examined at all.
A review of the field identified five recurring biases in financial applications of these models, covering look-ahead, survivorship, narrative, objective and cost bias, and reported that across 164 papers reviewed from 2023 to 2025, no single bias was discussed in more than 28 percent of studies.
The same review argues these biases often compound to create an illusion of validity, and that structural validity should be established before any result supports a deployment claim.
Under 28 percent is the number worth carrying. For any published result in this area, the base rate is that the most common methodological problems went unexamined.
What Point-in-Time Discipline Now Requires
The practical implication is that data hygiene has to extend to the whole workflow rather than stopping at the dataset:
- Test on genuinely post-cutoff periods, after the model’s training data ends
- Check the model’s recall directly, by asking it for historical values and observing the error rate
- Prefer granular data for tasks where memorisation risk is highest
- Consider smaller models where the task does not require frontier capability
- Compare against buy-and-hold, not against zero, on out-of-sample periods
- Document which biases were tested for, since the review suggests most work does not
The second point is unusually cheap to run. A model that reproduces historical index levels with very small errors inside its training window, and much larger errors outside it, is recalling rather than forecasting, and that test takes minutes.
What This Doesn’t Say
None of this establishes that these tools have no application in investment research. The same research that documents the problem also identifies the conditions under which it is small, which is a more useful contribution than a blanket warning.
What it does establish is that a strong backtest is now weaker evidence than it used to be, because a category of contamination exists that traditional data controls were never designed to catch.
The reasonable posture is to treat any result generated over a period the model has seen as unverified until it has been repeated on data that came afterwards. That is a slower process than the current pace of publication in this area, which is roughly the point the review is making.
