Putting TimesFM 3.0, Chronos-2, and Toto 2.0 to the Test: Time-Series LLMs Can't Beat Simple Baselines

A developer ran a zero-shot benchmark of several recently hyped time-series foundation models, and the results were surprising: on stock price forecasting tasks, Google's TimesFM 3.0, Amazon's Chronos-2, and Datadog's Toto 2.0 all failed to outperform simple baselines.

Experimental Design: Zero-Shot Challenge on Market Data

The test covered 24 large-cap U.S. stocks and ETFs, including SPY, QQQ, and IWM, with data from Yahoo Finance spanning January 2014 to November 2026. The evaluation window was 504 trading days starting from November 2024, using 512 historical closing prices as context to forecast across three horizons: 1 day, 5 days, and 20 days.

All models ran in zero-shot mode with default configurations and no fine-tuning. The comparison baselines included: the last observed price, a direction strategy that always predicts an uptrend, and a simulated random walk matched to drift and volatility. The experiment was also extended to hourly and 5-minute candlestick data.

The code is open-sourced in the GitHub repository moneyheap/tsfm-stocks.

Zero-shot learning refers to a model making predictions directly on entirely new inputs without any fine-tuning on task-specific data. For time-series foundation models, this means the model relies entirely on general time-series patterns learned during pre-training rather than adjusting parameters for stock data. The advantage of zero-shot evaluation is that it cleanly measures a model's generalization ability—if the model has genuinely learned transferable time-series regularities during pre-training, its zero-shot performance should beat naive baselines; conversely, if zero-shot performance is indistinguishable from random guessing, it indicates that pre-training knowledge cannot transfer to that domain. This also explains why the experiment specifically emphasizes "using default configurations and no fine-tuning": any fine-tuning could introduce data leakage or overfitting, thereby obscuring judgments about the model's inherent capabilities.

Directional Accuracy: No Better Than a Coin Flip

The next-day directional accuracy of the three models ranged from 49.9% to 51.1%. Meanwhile, the most naive strategy—predicting the market will rise every single day—achieved 52.2% accuracy.

A more revealing finding: when the researcher fed simulated random walk data—data constructed to contain no exploitable predictive signal whatsoever—into the models, their performance was roughly the same as on real stock prices. This means the models' directional judgments had almost nothing to do with whether real signal existed in the price series.

Repeating the experiment on hourly and 5-minute data yielded similar results: directional accuracy remained near random levels. On 5-minute data, a simple one-line mean-reversion heuristic actually performed better.

Random walk is a classic model in finance describing asset price movements, whose core assumption is that the next price change is independent of history and its direction is unpredictable. Under the random walk hypothesis, the best forecast is the current price (i.e., the "last observed price" baseline), and directional accuracy should approach 50%. Feeding simulated random walk data into the models was precisely to construct a "known no-signal" control group: if the models' directional accuracy on random walks is nearly identical to that on real stock prices, it shows that the models' directional judgments are not based on predictable market regularities, but rather on fitting the statistical characteristics of the series—or perhaps merely a mechanical response to autocorrelation or volatility clustering. This control design is a key experimental method for determining whether time-series models have truly "learned signal."

Simulated Trading: Model Strategies Underperform Buy-and-Hold

The researcher also tested a simple long/cash switching strategy: hold the stock when the model predicts an uptrend, otherwise hold cash. Starting capital was $10,000, with a 0.05% transaction cost per switch.

The time-series model strategies ended with final assets between $10,893 and $12,181, while a simple buy-and-hold strategy reached $15,036. The gap is stark.

The author acknowledges that this is only a simple strategy and not an exhaustive assessment of trading profitability, but the results still tell us something.

Why Do Time-Series Foundation Models Fail on Financial Data?

This experiment reveals a core contradiction: time-series foundation models perform brilliantly on general forecasting tasks, but financial price series have an extremely low signal-to-noise ratio. The models' "predictive ability" on random walks was nearly identical to that on real stock prices, suggesting they may merely be fitting the statistical characteristics of the series rather than extracting genuine market signal.

Another noteworthy detail is that in zero-shot mode, models cannot leverage market-specific structural information such as trading volume, macroeconomic events, or market sentiment. Even with fine-tuning, the non-stationarity of financial data would expose model training to severe overfitting risk.

Signal-to-noise ratio (SNR) refers to the ratio of predictable signal power to random noise power in data. The extremely low SNR of financial price series means that the vast majority of daily price movements are unpredictable random fluctuations, with only a minuscule amount of systematic signal that models can capture. Time-series foundation models excel on tasks like traffic forecasting and demand planning because those scenarios have higher SNR, with clear and stable periodic and trend structures. In financial data, weak signals are easily drowned out by noise, and even if models learn some statistical regularities, it's difficult to translate them into a stable directional forecasting edge. Moreover, non-stationarity means that the statistical distributions of financial data change over time, so patterns learned on the training set may fail in the future—further exacerbating overfitting risk and making it difficult for fine-tuning to deliver reliable improvements.

A Rational Look at AI Forecasting Capabilities

This test does not deny the value of time-series foundation models. TimesFM, Chronos, and Toto are designed for general time-series forecasting scenarios such as traffic forecasting, demand planning, and sensor data analysis—scenarios that typically have clearer periodic and trend structures.

But in financial markets—a recognized high-noise, low-SNR environment—simple baselines are often the strongest opponent. For developers looking to apply time-series large models directly to quantitative trading, this experiment provides a necessary reality check: under default configurations and zero-shot conditions, the models have yet to demonstrate a forecasting advantage over naive methods.