TempoPFN: Synthetic Pre-training of Linear RNNs Enables Zero-Shot Time Series Forecasting

Time series forecasting is having its "foundation model moment." TempoPFN, which sparked discussion on Reddit, proposes a counterintuitive path: instead of relying on massive amounts of real-world data, it pre-trains a linear RNN on carefully designed synthetic time series, enabling the model to perform zero-shot forecasting directly on datasets it has never seen. The work has been accepted at NeurIPS 2026, and the library, model weights, and paper have all been open-sourced.

Core Idea: Synthetic Data Instead of Real Corpora

According to the author team (Vladyslav Moroshan, Julien Siems, Arber Zela, Timur Carstensen, Frank Hutter), the core contribution of TempoPFN is a library of synthetic time series generators and augmentation methods, purpose-built for training time series foundation models. Unlike language models that depend on internet text, the time series field lacks a unified "pre-training corpus," so TempoPFN chooses to synthesize actively: controllable generators construct sequences with diverse statistical properties, exposing the model to a sufficiently rich set of patterns during pre-training.

The value of this approach lies in sidestepping two major pain points of real data—incomplete domain coverage and high acquisition costs. The library-style design of the generators means researchers can combine and extend them as needed, rather than being tied to a single dataset.

Foundation Model typically refers to a model pre-trained on large-scale, diverse data that can be adapted to downstream tasks with minimal fine-tuning or even zero-shot transfer. This paradigm is mature in language models, but the time series field has long lacked a unified large-scale pre-training corpus, causing models to be trained on a single dataset or a single domain. TempoPFN's synthetic pre-training attempts to replicate the "large-scale pre-training + downstream transfer" path in the time series domain: replacing real corpora with controllable generators, giving the model cross-domain pattern coverage during pre-training, and thereby achieving zero-shot generalization.

Why Choose a Linear RNN Over a Transformer

The model pre-trained in the paper is a linear RNN. At a time when Transformers dominate foundation models, this choice is pointed: time series are often very long, and the quadratic complexity of attention mechanisms is costly on long sequences; linear RNNs have linear complexity at inference time, making them better suited to long-context and high-throughput forecasting scenarios.

The combination of synthetic pre-training and a linear architecture targets the goal of "zero-shot"—the model is not fine-tuned for a specific dataset but directly produces forecasts on new sequences. This has clear implications for real-world deployment: users do not need to retrain the model for every new business sequence.

A linear RNN refers to a recurrent network whose state transition contains no nonlinear activation functions, with hidden state updates that can be written as (h_t = A h_t + B x_t). The key advantage of this structure is that both training and inference can be computed efficiently via parallel scans or convolution kernels, and inference complexity grows linearly with sequence length. Compared with classical nonlinear RNNs, linear RNNs have limited expressiveness but gain efficiency and stability on long sequences; compared with Transformers, they avoid the quadratic complexity of attention. Recently, in research on state space models (such as the Mamba family), linear recurrent structures have regained attention, and TempoPFN's use of them for time series foundation models is precisely due to the efficiency advantage in long-context forecasting scenarios.

Already Adopted by Toto-2 and TS-ICL

One noteworthy signal is that the Reddit post mentions the synthetic data has already been used by Toto-2 and TS-ICL. This shows that TempoPFN's contribution goes beyond a single model—its generator library is becoming part of the infrastructure for training time series foundation models. For the community, a reusable data generation pipeline often has more long-term value than a single checkpoint.

Toto-2 is a series of time series foundation models from Datadog, focused on forecasting for observability and monitoring scenarios; TS-ICL focuses on in-context learning (ICL) for time series, i.e., enabling a model to perform forecasts on new tasks based solely on input context without updating parameters. Both rely on large-scale, diverse pre-training data, and their adoption of TempoPFN's synthetic generator library indicates that the library already has cross-team reuse value in time series foundation model training. For the community, sharing at the "data recipe" level often drives progress in the field more than sharing a single model's weights.

Open-Source Resources and Getting Started

The authors have released complete resources: the paper on arXiv (2510.25502), the code in the GitHub repository automl/TempoPFN, and the model weights hosted on Hugging Face at AutoML-org/TempoPFN. These three entry points cover the full chain of "read the method—run the code—use the model directly," making it relatively friendly for researchers who want to reproduce or build on the work.

In terms of positioning, TempoPFN is closer to a research infrastructure than an end-to-end product: the generator library is used to create data, the linear RNN is used to validate the effectiveness of synthetic pre-training, and zero-shot forecasting is the task form for evaluating the results.

A Few Questions Worth Pondering

The boundary of synthetic data diversity determines how "wild" the real sequences the model can generalize to; whether the trade-off in the linear RNN's expressiveness will reveal weaknesses on certain strongly nonlinear sequence patterns; and when synthetic pre-training is reused by multiple models, whether the community will form a "data recipe" consensus similar to NLP. The answers to these questions may have to wait for more follow-up work based on this library.

One-line Summary

TempoPFN pre-trains a linear RNN on synthetic time series to achieve zero-shot forecasting, fully open-sourced.

Paragraph Summary

TempoPFN is a time series foundation model work accepted at NeurIPS 2026, whose core is a library of synthetic time series generators and augmentation methods. Unlike language models that depend on internet text, time series lacks a unified pre-training corpus, so TempoPFN chooses to actively synthesize sequences with diverse statistical properties, sidestepping the incomplete domain coverage and high acquisition costs of real data. The pre-trained model is a linear RNN, whose linear inference complexity suits long sequences and high-throughput scenarios, with the goal of direct zero-shot forecasting on unseen datasets. Its synthetic data has already been adopted by Toto-2 and TS-ICL, demonstrating its infrastructure value. The paper, code, and model weights are all open-sourced, positioned as research infrastructure rather than an end-to-end product.