QiYao-I: A Manifold Foundation Model for Irregular Multivariate Time Series Forecasting

Time series forecasting is ubiquitous across finance, healthcare, industrial monitoring, and beyond — yet real-world data is rarely "clean." Inconsistent sensor sampling intervals, asynchronous recording across variables, and frequent missing observations make it difficult for most time series foundation models to perform well. A new paper on arXiv introduces QiYao-I, a manifold-based foundation model designed specifically to address these pain points.

The Core Challenge of Irregular Multivariate Time Series

Real-world collected data typically exhibits two prominent characteristics: irregular sampling (observations distributed unevenly along the time axis) and asynchronous recording (different variables captured at different moments). In intensive care settings, for example, heart rate might be recorded every second while blood test results are updated only once every few hours.

Most existing time series foundation models are built on regularly sampled sequences, implicitly assuming uniformly spaced time steps. When faced with truly irregular time intervals and asynchronous cross-variable dependencies, their generalization capability degrades significantly. Simply interpolating or padding the data not only introduces noise, but can also destroy information embedded in the sampling structure itself — sampling density can sometimes be a valuable signal in its own right.

QiYao-I's Two Key Technical Innovations

The paper presents two core mechanisms aimed at enabling the model to naturally understand irregularity.

Sampling-Conditioned Temporal Manifold Attention

QiYao-I introduces a sampling-conditioned temporal manifold attention mechanism. Rather than treating time steps as discrete integer indices, it maps real-valued timestamps into a learnable temporal manifold feature space and injects the derived "temporal manifold bias" into the attention layers.

This design allows the model to simultaneously capture two types of information: the irregular time intervals between observations, and the local sampling structure (whether observations in a given window are dense or sparse). Compared to simply encoding time information as positional vectors, the manifold representation better reflects the geometric nature of time in a continuous space.

What is a "manifold"? A manifold is a central concept in differential geometry — intuitively, a continuous space that locally resembles Euclidean space but may be globally curved (the Earth's surface is a classic two-dimensional manifold). The motivation for introducing manifolds into time series modeling is that real-world timestamps are not isolated numbers; they are embedded in a continuous space with geometric structure. Traditional sinusoidal/cosine positional encodings (as used in the original Transformer paper) were designed for uniformly spaced integer steps and struggle to express the geometric relationships of irregular intervals like "sampled once after 0.5 seconds, then again 3.7 seconds later." Mapping timestamps into a learnable manifold feature space essentially lets the model discover which geometric structure best describes the temporal distribution of the current dataset, rather than imposing a predetermined periodic function. The "manifold bias" in the attention layer acts as a soft prior: nearby observations should also be close to each other on the manifold, naturally endowing the model with an inductive bias toward temporal continuity.

Frequency-Aware Dynamic Variable Interaction

The second innovation is a frequency-aware dynamic variable interaction mechanism. Under asynchronous observation, different variables do not always have alignable information to exchange at every time point. This mechanism selectively executes cross-variable message passing, allowing variables to share information only under appropriate conditions — thereby avoiding spurious cross-variable correlations when synchronized observations are unavailable.

The addition of frequency awareness means the model can distinguish the individual rhythms of change across different variables, which is especially important for handling multivariate data with vastly different sampling rates.

What is "message passing"? Message passing originates from the graph neural network (GNN) domain, referring to an iterative mechanism where nodes encode their own information and pass it to neighboring nodes, which then aggregate and update. In multivariate time series, treating each variable as a node on a graph allows cross-variable interaction to be modeled as message passing. However, asynchronous observation introduces a key challenge: if variable A has an observation at t=1 but variable B does not, forcing B to pass a message to A may only propagate interpolation noise. The modifiers "dynamic" and "frequency-aware" jointly address this problem — "dynamic" means the graph structure for message passing is not fixed but constructed dynamically based on the time window and observation availability; "frequency-aware" enables the model to recognize each variable's own oscillation/change period, preventing short-term fluctuations of high-frequency variables (e.g., heart rate) from being mistakenly passed as trend signals for low-frequency variables (e.g., body temperature).

Experimental Performance and Generalization

The paper conducts extensive experiments on multiple real-world irregular multivariate forecasting benchmarks. Results show that QiYao-I outperforms two categories of comparison methods:

• Time series foundation models: General-purpose foundation models that assume regularly sampled sequences;
• End-to-end irregular forecasting models: Non-foundation-model methods specifically designed for irregular data.

Perhaps more noteworthy is the strong generalization ability demonstrated under zero-shot and few-shot settings. This indicates that QiYao-I is not simply a specialized model overfitted to a particular dataset, but genuinely possesses the transfer characteristics expected of a foundation model — remaining competitive even on unseen tasks or those with very few samples.

Why do zero-shot and few-shot evaluations matter? These settings are key indicators of whether a model truly qualifies as a "foundation model." Zero-shot means the model makes predictions on a dataset it has never seen any training samples from, relying entirely on knowledge accumulated during pretraining. Few-shot allows lightweight fine-tuning with a very small number of target-domain samples (e.g., a few dozen sequences). For time series, these settings are particularly challenging because datasets differ enormously in scale, sampling rate, and variable semantics — cross-domain transfer is far harder than in images or text. QiYao-I's competitiveness in both settings suggests its manifold representations and dynamic interaction mechanisms have learned structural patterns independent of specific datasets, rather than memorizing the statistics of particular training sets. This property has significant practical value: when new sensors come online or new medical indicators emerge, the model can be deployed quickly without collecting large amounts of labeled data.

Significance and Outlook

QiYao-I's approach represents a directional shift in time series foundation models — from "regularity-first" to "embracing irregularity." By mapping timestamps into a learnable manifold and carefully controlling asynchronous variable interaction, it attempts to absorb the complexity of real-world data at the modeling level, rather than relying on preprocessing to "force alignment."

For domains naturally filled with irregular sampling — medical records, IoT, high-frequency financial data — this kind of approach has the potential to reduce data engineering overhead and improve forecasting reliability. That said, as a newly released preprint, its robustness at larger deployment scales, computational costs, and across diverse domains remains to be validated. Overall, it provides a compelling new baseline worth watching for irregular multivariate time series forecasting.