DAWIS: Unifying Filtering and Smoothing in Data Assimilation via Windowed Inverse Sampling

Generative Models Meet Data Assimilation

Flow models and diffusion models have expanded well beyond image generation in recent years, emerging as flexible and powerful tools for dynamical system forecasting. When combined with inference-time guidance, they offer a compelling path toward high-dimensional, non-Gaussian data assimilation (DA).

The core task of data assimilation is to fuse model predictions with real-world observations to estimate the latent state of a system — a foundational problem in meteorology, oceanography, geophysics, and related fields. However, existing generative model-based filtering methods share a structural limitation: they are anchored to a fixed historical window and only assimilate the most recent observation.

This means that when new observations arrive, these methods cannot go back and revise past state estimates. The estimates are locked to a segment of history, and subsequent observations — which may contradict that history — cause errors to accumulate throughout the assimilation process. This is precisely the pain point that the paper proposing DAWIS, available on arXiv, sets out to address.

Background: Classical Data Assimilation Methods — The dominant algorithms in traditional data assimilation include the Kalman Filter and its nonlinear extensions: the Ensemble Kalman Filter (EnKF) and the Particle Filter. Kalman-based methods assume Gaussian state and observation errors, performing well in linear systems but losing accuracy in high-dimensional, non-Gaussian settings. Particle filters can theoretically handle arbitrary distributions, but suffer from "particle degeneracy" in high-dimensional spaces, with computational costs that escalate rapidly. This dilemma has spurred research into using generative models to construct more flexible priors — diffusion models and flow models can implicitly learn complex high-dimensional distributions without explicit Gaussian assumptions, providing more expressive priors for non-Gaussian, nonlinear assimilation problems.

What DAWIS Does

The paper introduces DAWIS (Data Assimilation with Windowed Inverse Sampling), a unified framework that covers three common data assimilation modes under a single construction: filtering, fixed-lag smoothing, and block smoothing.

The key innovation is that instead of using a single flow time for a state-level prior, DAWIS introduces a multitask stochastic interpolant that operates over a window of consecutive states and assigns each state in the window its own independent flow time.

In other words, while traditional methods treat the entire historical segment as a monolithic unit processed with a single time parameter, DAWIS decouples each state within the window and assigns each one an individually tunable level of treatment. This design gives the method a natural capacity for retrospective correction.

Stochastic Interpolants — A Brief Introduction — Stochastic interpolants are a mathematical framework for constructing generative models. Their core idea is to connect a noise distribution to a target data distribution via a path defined by a stochastic differential equation. Unlike diffusion models, which use fixed noise-addition and removal protocols, stochastic interpolants allow designers to freely choose the endpoints and form of the interpolation path, making them considerably more flexible. Flow Matching can be viewed as a special deterministic case of this framework. DAWIS introduces the concept of "multitask" into this framework, enabling a single model to simultaneously handle states at multiple time steps within the window — each state with its own independent "flow time" parameter — thereby coherently revising or preserving multiple states in a single generation pass.

Inverting the Window and Controlling Turning Points

A single assimilation cycle in DAWIS proceeds in two steps. First, it "inverts" the entire state window into a vector of per-state turning points. Second, it regenerates the window under observation guidance.

The turning points play a critical regulatory role — they control whether each state is held fixed, corrected, or regenerated from scratch during the regeneration process. States near the end of the window that are heavily influenced by new observations can be substantially revised, while earlier, more reliably established states can remain stable.

This mechanism elegantly addresses the problem raised at the outset: when new observations arrive, DAWIS can look back, reassess, and revise past states rather than being locked into history.

How Inference-Time Guidance Works — During a generative model's sampling process, guidance refers to using external conditions (in this case, actual observations) to adjust the sampling direction in real time, so that generated outputs are consistent with the observations. A common approach is to incorporate the gradient of the observation likelihood at each step of reverse diffusion or flow, "pulling" the sampling trajectory toward regions consistent with the observations — without retraining the model. The advantage of inference-time guidance lies in this decoupling: the generative model only needs to learn the prior distribution of states, while observation fusion is handled entirely during the sampling phase. The same pretrained model can therefore adapt to different observation operators and noise levels. DAWIS leverages this mechanism to inject information from newly arriving observations into the correction of individual states during window regeneration.

Absorbing the Forecast Step

The paper further shows that the same construction can absorb the forecast step into the assimilation cycle itself. This means a separate forecasting model is no longer needed to advance the system state — prediction and assimilation are unified within a single pipeline.

From an engineering standpoint, this is a meaningful simplification. Traditional data assimilation systems typically maintain separate forecasting and assimilation modules, and the coupling and error propagation between them are common headaches in real-world deployments. DAWIS's unified construction has the potential to reduce this system complexity.

Experimental Results and Open Source

The paper evaluates DAWIS on multiple challenging nonlinear systems, demonstrating its performance under sparse, noisy, and nonlinear observation conditions. Results show improvements over both filtering and smoothing baselines.

These are precisely the most difficult conditions encountered in real-world data assimilation — observations are often incomplete (sparse), corrupted by measurement error (noisy), and produced by highly nonlinear systems. Outperforming both filtering and smoothing baselines under these conditions suggests that the framework's unified design does not come at the cost of single-task performance.

The authors have open-sourced the DAWIS code at github.com/Erik-Wikingsson/DAWIS, enabling researchers to reproduce results and explore further.

Significance and Outlook

The value of DAWIS lies not only in performance gains, but in bringing filtering, fixed-lag smoothing, and block smoothing — tasks long studied in isolation — under a single framework. Through the combination of a multitask interpolant and per-state turning points, it redefines the operational paradigm of generative data assimilation in a tunable, retrospectively correctable way.

For fields that rely on data assimilation, such as weather forecasting and Earth system modeling, a unified method that can dynamically revise historical states as new observations arrive — and that embeds forecasting within the assimilation cycle — holds considerable practical promise. That said, the scalability and computational cost of this approach in large-scale operational systems remains to be examined by future work.

The Fundamental Distinction Between Filtering and Smoothing — Filtering estimates the current state using only current and past observations; it is an online, causal estimation procedure. Smoothing allows future observations to be used in revising past state estimates, generally achieving higher accuracy but requiring storage and processing of longer observation sequences. Fixed-lag smoothing is a compromise between the two — it only retrospectively corrects a fixed-length historical window, balancing accuracy with computational efficiency within an acceptable latency. Block smoothing treats an entire observation window as a single batch. DAWIS naturally reduces to any of these modes through parameter settings via its tunable per-state turning point vector, a unification that brings significant convenience to both engineering deployment and theoretical analysis.