TW3Cast: How a System Without Agent Routing Reached 3rd Place on GIFT-Eval Time Series Forecasting

A Time Series Forecasting System That Swims Against the Tide

In an era where large models and AI agents dominate benchmark leaderboards, a system that uses neither agents nor language models has climbed to 3rd place (out of 130 entries, ranked by average MASE) on the authoritative GIFT-Eval time series forecasting benchmark. That system is TW3Cast, with the ranking snapshot dated 2026-09-14.

The two entries ranked above it both belong to the leaderboard's "agentic category" — multi-step systems that use agents or language models to reason, generate, or filter predictions. By contrast, TW3Cast's approach feels almost retro in its simplicity: at its core is a routing table computed once on the training set and then completely frozen. No chain-of-thought reasoning, no dynamic decisions at runtime — everything is decided during training.

This design choice is itself a thought-provoking signal: in the rush toward complex agentic architectures, a carefully engineered "static routing + lightweight fine-tuned foundation models" solution can still go toe-to-toe with the most cutting-edge methods.

How the Frozen Routing Table Works

TW3Cast faces 97 combinations of dataset, frequency, and prediction horizon configurations on GIFT-Eval. For each configuration, the frozen routing table selects one of four modes:

Specialist mode: LoRA or full fine-tuning of one of three public foundation models — Chronos-2, TiRex, or Toto — with training data processed through explicit data-cleaning and augmentation rules;
Quantile blend with specialist: ensemble at the quantile level, built on top of the specialist;
Blend of base models: a combination of multiple foundation models;
Selection tournament: a best-of tournament conducted on a backtest split carved out from the training set.

The key point is that every decision in the routing table is made on this training-derived backtest — the test set is never touched. This is the technical foundation behind the system's claim of "model selection using only the training set."

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that injects a pair of low-rank matrices alongside the pretrained model's weight matrices. Only these small matrices are updated during training, while the original weights remain frozen. This typically reduces the number of trainable parameters to less than 1% of full fine-tuning, while achieving comparable results. In TW3Cast, applying LoRA fine-tuning to large time series foundation models like Chronos-2, TiRex, and Toto means a specialized version can be "cheaply" produced for each dataset configuration without incurring the full cost of retraining the entire model.

MASE (Mean Absolute Scaled Error) is a dimensionless error metric widely used in time series forecasting. It divides the absolute error by the mean absolute error of a naive baseline (typically a random walk) on the training set, enabling cross-series comparisons across different scales and frequencies — exactly the unified metric needed by multi-dataset benchmarks like GIFT-Eval.

The Low-Cost Candidate Admission Logic

One clever aspect of TW3Cast is its candidate admission mechanism. A specialist model is immediately accepted if it beats the current tournament result on the backtest. This means the cost of trying a new candidate is extremely low — just a few megabytes of storage and a few minutes of GPU time — and a failing candidate changes nothing.

This "low-risk trial-and-error" framework allows the system to continuously absorb stronger specialists in an incremental, controlled manner without risking noise or degradation. Compared to end-to-end training that typically demands substantial compute, this reflects a highly pragmatic engineering philosophy.

Triple Defense: Fighting Selection Bias

Any backtest-based model selection faces a fundamental risk: models that perform well on historical data may not generalize. TW3Cast employs three "guard mechanisms" to prevent selection bias from skewing results:

Dual criteria of accuracy and calibration: not just evaluating prediction accuracy, but also verifying whether uncertainty estimates are reliable;
Targeted asymmetric margins: applying stricter margin penalties to candidates that have "seen" the target series during training, preventing inflated performance from data leakage;
Conservative per-window gating: conservative admission thresholds for each prediction window.

Furthermore, these selection rules themselves are not arbitrary — they were chosen through a "temporal meta-backtest." This nested validation layer strengthens the robustness of the entire pipeline.

Calibration in probabilistic forecasting refers to whether a model's confidence intervals or quantile predictions are "honest": a "90% prediction interval" should actually cover the true value roughly 90% of the time — not be too narrow or too wide. A model with accurate point predictions but poor calibration has unreliable uncertainty estimates, causing downstream decision-makers (e.g., in inventory management or risk control) to make systematically biased judgments. TW3Cast treats calibration as an independent evaluation dimension alongside accuracy, reflecting a focus on real-world usability.

A temporal meta-backtest is essentially a "backtest of backtests": one backtest evaluates candidate models, while an earlier historical period validates whether the selection rules themselves are effective — avoiding the scenario where the selection rules overfit the same data. This is the standard approach for handling double-nested selection bias, and is particularly well-established in quantitative finance strategy evaluation.

The Numbers: Gains from Routing

The comparative data in the paper clearly demonstrates the value of the routing mechanism:

Using the best-performing base model alone: average MASE rank of 33.8
Running a tournament selection on every configuration: average MASE rank of 38.0
The full router: average MASE rank of 19.4

Interestingly, the pure tournament approach (38.0) actually performs worse than the best single model (33.8), suggesting that a naive "pick the best for each config" strategy can be hurt by overfitting to the backtest. The real gains come from the full router's integration of specialist fine-tuning, blending strategies, and guard mechanisms — together driving the rank from the 33.8 level down to 19.4.

This data reveals a phenomenon well-known in ensemble learning but often overlooked: the "winner-take-all" selection strategy (every-config tournament, 38.0) underperforms simply using the best single model (33.8). The reason is that per-configuration selection amplifies backtest noise — each independent selection accumulates a little overfitting, leading to greater cumulative error on the test set. The true gains come from the router combining three factors: reduced bias from fine-tuned specialists, variance cancellation from blending strategies, and active overfitting suppression from guard mechanisms. Together, they yield a systematic improvement in the bias-variance tradeoff. This parallels the logic of random forests versus single decision trees: not simply "picking the best," but using structured combination so that errors across components cancel each other out.

Reproducibility: Regenerate Every Leaderboard Number with One Script

TW3Cast is remarkably thorough in its openness. The authors have publicly released the routing table, specialist index, foundation models at pinned revisions, submitted score files, and dated snapshots of public scores. Every leaderboard number in the paper can be regenerated from these materials using a single script.

This commitment to "one-click reproducibility from source materials" is especially valuable at a time when reproducibility in AI research is under intense scrutiny. It transforms the claim "we ranked third" into an engineering fact that anyone can verify firsthand.

Implications for Time Series Forecasting

The significance of TW3Cast may lie less in its ranking and more in the sober reference point it provides: while agents and language models are undeniably powerful, a disciplined static selection framework paired with lightly fine-tuned open-source foundation models can still approach the top of the leaderboard.

For practitioners in industry, this path has obvious appeal — it is predictable, reproducible, incurs zero agent overhead at inference time, and has extremely low trial-and-error costs. At a time when many teams are wrestling with the costs and uncertainties of deploying complex agentic systems, TW3Cast demonstrates a more grounded, high-performance alternative.