MultiTurnRoute: A New Platform for Multi-Turn Dialogue LLM Routing

MultiTurnRoute platform uses Chain-of-Route for multi-turn dialogue-aware routing, cutting costs up to 20.3% with equal quality.
In multi-turn dialogues, model selection has cross-turn coupling: the chosen model's reply shapes next-turn context, and its cached prefix affects later costs, defeating single-turn routing. MultiTurnRoute unifies inputs—current query, dialogue history, and reusable prefix estimates—via a shared interface. Its Chain-of-Route algorithm uses a learned gating mechanism combining recent and recurrent history to predict quality based on prefix reuse. The Arena evaluation supports recorded ranking and simulated interaction. On WildChat and LMSYS-Chat-1M, CoR matches the strongest single-model quality while reducing normalized cost by up to 20.3% vs. the best baseline, with peak quality equal or better. This work advances multi-turn routing to a platform level for quality-cost trade-offs.
Title
MultiTurnRoute: A New Platform for Multi-Turn Dialogue LLM Routing
Meta Description
MultiTurnRoute is a platform for building and evaluating multi-turn LLM routers. Its Chain-of-Route algorithm uses recent and recurrent dialogue history to predict response quality, while estimating cost with reusable prefixes. On the WildChat and LMSYS-Chat-1M benchmarks, CoR reduces normalized cost by up to 20.3% relative to the strongest baseline while maintaining or improving peak quality.
Article Content
In multi-turn dialogues, choosing which language model answers the current question directly influences how the conversation unfolds in subsequent turns. More critically, the selected model's response shapes the context for the next query, and its cached prompt prefix can reduce the cost of using that model again. This cross-turn coupling makes traditional single-turn routing strategies inadequate in multi-turn settings.

Why Multi-Turn Routing Differs from Single-Turn
The core of single-turn routing is "who should handle the current question," whereas multi-turn routing must also consider "how this turn's choice affects the next turn." The original article notes that the selected model's response forms the context for the next query, and its cached prompt prefix reduces the cost of reusing that model. This means routing decisions are no longer isolated optimization problems but sequential decision problems with state propagation and cost accumulation.
This characteristic brings two direct challenges: first, quality assessment must span multiple turns—a high-scoring response in one turn may lead the conversation into a dead end; second, cost calculation must account for prefix reuse, otherwise the cost of frequently switching models will be overestimated. MultiTurnRoute is designed around these two pain points.
Prompt Prefix Caching is an optimization technique in large model inference: when multiple requests share the same opening text (the "prefix"), the inference engine can cache the key-value (KV) state corresponding to that prefix, and subsequent requests only need to compute the new portion, significantly reducing computation and latency. In multi-turn dialogues, consecutive use of the same model naturally shares historical context, resulting in higher reuse rates; frequent model switching leads to cache invalidation and increased costs. If routing decisions ignore this mechanism, they will misjudge the true cost of switching models.
Platform Architecture and the Chain-of-Route Algorithm
MultiTurnRoute provides a shared interface that allows both built-in routers and user-defined routers to access three types of information: the current query, the dialogue history, and an estimate of the reusable prefix length for each model. This interface design standardizes the "minimum information set required for routing," enabling fair comparison of different strategies under the same inputs.
Its core algorithm, Chain-of-Route (CoR), uses a learned gating mechanism that examines both recent dialogue history and recurrent dialogue history to predict each model's response quality. The cost component is estimated based on each model's reusable prefix length rather than simply summing token counts. The original article mentions that CoR incorporates both "recent" and "recurrent" historical signals in quality prediction, suggesting it attempts to capture short-term intent drift and long-term topic recurrence in conversations.
Gating Mechanism originates from recurrent neural networks (such as LSTM) and mixture-of-experts models (MoE); its role is to learn a weight that dynamically controls the contribution ratio of different information sources. In Chain-of-Route, the gating mechanism receives recent dialogue history and recurrent historical signals and outputs predictions of each model's response quality. This design allows the model to adaptively adjust its reliance on short-term intent and long-term topics based on dialogue state, rather than using a fixed window.
Evaluation Benchmarks and Arena Mechanism
MultiTurnRoute Arena offers two evaluation methods. The first ranks routers on recorded conversations, directly comparing session quality and cost. The second is interactive execution, where simulated users react to each response, thereby comparing different routers' performance in dynamic conversations. This dual-track design of "recording + simulation" compensates for the inability of static benchmarks to reflect changes in conversation direction.
The evaluation benchmarks are drawn from WildChat and LMSYS-Chat-1M, with quality annotations for each turn's response from five LLMs. According to the original data, on this benchmark, when CoR reaches the quality level of the strongest single model, its normalized cost is reduced by up to 20.3% relative to the strongest routing baseline, while peak response quality remains equal or improves. This result demonstrates that multi-turn-aware routing strategies indeed offer considerable optimization space in cost efficiency, not merely theoretical elegance.
WildChat and LMSYS-Chat-1M are two publicly available large-scale real conversation datasets. WildChat collects real interactions between users and models such as ChatGPT, covering diverse tasks and languages; LMSYS-Chat-1M comes from anonymous battle records of Chatbot Arena, containing over 1 million multi-turn conversations. Both provide rich dialogue history and model responses, suitable for evaluating routing strategies under real-world distributions. The original article uses these two datasets and annotates the quality of each turn's response from five LLMs, forming the foundation for multi-turn routing evaluation.
Implications for Multi-Model Application Development
The value of MultiTurnRoute extends beyond an academic benchmark. For teams actually building multi-model dialogue systems, it offers several actionable ideas. First, the router's input should include prefix reuse estimates; otherwise, the cost model will be distorted. Second, quality prediction needs to model both recent and recurrent history; a single window may miss key signals. Third, interactive evaluation is closer to real user experience than static recording and is worth introducing into internal evaluation processes.
Of course, the platform is currently based on annotated data from five LLMs, and the router's generalization ability to larger model pools or dialogues in different domains remains to be verified. How exactly CoR's gating mechanism learns and how sensitive it is to history length are details not elaborated in the original article, which are directions worth following up on.
Overall, MultiTurnRoute advances multi-turn routing from a vague engineering intuition to a platform level that is interface-able, benchmark-able, and interactively evaluable. For multi-model applications that need to finely balance quality and cost, this approach provides a worthwhile starting point.
One-line Summary
MultiTurnRoute platform uses the Chain-of-Route algorithm for multi-turn dialogue-aware routing, reducing costs by up to 20.3% while maintaining quality.
Paragraph Summary
In multi-turn dialogues, model selection involves cross-turn coupling: the selected model's response shapes the next turn's context, and its cached prefix affects subsequent costs, rendering traditional single-turn routing ineffective. The MultiTurnRoute platform provides a shared interface that unifies inputs of the current query, dialogue history, and reusable prefix estimates. Its core algorithm, Chain-of-Route, uses a learned gating mechanism to combine recent and recurrent history to predict each model's response quality, and estimates costs based on prefix reuse. The platform also offers an Arena evaluation mechanism supporting both recorded conversation ranking and simulated user interaction. On the WildChat and LMSYS-Chat-1M benchmarks, CoR achieves the strongest single-model quality while reducing normalized cost by up to 20.3% relative to the strongest baseline, with peak quality equal or improved. This work advances multi-turn routing to a platform level that is interface-able, benchmark-able, and interactively evaluable, providing a referenceable approach for quality-cost trade-offs in multi-model applications.
Related articles

Gemini 4 Hydrogen Released: A 2.5x Faster Everyday AI Model
Google releases Gemini 4 Hydrogen, claiming 2.5x the speed of Gemini 3.8 Flash, with four-tier thinking control, a million-token context, and Orchestrator multi-model collaboration. API pricing is $1.50 per million input tokens.

Gemini 4 Hydrogen Hands-On: Google's Fastest and Cheapest AI Model
Hands-on with Gemini 4 Hydrogen: the fastest, cheapest model in the Gemini 4 family. We break down the Carbon/Argon/Hydrogen split, API pricing traps, free-tier limits, and the cost trade-offs of four thinking levels—with a real bug-debugging demo.

Gemini 4 Hydrogen vs Sonnet Comparison Test: Uses Only 1% of Weekly Quota, Performance on Par
Hands-on test: Gemini 4 Hydrogen vs Sonnet on the same CRUD task—Hydrogen used only 1% weekly quota with comparable performance and better UI details, while Anthropic's model used 7%. Developer says daily tasks don't require expensive models.