Model Router
A model routing service introduced by Microsoft Azure Foundry that automatically directs requests to the most suitable language model based on task complexity, optimizing cost and latency while maintaining output quality.
Core Facts
Timeline (last 90 days)
Model routing decisions typically occur at the start of a request and cannot dynamically adjust based on the complexity that is revealed as a task progresses.
The capability ceiling of a model router is always lower than that of the large language model it dispatches, because using a weaker model to direct a stronger one locks the ceiling at the weakest link.
Model Router Auto-Evaluation is an open-source project that can automatically evaluate the performance of Foundry Model Router against any baseline model across three dimensions: quality, cost, and latency.
In the author's tests, average latency was slightly higher with Model Router than with GPT 5.6 Sol used directly, and quality win rate declined.
From a task-type perspective, Model Router is slightly slower on general knowledge and summarization tasks, but performs well on reasoning tasks.
In the author's test of 10 prompts, Model Router delivered a 37.7% cost saving compared to using GPT 5.6 Sol directly.
Microsoft Foundry's Model Router routes prompts to the most suitable language model, rather than invoking the most powerful model for every request.
After enabling balanced mode in the cake ordering agent scenario, the router repeatedly and stably selected GPT-5.6 nano to handle order placement tasks.
When faced with math problems requiring step-by-step reasoning, the router automatically switched to the reasoning model Grok 4.1 Fast Reasoning.
The model router provides a monitoring dashboard where token consumption, input and output of each request, and the actual model relied upon can be viewed.
9 more timeline events
All Facts (12)
There are two paths to getting the model to output an accurate solution: writing detailed requirements all at once, or using follow-up questioning to have the model think on the user's behalf and accumulate context.
65%UnverifiedModel routing decisions typically occur at the start of a request and cannot dynamically adjust based on the complexity that is revealed as a task progresses.
50%UnverifiedThe capability ceiling of a model router is always lower than that of the large language model it dispatches, because using a weaker model to direct a stronger one locks the ceiling at the weakest link.
50%UnverifiedMicrosoft Foundry's Model Router routes prompts to the most suitable language model, rather than invoking the most powerful model for every request.
50%UnverifiedModel Router Auto-Evaluation is an open-source project that can automatically evaluate the performance of Foundry Model Router against any baseline model across three dimensions: quality, cost, and latency.
50%UnverifiedThe Model Router offers three routing modes: quality-biased, cost-biased, and balanced.
50%UnverifiedThe model router provides a monitoring dashboard where token consumption, input and output of each request, and the actual model relied upon can be viewed.
50%UnverifiedWhen the routing results for a certain type of request are highly concentrated, the routing layer can be removed and the model bound directly to reduce latency and scheduling costs; if routing results remain consistently dispersed, the router's dynamic decision-making should be retained.
50%UnverifiedMicrosoft Foundry's Model Router automatically selects the most appropriate model for each request.
50%UnverifiedThe Model Router covers models from multiple providers including OpenAI, Claude, Grok, Llama, and DeepSeek.
50%UnverifiedThe Model Router is deployed in the same way as a regular model and can be found and deployed on the Model tab in the Discover section of Foundry.
50%UnverifiedWhen deploying the Model Router, multiple deployment types are supported, including Global Standard, Standard, and Data Zone Standard.
50%