Model Router
A model routing service introduced by Microsoft Azure Foundry that automatically directs requests to the most suitable language model based on task complexity, optimizing cost and latency while maintaining output quality.
Timeline (last 90 days)
The capability ceiling of a model router is always lower than that of the large language model it dispatches, because using a weaker model to direct a stronger one locks the ceiling at the weakest link.
Model routing decisions typically occur at the start of a request and cannot dynamically adjust based on the complexity that is revealed as a task progresses.
In the author's test of 10 prompts, Model Router delivered a 37.7% cost saving compared to using GPT 5.6 Sol directly.
Microsoft Foundry's Model Router routes prompts to the most suitable language model, rather than invoking the most powerful model for every request.
From a task-type perspective, Model Router is slightly slower on general knowledge and summarization tasks, but performs well on reasoning tasks.
In the author's tests, average latency was slightly higher with Model Router than with GPT 5.6 Sol used directly, and quality win rate declined.
Model Router Auto-Evaluation is an open-source project that can automatically evaluate the performance of Foundry Model Router against any baseline model across three dimensions: quality, cost, and latency.
When faced with math problems requiring step-by-step reasoning, the router automatically switched to the reasoning model Grok 4.1 Fast Reasoning.
When faced with queries requiring factual verification (such as whether Nairobi is a capital), the router assigned the request to GPT-5.5 for processing.
Microsoft Foundry's Model Router automatically selects the most appropriate model for each request.
8 more timeline events
All Facts (14)
The capability ceiling of a model router is always lower than that of the large language model it dispatches, because using a weaker model to direct a stronger one locks the ceiling at the weakest link.
50%UnverifiedModel routing decisions typically occur at the start of a request and cannot dynamically adjust based on the complexity that is revealed as a task progresses.
50%UnverifiedMicrosoft Foundry's Model Router routes prompts to the most suitable language model, rather than invoking the most powerful model for every request.
50%UnverifiedModel Router Auto-Evaluation is an open-source project that can automatically evaluate the performance of Foundry Model Router against any baseline model across three dimensions: quality, cost, and latency.
50%UnverifiedIn the author's test of 10 prompts, Model Router delivered a 37.7% cost saving compared to using GPT 5.6 Sol directly.
50%UnverifiedIn the author's tests, average latency was slightly higher with Model Router than with GPT 5.6 Sol used directly, and quality win rate declined.
50%UnverifiedFrom a task-type perspective, Model Router is slightly slower on general knowledge and summarization tasks, but performs well on reasoning tasks.
50%UnverifiedMicrosoft Foundry's Model Router automatically selects the most appropriate model for each request.
50%UnverifiedThe Model Router covers models from multiple providers including OpenAI, Claude, Grok, Llama, and DeepSeek.
50%UnverifiedThe Model Router is deployed in the same way as a regular model and can be found and deployed on the Model tab in the Discover section of Foundry.
50%UnverifiedWhen deploying the Model Router, multiple deployment types are supported, including Global Standard, Standard, and Data Zone Standard.
50%UnverifiedThe Model Router offers three routing modes: quality-biased, cost-biased, and balanced.
50%UnverifiedThe model router provides a monitoring dashboard where token consumption, input and output of each request, and the actual model relied upon can be viewed.
50%UnverifiedWhen the routing results for a certain type of request are highly concentrated, the routing layer can be removed and the model bound directly to reduce latency and scheduling costs; if routing results remain consistently dispersed, the router's dynamic decision-making should be retained.
50%