decision_router
A Torana plugin that determines movement along the model ladder, either up, down, or holding position, based on user messages and signal counters.
Timeline (last 90 days)
The Torana team developed the decision_router plugin, which connects the Laya decision model to the programming agent to decide whether to use a cheap model or an expensive model in each round of dialogue.
decision_router maintains for each provider a model ladder sorted by capability/cost, and the router can only move one rung along the ladder per round (up, down, or stay in place), and cannot jump to an arbitrary model.
The decision_router plugin has four modes: shadow (only record suggestions), advise (show suggestions and estimated cache rebuild cost), confirm (manually accept), and auto (automatically step up within the switch limit and cost guardrails, while stepping down requires approval).
In a test of 48 synthesized programming requests, under the default four-option question Laya matched the team's annotation on only 21 of them, with a maximum confidence of only 0.26.
The decision_router plugin's default threshold is 0.8, and across all test variants not a single one of Laya's answers reached that threshold.
In a four-choice scenario, even if the top-1 probability reaches 0.64, the normalized confidence may be only 0.26, which is why the plugin, comparing confidence against the 0.8 threshold, can never pass.
The median routing decision time in decision_router is 112 milliseconds, with a p95 of 115 milliseconds; it is queried only once per user turn, and zero calls are made when tool results are passed on.
Because Laya only receives the latest user message and the counter, it cannot perceive which rung of the ladder it is currently on, so the hold it outputs is a judgment made without receiving the premise.
All Facts (8)
The Torana team developed the decision_router plugin, which connects the Laya decision model to the programming agent to decide whether to use a cheap model or an expensive model in each round of dialogue.
50%Unverifieddecision_router maintains for each provider a model ladder sorted by capability/cost, and the router can only move one rung along the ladder per round (up, down, or stay in place), and cannot jump to an arbitrary model.
50%UnverifiedThe decision_router plugin has four modes: shadow (only record suggestions), advise (show suggestions and estimated cache rebuild cost), confirm (manually accept), and auto (automatically step up within the switch limit and cost guardrails, while stepping down requires approval).
50%UnverifiedIn a test of 48 synthesized programming requests, under the default four-option question Laya matched the team's annotation on only 21 of them, with a maximum confidence of only 0.26.
50%UnverifiedThe decision_router plugin's default threshold is 0.8, and across all test variants not a single one of Laya's answers reached that threshold.
50%UnverifiedIn a four-choice scenario, even if the top-1 probability reaches 0.64, the normalized confidence may be only 0.26, which is why the plugin, comparing confidence against the 0.8 threshold, can never pass.
50%UnverifiedThe median routing decision time in decision_router is 112 milliseconds, with a p95 of 115 milliseconds; it is queried only once per user turn, and zero calls are made when tool results are passed on.
50%UnverifiedBecause Laya only receives the latest user message and the counter, it cannot perceive which rung of the ladder it is currently on, so the hold it outputs is a judgment made without receiving the premise.
50%