Monitors
An asynchronous sampling referee feature in TokenRouter that runs after a response has completed and does not occupy the request path. It performs deterministic sampling at a rate configured per monitor and operates under a daily referee budget. Because it runs outside the request lifecycle, it can evaluate outputs for quality or policy concerns without adding latency to serving.
Timeline (last 90 days)
In TokenRouter, Monitors are asynchronous, sampling-run LLM judges that run a few seconds after the response completes, do not occupy the request path, and add no latency.
TokenRouter's Monitors cap the input of each judgment at about 4k tokens.
In TokenRouter, each monitor has a daily judge budget, and when it is used up the monitor automatically pauses and automatically resumes at UTC midnight; leaving it blank means no limit.
Judge calls go through the organization's own provider key, with zero markup, are rate-limited through the gateway, are logged with source=monitor, and are counted toward the organization's default team.
Monitors take tiered follow-up actions on findings: Flag is always on and enters the review queue; Notify must be enabled and sends at most one email per cooldown window; Disable key must be enabled and has a severity threshold, and automatic disabling should be regarded as a circuit breaker in severe scenarios rather than a content filter.
All Facts (5)
In TokenRouter, Monitors are asynchronous, sampling-run LLM judges that run a few seconds after the response completes, do not occupy the request path, and add no latency.
50%UnverifiedTokenRouter's Monitors cap the input of each judgment at about 4k tokens.
50%UnverifiedIn TokenRouter, each monitor has a daily judge budget, and when it is used up the monitor automatically pauses and automatically resumes at UTC midnight; leaving it blank means no limit.
50%UnverifiedJudge calls go through the organization's own provider key, with zero markup, are rate-limited through the gateway, are logged with source=monitor, and are counted toward the organization's default team.
50%UnverifiedMonitors take tiered follow-up actions on findings: Flag is always on and enters the review queue; Notify must be enabled and sends at most one email per cooldown window; Disable key must be enabled and has a severity threshold, and automatic disabling should be regarded as a circuit breaker in severe scenarios rather than a content filter.
50%