Token Budget
Timeline (last 90 days)
Existing interventions such as token budgets or guiding prompts mainly regulate the length of reasoning rather than exerting fine-grained control over specific reasoning behaviors, and thus may inadvertently harm necessary planning or verification steps.
A mature practice is to use the per-task token budget as a feedback signal for specification quality: once it is exceeded, pause execution, go back and clarify the boundaries and completion criteria, and then let the agent continue.
Tokens should be reserved for the output and the message framing, and then the input quota allocated.
Long-context calls need budgets set per team, with input token histograms recorded weekly, where peaks correspond to bad prompts or missing RAG.
All Facts (4)
Existing interventions such as token budgets or guiding prompts mainly regulate the length of reasoning rather than exerting fine-grained control over specific reasoning behaviors, and thus may inadvertently harm necessary planning or verification steps.
50%UnverifiedA mature practice is to use the per-task token budget as a feedback signal for specification quality: once it is exceeded, pause execution, go back and clarify the boundaries and completion criteria, and then let the agent continue.
50%UnverifiedTokens should be reserved for the output and the message framing, and then the input quota allocated.
50%UnverifiedLong-context calls need budgets set per team, with input token histograms recorded weekly, where peaks correspond to bad prompts or missing RAG.
50%