Token Load
An evaluation metric measuring all token consumption during agent execution, including failed attempts, used to reflect actual deployment cost.
Timeline (last 90 days)
The researchers developed an evaluation framework comprising five complementary metrics: VCR, Model-Harness Fit, Cross-Model Fit, Interaction Residuals and Token Load.
Token Load measures the total token demand of Agent execution, including consumption from failed attempts.
VCR should become an important pre-deployment check, and Token Load should be incorporated into cost models.
All Facts (3)
The researchers developed an evaluation framework comprising five complementary metrics: VCR, Model-Harness Fit, Cross-Model Fit, Interaction Residuals and Token Load.
50%UnverifiedToken Load measures the total token demand of Agent execution, including consumption from failed attempts.
50%UnverifiedVCR should become an important pre-deployment check, and Token Load should be incorporated into cost models.
50%