Test-Time Scaling
A concept originating from OpenAI's Strawberry project, in which additional compute is allocated during the inference phase to enhance model performance. By expanding the search over output space, it compensates for limitations in parameter count and can bring edge model performance close to that of large cloud-based models.
Core Facts
Test-time Compute Scaling improves model performance by investing more computational resources during the inference stage, and research indicates it is more efficient than simply increasing model parameter count on certain tasks.
Test-time compute scaling improves performance on complex reasoning tasks by investing more computation during the inference stage through methods such as extending the chain of thought, multi-path sampling with best-of-n selection, and self-verification reflection.
Inference-time computation improves output quality by allocating more computational resources during the inference stage while keeping model parameters fixed.
Timeline (last 90 days)
Test-Time Scaling refers to methods that improve model performance at the inference stage by increasing computation (such as multiple sampling, longer CoT, tree search, or voting mechanisms).
Test-time scaling yields significant gains on verifiable tasks such as math and code, because the extra computation can be paired with a verifier to filter out correct answers; but when reliable verification signals are lacking, more sampling merely draws repeatedly from within the error distribution.
Traditional parallel retrieval is fast but often sacrifices accuracy, while sequential search yields more precise results but introduces noticeable latency.
All Facts (6)
Test-time Compute Scaling improves model performance by investing more computational resources during the inference stage, and research indicates it is more efficient than simply increasing model parameter count on certain tasks.
90%VerifiedTest-time compute scaling improves performance on complex reasoning tasks by investing more computation during the inference stage through methods such as extending the chain of thought, multi-path sampling with best-of-n selection, and self-verification reflection.
65%VerifiedInference-time computation improves output quality by allocating more computational resources during the inference stage while keeping model parameters fixed.
65%UnverifiedTest-Time Scaling refers to methods that improve model performance at the inference stage by increasing computation (such as multiple sampling, longer CoT, tree search, or voting mechanisms).
50%UnverifiedTest-time scaling yields significant gains on verifiable tasks such as math and code, because the extra computation can be paired with a verifier to filter out correct answers; but when reliable verification signals are lacking, more sampling merely draws repeatedly from within the error distribution.
50%UnverifiedTraditional parallel retrieval is fast but often sacrifices accuracy, while sequential search yields more precise results but introduces noticeable latency.
50%