Inference-Time Search
A paradigm that improves generation quality by introducing search and scoring mechanisms at the inference stage without updating model parameters. It typically trades additional inference compute for better candidates.
Timeline (last 90 days)
Inference-time search does not update the generative model's parameters, so it does not introduce training instability and also makes it easy to switch optimization objectives quickly.
The cost of inference-time search is higher inference cost, because multiple candidates need to be generated and evaluated.
The article does not give a complete cost curve for how the compute overhead of inference-time search scales with the number of candidates.
In sampling-based inference methods, how to define and measure diversity remains an open topic.
All Facts (4)
Inference-time search does not update the generative model's parameters, so it does not introduce training instability and also makes it easy to switch optimization objectives quickly.
50%UnverifiedThe cost of inference-time search is higher inference cost, because multiple candidates need to be generated and evaluated.
50%UnverifiedThe article does not give a complete cost curve for how the compute overhead of inference-time search scales with the number of candidates.
50%UnverifiedIn sampling-based inference methods, how to define and measure diversity remains an open topic.
50%