LP-SMC
A sequential Monte Carlo decoding method that improves inference in diffusion language models. Through Chopthin resampling and an input weight floor, it preserves the ancestral lineages of low-weight particles, balancing concentrated computation against diversity and improving performance on mathematical and code generation tasks.
Timeline (last 90 days)
LP-SMC proposes two core changes: Chopthin resampling and an input weight floor.
LP-SMC improves oracle coverage by 1.8 percentage points to 25.6 percentage points.
When using a general-purpose reward model selector, LP-SMC achieves end-to-end accuracy improvements on eleven of twelve settings.
The coverage gains persist when using an external reward model, indicating that diversity-preserving resampling and the guidance signal each independently contribute to the effectiveness of inference-time search.
LP-SMC's changes are concentrated in the resampling step and require neither modifying the model itself nor additional training.
Both Chopthin resampling and the weight floor introduce hyperparameters, which need to be tuned for specific tasks in practical deployment.
All Facts (6)
LP-SMC proposes two core changes: Chopthin resampling and an input weight floor.
50%UnverifiedLP-SMC improves oracle coverage by 1.8 percentage points to 25.6 percentage points.
50%UnverifiedWhen using a general-purpose reward model selector, LP-SMC achieves end-to-end accuracy improvements on eleven of twelve settings.
50%UnverifiedThe coverage gains persist when using an external reward model, indicating that diversity-preserving resampling and the guidance signal each independently contribute to the effectiveness of inference-time search.
50%UnverifiedLP-SMC's changes are concentrated in the resampling step and require neither modifying the model itself nor additional training.
50%UnverifiedBoth Chopthin resampling and the weight floor introduce hyperparameters, which need to be tuned for specific tasks in practical deployment.
50%