RoCo-PRM
An independent process reward model obtained by automatically labeling step scores using the routing statistics of a strong MoE model and distilling them. It requires no manual process labels and can transfer across model architectures and tasks.
Timeline (last 90 days)
The researchers propose the RoCo-PRM method, which uses a strong MoE model to automatically annotate reasoning trajectories with routing-based step scores, then distills these scores into an independent PRM.
Experiments show that RoCo-PRM surpasses baselines on multiple tasks and can effectively transfer across model architectures and tasks.
In RoCo-PRM, the teacher is a strong MoE model whose soft labels come from step scores computed from routing statistics rather than output logits; the student is an independent PRM.
All Facts (3)
The researchers propose the RoCo-PRM method, which uses a strong MoE model to automatically annotate reasoning trajectories with routing-based step scores, then distills these scores into an independent PRM.
50%UnverifiedExperiments show that RoCo-PRM surpasses baselines on multiple tasks and can effectively transfer across model architectures and tasks.
50%UnverifiedIn RoCo-PRM, the teacher is a strong MoE model whose soft labels come from step scores computed from routing statistics rather than output logits; the student is an independent PRM.
50%