RLEC
A reinforcement-learning-based training method that enables a single language model to dynamically decide whether to answer or abstain according to an explicit confidence target (p-value) specified by the user, achieving behavioral calibration by preserving discriminative learning signals and introducing a stable reward family and reward scheduling.
Timeline (last 90 days)
The RLEC method is the first to achieve responding to explicit confidence targets with a single model.
The paper attributes the inability of prior methods to distinguish different confidence targets to two problems: the reward signals that distinguish different p values are often lost during optimization, and the raw error penalty grows unboundedly as p approaches 1.
RLEC proposes three key designs: preserving the learning signal needed to distinguish confidence targets, introducing a stable reward family, and reward scheduling.
The study conducted experiments on multiple benchmarks and across different model scales, and the results show that RLEC largely preserves the model's general capabilities while achieving competitive calibration performance.
After training, the RLEC-trained model can directly respond to different confidence requirements without an external decision system or post-hoc threshold calibration.
All Facts (5)
The RLEC method is the first to achieve responding to explicit confidence targets with a single model.
50%UnverifiedThe paper attributes the inability of prior methods to distinguish different confidence targets to two problems: the reward signals that distinguish different p values are often lost during optimization, and the raw error penalty grows unboundedly as p approaches 1.
50%UnverifiedRLEC proposes three key designs: preserving the learning signal needed to distinguish confidence targets, introducing a stable reward family, and reward scheduling.
50%UnverifiedThe study conducted experiments on multiple benchmarks and across different model scales, and the results show that RLEC largely preserves the model's general capabilities while achieving competitive calibration performance.
50%UnverifiedAfter training, the RLEC-trained model can directly respond to different confidence requirements without an external decision system or post-hoc threshold calibration.
50%