ASC
Authored-Sentence Credit is a reward attribution mechanism for long-form RAG training. It moves reinforcement learning advantage values from the answer granularity down to the sentence granularity, requiring that a sentence be written by the model, provide more information than the question, and be supported by retrieved passages, thereby preventing reward acquisition through copying source text.
Timeline (last 90 days)
The paper proposes Authored-Sentence Credit (ASC), which moves GRPO's reward and advantage value from the answer granularity down to the sentence granularity, and independently judges whether each sentence deserves reward.
Under the ASC mechanism, a sentence must simultaneously satisfy three conditions to receive reward: it was written by the model rather than copied from the source text, it provides more information than the question itself, and it is supported by the retrieved passages.
The study compared on four models, two in-domain corpora, and three held-out corpora, and the results show that only ASC simultaneously reduces the 'authored sentence' error rate and increases supported sentences.
Under the held-out evaluator, ASC's median error rate drops by 38%, the median improvement in supported sentences reaches 46%, and this holds in all test settings.
Two LLM judges, when comparing full answers, found that no baseline method beats the original base model, while ASC's answers were rated as 'on par' with the base model.
Most previous mainstream faithfulness defense schemes fall into the dilemma of 'either degrade or refuse', while ASC bypasses this trap.
All Facts (6)
Under the ASC mechanism, a sentence must simultaneously satisfy three conditions to receive reward: it was written by the model rather than copied from the source text, it provides more information than the question itself, and it is supported by the retrieved passages.
50%UnverifiedThe study compared on four models, two in-domain corpora, and three held-out corpora, and the results show that only ASC simultaneously reduces the 'authored sentence' error rate and increases supported sentences.
50%UnverifiedUnder the held-out evaluator, ASC's median error rate drops by 38%, the median improvement in supported sentences reaches 46%, and this holds in all test settings.
50%UnverifiedTwo LLM judges, when comparing full answers, found that no baseline method beats the original base model, while ASC's answers were rated as 'on par' with the base model.
50%UnverifiedMost previous mainstream faithfulness defense schemes fall into the dilemma of 'either degrade or refuse', while ASC bypasses this trap.
50%UnverifiedThe paper proposes Authored-Sentence Credit (ASC), which moves GRPO's reward and advantage value from the answer granularity down to the sentence granularity, and independently judges whether each sentence deserves reward.
50%