OPRD
A training method for long-horizon coding agents. It introduces reminder signals at two granularities (turn-local and global) during training and adopts an outcome-based routing mechanism: successful rollouts undergo self-imitation learning, while failed rollouts receive reminder-conditioned reverse KL supervision, improving long-horizon task ability without relying on inference-time reminders.
Timeline (last 90 days)
OPRD (On-Policy Reminder Distillation) is a new method that addresses, at the level of the training mechanism, the problems of agents dropping constraints and ignoring context in long-horizon coding tasks.
The paper's diagnostic analysis finds that coding agents often fail not because of missing information, but because the relevant information is already in the context and the model does not act on it.
OPRD's design idea is to give reminder signals during training and completely remove the reminders at inference time, so that the student model ultimately learns to complete long-horizon tasks independently without reminders.
OPRD adopts outcome-based routing: successful rollouts follow an autoregressive self-imitation path, and failed rollouts receive reminder-conditioned reverse KL supervision.
OPRD's teacher and student share the same two-stage initialization: first Coding SFT, then Instruction-Following SFT.
Relative to the open-source base model Qwen3.5-4B, OPRD improves the average task resolution rate by 11.3 percentage points across the three benchmarks SWE-bench Verified, SWE-bench Pro, and SWE-bench Multilingual.
Under the same diagnostic protocol, OPRD reduced the number of aggregated error events from 446 to 222, a decrease of 50.2%.
All Facts (7)
OPRD adopts outcome-based routing: successful rollouts follow an autoregressive self-imitation path, and failed rollouts receive reminder-conditioned reverse KL supervision.
50%UnverifiedThe paper's diagnostic analysis finds that coding agents often fail not because of missing information, but because the relevant information is already in the context and the model does not act on it.
50%UnverifiedOPRD's design idea is to give reminder signals during training and completely remove the reminders at inference time, so that the student model ultimately learns to complete long-horizon tasks independently without reminders.
50%UnverifiedUnder the same diagnostic protocol, OPRD reduced the number of aggregated error events from 446 to 222, a decrease of 50.2%.
50%UnverifiedOPRD's teacher and student share the same two-stage initialization: first Coding SFT, then Instruction-Following SFT.
50%UnverifiedRelative to the open-source base model Qwen3.5-4B, OPRD improves the average task resolution rate by 11.3 percentage points across the three benchmarks SWE-bench Verified, SWE-bench Pro, and SWE-bench Multilingual.
50%UnverifiedOPRD (On-Policy Reminder Distillation) is a new method that addresses, at the level of the training mechanism, the problems of agents dropping constraints and ignoring context in long-horizon coding tasks.
50%