SDAE
Subtask Decomposition Advantage Estimator, the core component of RLDS. It splits trajectory-level rewards according to a subtask taxonomy, computes group-relative advantages at the subtask level, and achieves token-level credit assignment through reflection tokens, serving as a replacement for the scalar GRPO advantage function.
Timeline (last 90 days)
The core component of RLDS is the Subtask-Decomposed Advantage Estimation (SDAE), designed as a replacement for the scalar GRPO advantage function.
SDAE operates in three steps: decomposing rewards according to a fixed subtask taxonomy, computing group-relative advantages per subtask, and assigning token-level credit weighted by importance.
SDAE uses reflection tokens to locate which steps have a critical influence on the execution of a given subtask, directing gradients to concentrate at those positions.
All Facts (3)
The core component of RLDS is the Subtask-Decomposed Advantage Estimation (SDAE), designed as a replacement for the scalar GRPO advantage function.
50%UnverifiedSDAE operates in three steps: decomposing rewards according to a fixed subtask taxonomy, computing group-relative advantages per subtask, and assigning token-level credit weighted by importance.
50%UnverifiedSDAE uses reflection tokens to locate which steps have a critical influence on the execution of a given subtask, directing gradients to concentrate at those positions.
50%