Pathwise Derivative
A gradient estimation method that represents random sampling as a combination of a deterministic function and independent noise and directly differentiates the probabilistic output. It has lower variance than REINFORCE and is more stable in decision-head optimization.
Timeline (last 90 days)
Reparameterization tricks (such as Gumbel-Softmax or pathwise derivatives) allow sampling from a continuous distribution while maintaining gradient flow, thereby embedding the rule-satisfaction probability into end-to-end training.
Using probability as the loss and optimizing via the pathwise derivative avoids the performance collapse caused by the REINFORCE reward; using the REINFORCE reward r or r minus p achieves a similar effect.
All Facts (2)
Reparameterization tricks (such as Gumbel-Softmax or pathwise derivatives) allow sampling from a continuous distribution while maintaining gradient flow, thereby embedding the rule-satisfaction probability into end-to-end training.
50%UnverifiedUsing probability as the loss and optimizing via the pathwise derivative avoids the performance collapse caused by the REINFORCE reward; using the REINFORCE reward r or r minus p achieves a similar effect.
50%