Implicit Reward Margin Magnitude
In a preference optimization model, the absolute value of the difference in implicit reward scores between a pair of responses. It measures how well the model distinguishes the pair and can serve as a proxy for selecting effective ties.
Timeline (last 90 days)
The magnitude of the implicit reward margin of a tie is used as a proxy for its predicted influence; under certain conditions, the reward margin magnitude is positively correlated with the influence magnitude.
The implicit reward margin magnitude refers to the difference between the reward scores implicitly assigned by the model to a pair of responses in a preference optimization model (such as the Bradley-Terry model), i.e., |r(A)-r(B)|.
All Facts (2)
The magnitude of the implicit reward margin of a tie is used as a proxy for its predicted influence; under certain conditions, the reward margin magnitude is positively correlated with the influence magnitude.
50%UnverifiedThe implicit reward margin magnitude refers to the difference between the reward scores implicitly assigned by the model to a pair of responses in a preference optimization model (such as the Bradley-Terry model), i.e., |r(A)-r(B)|.
50%