Unverified50% confidenceFactExact time
Under extreme quantization, the tail differences between P and Q are amplified, and the drift term may dominate the total gradient, manifesting as a sudden divergence of the loss or a collapse of the policy.
1
Sources
50%
Confidence
Long-term
Relevance
10/26/2026
First Seen
Sources
Hugging Face Journal Club: How Score Centering Stabilizes Offline Reinforcement Learning
youtubeHugging Face10/26/2026
Related Entities
Related Claims
UnverifiedThe mismatch between P and Q introduces a drift term, causing model weights to be incorrectly updated even without a learning signal.72% similarVerifiedGradient descent iteratively adjusts weights in the opposite direction of the loss function's gradient, causing the prediction error to converge progressively.72% similarUnverifiedTheoretical last-iterate convergence guarantees for Q-Learning have long lagged behind averaging-iterate analyses, as well as analogous results for methods such as policy gradient.71% similarUnverifiedHigh UTD exacerbates the deadly triad problem (instability arising from the coexistence of function approximation, bootstrapping, and off-policy learning), which can lead to Q-value divergence or value function collapse.71% similarUnverifiedGradient weighting on high-similarity negatives persistently harms downstream generalization, i.e., excessively pushing the model away from certain samples disrupts the overall structure of the embedding space, so retrieval performance drops rather than rises (termed 'gradient distortion').70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1100996API
curl https://kongchang.com/api/v1/knowledge/claims/1100996MCP
get_claim(id=1100996)