Unverified50% confidenceFactExact time
Low-bit Adam collapses in rare-input scenarios because the gradient signal produced by rare samples is weak, and after coarse-graining the momentum the weak signal is overwhelmed, causing the corresponding parameters to fail to update.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFine-tuning on data with narrow defects can sometimes cause models to become broadly misaligned.72% similarUnverifiedLosing the prefill checkpoint at the end of the prompt under sparse sampling conditions can cause state corruption, output deviation, and even non-reproducible generation results.72% similarUnverifiedWhen all sampled outputs in a group are incorrect, the group-relative advantage collapses to zero, causing the model to receive no effective gradient feedback in high-uncertainty scenarios.70% similarUnverifiedA severely overspecified model parameter count will cause an OOM (out-of-memory) error and prevent startup, while a slightly overspecified count will borrow regular memory and cause a significant drop in speed69% similarUnverifiedHigh bias means the model is too simple and leads to underfitting, while high variance means the model has learned noise and leads to overfitting, and the two tend to trade off against each other.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990625API
curl https://kongchang.com/api/v1/knowledge/claims/990625MCP
get_claim(id=990625)