Unverified50% confidenceBenchmarkExact time
On 1024-step sequences, the average gradient of the initial token for PLUG and ILRM remains stably at around the 10⁻⁴ order of magnitude, while the standard GRU gradient drops as low as approximately 10⁻¹⁴⁵, resulting in numerical underflow.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
UnverifiedINT4量化将原本由65536个不同值（FP16）表示的权重压缩到仅16个离散级别58% similarUnverifiedDevelopers ultimately selected the variant with the lowest KL divergence (0.028), resulting in a 1.83% drop on GSM8K, no change on MMLU, and an improvement in HarmBench compliance rate from 30.8% to 100%56% similarUnverifiedW4A4量化将权重和激活值均压缩至4位整数，理论上相比FP16可将显存占用降低至约1/456% similarVerifiedThe maximum value of the derivative of the Sigmoid function is only 0.25, leading to the vanishing gradient problem in deep networks.56% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/902251API
curl https://kongchang.com/api/v1/knowledge/claims/902251MCP
get_claim(id=902251)