Unverified50% confidenceBenchmarkExact time
After UltraBERT distillation, the 380M model improved from 90.2 to 90.8 on GLUE, but under high masking rates it instead harmed retrieval performance.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe 35B model achieved a pass rate of 90/90 under the constrained context, while the 4B model achieved 72/90.63% similarUnverifiedResearch has shown that in certain models, ReLU² can raise the sparsity rate to above 90%.61% similarUnverifiedScaling the model size by approximately one hundred thousand times improved the base LLM's score on ARC from only 0% to approximately 10%.58% similarUnverifiedAt epoch 290, the loss rebounded (+0.002), indicating that the model may have approached the performance ceiling of its current configuration.58% similarUnverifiedA higher physical ceiling on the output window does not mean the effective output length increases proportionally; a model that can generate 1 million tokens but starts hallucinating at 200K tokens has less practical value than one that stably outputs 300K tokens.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1044468API
curl https://kongchang.com/api/v1/knowledge/claims/1044468MCP
get_claim(id=1044468)