Unverified50% confidenceFactExact time
The model is trained in two stages: first masked language model pre-training on about 20 million sentences from FineWeb-Edu, then correction fine-tuning on 12.02 million (noisy, clean) pairs, of which 30% are identity pairs.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedAfter 15 training epochs on the demonstration dataset, the text classification model achieves an accuracy of 99%, with a total of 110,000 model parameters (16,000 for Attention, 33,000 for FFN, and the remainder for Embedding)77% similarUnverified实验数据显示，经过5-10轮迭代训练后，模型在低频词汇和长尾知识上的覆盖率下降超过40%，而输出文本在表面流畅性上几乎没有可察觉的退化71% similarUnverifiedThe 29-million-parameter encoder trained by the researchers achieved a popularity-weighted cross-fold AUC of only 0.486 on 985 statements, indistinguishable from the corpus prior.70% similarUnverifiedThe 9-billion-parameter model in the paper lagged 6 points behind a clean summarization baseline before being trained with reinforcement learning.69% similarUnverifiedFine-tuning on a few hundred real data samples on top of synthetic pre-training can often improve mAP by 10 to 20 percentage points.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1009455API
curl https://kongchang.com/api/v1/knowledge/claims/1009455MCP
get_claim(id=1009455)