Unverified50% confidenceBenchmarkExact time
In next-word prediction comparison experiments on the CNN/DailyMail dataset, ILRN and Transformer each had about 532K parameters, were each trained on 20 million next-word targets, and were trained independently under two context windows of 96 and 256.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe model is trained in two stages: first masked language model pre-training on about 20 million sentences from FineWeb-Edu, then correction fine-tuning on 12.02 million (noisy, clean) pairs, of which 30% are identity pairs.73% similarUnverifiedOn the CNN2-with-pruning task, trainable parameters were compressed to 204, with an accuracy of 81.25%.71% similarUnverifiedThe 29-million-parameter encoder trained by the researchers achieved a popularity-weighted cross-fold AUC of only 0.486 on 985 statements, indistinguishable from the corpus prior.69% similarUnverifiedAfter 15 training epochs on the demonstration dataset, the text classification model achieves an accuracy of 99%, with a total of 110,000 model parameters (16,000 for Attention, 33,000 for FFN, and the remainder for Embedding)68% similarUnverifiedFine-tuning on a few hundred real data samples on top of synthetic pre-training can often improve mAP by 10 to 20 percentage points.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013014API
curl https://kongchang.com/api/v1/knowledge/claims/1013014MCP
get_claim(id=1013014)