Unverified50% confidenceFactExact time
SFT mainly teaches the model format and persona rather than knowledge; LIMA expanded examples from 2,000 to 32,000, with almost no change in quality.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
Unverified典型的SFT数据集可能只有数万到数十万条样本，远小于预训练数据规模66% similarUnverifiedThe teacher model's prediction quality is not uniform across different time steps; at some time steps (especially at the endpoints where t is close to 0 or close to 1) the predictions themselves are relatively noisy.57% similarUnverifiedThe figure "pretraining throughput reaching 170% of a single H100" is jointly affected by batch size, sequence length, model scale, precision format, and interconnect bandwidth, and is not a general conclusion; the ratio may change with a different model or hyperparameters.56% similarUnverified权重绑定在中小规模模型上(如GPT-2 124M)通常能提升性能并减少约30%的嵌入相关参数，但在超大规模模型中收益递减55% similarUnverifiedExperiments conducted by the research team on models such as GPT-2 showed that after only 5–9 generations of recursive training, the quality of model outputs degrades significantly.54% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1051004API
curl https://kongchang.com/api/v1/knowledge/claims/1051004MCP
get_claim(id=1051004)