Expired60% confidenceFactTime unknown
The pre-training stage uses Cross-Entropy Loss to measure the gap between the predicted distribution and the actual next token.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Complete Guide to LLM Training: Pre-training, SFT Fine-tuning, and Preference Alignment Explained
bilibili小全栈
Related Claims
VerifiedCross-entropy loss originates from KL divergence in information theory and measures the degree of difference between the true probability distribution and the predicted probability distribution.71% similarUnverifiedThe training objective of large language models is to minimize the prediction loss of the next token (cross-entropy loss), not to verify the factual accuracy of generated content.70% similarUnverified批量梯度下降使用全部训练数据计算梯度，随机梯度下降每次只用一个样本，小批量梯度下降在两者之间取得平衡70% similarVerifiedCross-Entropy Loss is commonly used for classification tasks, and Mean Squared Error (MSE) is commonly used for regression tasks.68% similarUnverified如果交叉熵稳步下降而Dice波动剧烈，问题几乎可以确定出在Dice损失的配置上68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/55195API
curl https://kongchang.com/api/v1/knowledge/claims/55195MCP
get_claim(id=55195)