Unverified50% confidenceFactExact time
In language model training, minimizing cross-entropy and maximizing likelihood are two ways of saying the same thing, and CrossEntropyLoss in PyTorch directly computes the negative log-likelihood.
1
Sources
50%
Confidence
Long-term
Relevance
10/31/2026
First Seen
Sources
LLM for Absolute Beginners: A Complete Learning Path from Principles to Practice
bilibili数据拾光屋10/20/2026
Related Entities
Related Claims
UnverifiedThe model uses binary cross-entropy loss and is trained end-to-end against oracle extractive labels.75% similarUnverifiedThe concept of perplexity originates from entropy (Shannon Entropy) in information theory, and is essentially the average branching factor per prediction step of a language model.74% similarVerifiedThe objective function for pre-training is typically the cross-entropy loss in causal language modeling, i.e., maximizing the probability of correctly predicting the next token given all preceding tokens.71% similarVerifiedThe semantic entropy method is technically implemented by obtaining candidate answers through multiple samplings, using an NLI model to determine semantic equivalence for clustering, and then computing entropy over the semantic categories.71% similarVerifiedModern large language models are based on the Transformer architecture and generate text by predicting the probability distribution of the next token, with the training objective of minimizing cross-entropy loss70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1121466API
curl https://kongchang.com/api/v1/knowledge/claims/1121466MCP
get_claim(id=1121466)