Cross-Entropy
A quantity in information theory that measures the difference between two probability distributions. In LLM decoding it is used as a constraint to limit the degree to which sampling deviates from the correct answer.
Timeline (last 90 days)
With sigmoid activation combined with cross-entropy loss, differentiating the loss with respect to the weights simplifies to ∂L/∂w = (ŷ - y) · x.
The forward propagation process of logistic regression is: linear combination z = w·x + b, activated through sigmoid as ŷ = 1/(1+e^(-z)), followed by cross-entropy loss calculation.
MSE suffers from gradient saturation when Sigmoid outputs are close to 0 or 1, causing training to be extremely slow, whereas cross-entropy has larger gradients when predictions deviate severely
该项目将博弈问题建模为监督分类任务，神经网络输入为编码后的棋盘状态，输出为9个格子中最佳落子位置的概率分布，通过交叉熵损失学习模仿Minimax决策
交叉熵损失函数 H(p,q) = -Σp(x)log q(x) 的最小值在q=p时取到，等于真实分布的熵
当模型对某个训练样本的预测概率接近1时，该样本对应的交叉熵损失趋近于零，模型几乎无法获得梯度更新信号
监督学习本质上是函数逼近问题，通过最小化损失函数实现，常见损失函数包括回归任务的均方误差（MSE）和分类任务的交叉熵（Cross Entropy）
All Facts (7)
With sigmoid activation combined with cross-entropy loss, differentiating the loss with respect to the weights simplifies to ∂L/∂w = (ŷ - y) · x.
50%UnverifiedThe forward propagation process of logistic regression is: linear combination z = w·x + b, activated through sigmoid as ŷ = 1/(1+e^(-z)), followed by cross-entropy loss calculation.
50%UnverifiedMSE suffers from gradient saturation when Sigmoid outputs are close to 0 or 1, causing training to be extremely slow, whereas cross-entropy has larger gradients when predictions deviate severely
50%Unverified该项目将博弈问题建模为监督分类任务，神经网络输入为编码后的棋盘状态，输出为9个格子中最佳落子位置的概率分布，通过交叉熵损失学习模仿Minimax决策
50%Unverified交叉熵损失函数 H(p,q) = -Σp(x)log q(x) 的最小值在q=p时取到，等于真实分布的熵
50%Unverified当模型对某个训练样本的预测概率接近1时，该样本对应的交叉熵损失趋近于零，模型几乎无法获得梯度更新信号
50%Unverified监督学习本质上是函数逼近问题，通过最小化损失函数实现，常见损失函数包括回归任务的均方误差（MSE）和分类任务的交叉熵（Cross Entropy）
50%