Dark Knowledge
A core concept in knowledge distillation referring to the inter-class relational information embedded in the soft-label probability distributions output by a teacher model. It goes beyond the binary correct/incorrect judgment of hard labels and reflects the model's implicit understanding of similarities among different classes.
Timeline (last 90 days)
Soft labels are the probability distribution output by the teacher model after softmax normalization, carrying similarity information between classes, and their information content far exceeds that of a single hard label.
暗知识（tacit knowledge）按Michael Polanyi的定义是'我们知道但无法言说'的部分，很难被AI从训练数据中学到
All Facts (2)
Soft labels are the probability distribution output by the teacher model after softmax normalization, carrying similarity information between classes, and their information content far exceeds that of a single hard label.
50%Unverified暗知识（tacit knowledge）按Michael Polanyi的定义是'我们知道但无法言说'的部分，很难被AI从训练数据中学到
50%