logits
The unnormalized output vector produced by a neural network before the softmax layer. It contains the model's raw confidence for each class and is the most valuable supervision signal for distillation attacks.
Timeline (last 90 days)
In autoregressive language models, logits are the unnormalized scores the model gives to each token in the vocabulary, which only become probabilities after softmax.
In distillation attacks, if the API exposes logits or temperature-scaled softmax outputs, attackers can train a substitute closer to the original model with fewer queries.
Reducing the amount of information returned by the API (such as returning only the final text result without exposing logits distributions, top-k probabilities, or token-level confidence) can significantly increase the difficulty of distillation attacks, because the supervision signal available to the attacker becomes sparser.
All Facts (4)
In autoregressive language models, logits are the unnormalized scores the model gives to each token in the vocabulary, which only become probabilities after softmax.
50%UnverifiedIn distillation attacks, if the API exposes logits or temperature-scaled softmax outputs, attackers can train a substitute closer to the original model with fewer queries.
50%UnverifiedReducing the amount of information returned by the API (such as returning only the final text result without exposing logits distributions, top-k probabilities, or token-level confidence) can significantly increase the difficulty of distillation attacks, because the supervision signal available to the attacker becomes sparser.
50%Unverified约束解码通过维护由JSON Schema推导的有限状态机，在token采样前将非法token的logit置为负无穷，使JSON结构从统计概率问题升级为数学确定性保证
50%