[KongchangAI]
ConceptPrivileged Context

Privileged Context

Additional information visible only to the teacher in self-distillation (such as source material, answers or domain documents), used to compute the teacher's next-token distribution, which is compared with the student model's unconditional distribution to determine supervision signals.

Core Facts

Timeline (last 90 days)

Nov 17

LEAF-OPD gives the teacher model an additional instruction (privileged context) on forget data to guide the student to learn to refuse, while on retain data the teacher does not use privileged context; the student always runs without privileged context.

Unverified50%
Nov 10

Privileged context in self-distillation refers to additional information visible only to the teacher, such as source material, answers, or domain documents.

Unverified50%
Oct 12

Squint OPSD adopts a teacher-student distillation paradigm, where the teacher model receives a downsampled image view and is given the ground-truth text transcription as privileged context.

Unverified50%
Oct 12

The concept of privileged context originates from the Learning Using Privileged Information (LUPI) framework, which provides additional information during training that is unavailable at inference time to improve learning.

Verified75%

All Facts (4)

Source Articles