Distilling DeepSeek to GPT-OSS: Why Censorship Doesn't Transfer

A Counterintuitive Finding: Distillation Transfers Capabilities but Loses Censorship

Knowledge Distillation is one of the core techniques for AI model compression and capability transfer today. The basic idea is straightforward: a smaller "student model" learns from the output behavior of a larger or more powerful "teacher model," thereby achieving comparable performance at a lower cost.

Knowledge distillation was formally introduced by Geoffrey Hinton et al. in 2015. Its core insight is to use the teacher model's soft labels—probability distributions rather than hard labels—to guide the student model's training. Soft labels contain relational information between categories (i.e., "dark knowledge"). For example, the teacher model might assign 70% probability to a response being correct and 20% to it being partially correct—this distributional information is far more helpful for the student model than a simple right-or-wrong judgment. In the large language model domain, distillation is typically achieved by having the student model mimic the teacher model's output text across a large set of prompts. This approach is sometimes called "black-box distillation" since it doesn't require access to the teacher model's internal weights.

However, a recent project that sparked discussion on Hacker News raised a counterintuitive observation: When you distill DeepSeek into GPT-OSS, the model's censorship mechanisms do not transfer along with it. In other words, the student model learns the teacher model's capabilities but doesn't inherit its built-in content censorship behavior.

DeepSeek is a large language model series developed by the Chinese company DeepSeek, which has attracted global attention for its cost-effectiveness and open-source strategy. Its latest versions, DeepSeek-V3 and DeepSeek-R1, approach or even surpass GPT-4-level performance on multiple benchmarks while employing a Mixture of Experts (MoE) architecture to reduce inference costs. GPT-OSS is a model variant built on open-source frameworks, aimed at acquiring frontier model capabilities through techniques like distillation. The backdrop of this distillation experiment is that as Chinese models rapidly close the capability gap, whether their built-in compliance censorship mechanisms (refusal behaviors on specific political topics) would propagate through distillation has become a practical technical and governance question.

This finding might seem like a mere technical detail, but it touches on a critical issue in current large model governance and the open-source ecosystem: To what extent can a model's "capabilities" and its "value alignment/censorship constraints" be separated?

What Does Knowledge Distillation Actually Transfer?

The Fundamental Difference Between Underlying Capabilities and Behavioral Constraints

To understand this phenomenon, we first need to distinguish between two types of "knowledge" in a model:

• Underlying capabilities: Language understanding, reasoning, knowledge recall, code generation, etc.—these are general capabilities acquired through massive pretraining.
• Behavioral constraints: Including safety alignment, content censorship, refusal strategies, etc.—these are typically reinforced during post-training (such as RLHF, instruction fine-tuning).

RLHF (Reinforcement Learning from Human Feedback) is the current mainstream alignment approach. Its workflow typically involves three stages: first, supervised fine-tuning (SFT) teaches the model the basic format for following instructions; then a reward model is trained, which scores model outputs based on human annotator preferences; finally, reinforcement learning algorithms like PPO (Proximal Policy Optimization) are used to optimize the model to produce outputs that score higher. Censorship behavior is reinforced precisely through this process—when the model refuses to answer sensitive topics, it receives high rewards, forming a conditioned-reflex-like refusal pattern. This mechanism is essentially a behavioral policy layer stacked on top of the model's existing capabilities.

As a model trained in China's compliance environment, DeepSeek's censorship behavior on certain politically sensitive topics largely stems from alignment data and refusal patterns in the post-training phase. If the distillation process primarily targets the model's output distribution on routine tasks, then these censorship behaviors—which are highly dependent on specific trigger conditions—may "slip through" without being effectively transferred.

Why Censorship Mechanisms Don't Follow Distillation

Censorship behavior typically manifests as "refusal" or "avoidance" patterns triggered by specific inputs. Such behaviors represent an extremely small proportion of the training data and are highly concentrated around specific topics. If the prompt dataset used for distillation doesn't include these sensitive trigger scenarios, the student model naturally won't learn the corresponding refusal strategies.

The design of the prompt dataset used during distillation is crucial—it determines which behaviors the student model gets to "observe" from the teacher model. A typical distillation dataset contains a large volume of routine tasks—mathematical reasoning, code generation, text summarization, multi-turn dialogue, etc.—to ensure comprehensive capability transfer. However, sensitive prompts that trigger censorship behavior represent an extremely small fraction of the overall data distribution (typically far less than 1%), and researchers often actively avoid such controversial content when constructing distillation datasets. This means the student model has virtually no opportunity to observe the teacher model's refusal behavior during training, and naturally cannot learn this pattern. It's similar to a student who has never encountered a certain type of exam question and naturally won't master the corresponding problem-solving strategy.

Furthermore, GPT-OSS as the student model may already carry its own alignment baseline. When it absorbs DeepSeek's capability outputs, its existing behavioral framework takes precedence, making it difficult for the teacher model's specific censorship logic to override it.

Implications for Open-Source Model Governance and AI Safety

Censorship Is Not an Inseparable Part of Model Weights

The most important implication of this project is that model censorship is not an inseparable part of model weights, but rather a relatively separable behavioral policy layer. This means that through distillation, fine-tuning, and similar techniques, researchers can change or remove a model's original behavioral constraints while preserving its capabilities.

In AI safety research, there's the concept of an "alignment tax"—the model performance that must be sacrificed to achieve safety alignment. Research shows that excessive safety training can degrade model performance on normal tasks—for instance, an overly cautious model might refuse to answer completely harmless questions. The phenomenon of distillation transferring capabilities while losing censorship confirms from another angle that tension exists between capabilities and safety constraints: they are encoded differently within the model. Capabilities are more represented as distributed knowledge representations, while censorship is closer to a conditional-trigger shallow strategy. Methods like Anthropic's "Constitutional AI" attempt to embed values more deeply into the training process, precisely to address this separability problem.

For the open-source community, this is a double-edged sword:

• On one hand, it lowers the technical barrier to "de-censoring," allowing researchers to obtain more neutral model behavior with fewer refusals;
• On the other hand, it demonstrates that existing safety alignment mechanisms may be more fragile than imagined—malicious actors could use similar techniques to bypass safety protections.

AI Safety Alignment Needs Deeper Integration

If censorship behavior can be easily lost during distillation, then safety alignment strategies that rely on surface-level post-training appear insufficiently robust. Truly robust alignment may need to be integrated at the pretraining level or at deeper architectural levels, rather than relying solely on stacking refusal samples after the fact.

This is also a frontier direction in current AI safety research: how to make alignment an "intrinsic property" of model capabilities, rather than a shell that can be easily peeled away.

This finding also poses serious challenges to global AI governance frameworks. Current regulatory approaches in most countries rely on model providers performing safety alignment before deployment—a "front door control" strategy. But if downstream users can easily remove these constraints through techniques like distillation, the effectiveness of regulation is significantly undermined. The EU's AI Act, the U.S. AI Executive Order, and China's Generative AI management measures all face similar enforcement dilemmas. Once open-source models release their weights, downstream usage becomes nearly impossible to fully control. This has driven academic research into directions like "irreversible alignment" and "model watermarking"—with the goal of maintaining safety properties even after fine-tuning or distillation.

Experimental Boundaries Worth Keeping in Perspective

As a Show HN project (20 upvotes, 9 comments), its discussion scale is still relatively limited, and the conclusions require more rigorous experimental validation. Several noteworthy questions include:

• Sample coverage: Is the "disappearance" of censorship behavior fundamentally untransferable, or is it simply because the distillation dataset didn't cover trigger scenarios?
• Measurement standards: How to systematically measure a model's "degree of censorship"—there is currently no unified benchmark.
• Reproducibility: Different distillation configurations, temperature parameters, and data scales might yield different conclusions.

Therefore, this should be viewed more as an experimental observation that provokes thought rather than a definitive conclusion.

Conclusion: The Separability of Capabilities and Constraints Deserves Deep Investigation

The finding that "distillation doesn't transfer censorship" fundamentally reveals the separability of capabilities and constraints in large models. It reminds us that a model's behavioral boundaries are far more fragile and dynamic than its weight files. For the open-source ecosystem, model governance, and AI safety research, this is both a technical proposition worth deep exploration and a risk signal that must be taken seriously.

As open-source model capabilities continue to approach closed-source frontiers, how to build truly robust alignment mechanisms while preserving openness will become a core challenge that the entire industry cannot avoid.