Qwen 3.8 27B Uncensored Model Comparison: 167 GPU-Hours of Testing Reveals the Truth

The Most Comprehensive Qwen Uncensored Model Comparison

An 11-day, 167 GPU-hour large-scale test reveals the true performance of Qwen 3.8 27B uncensored variants. The Abliterlitics team rigorously validated 8 model variants on HuggingFace claiming to be "uncensored," exposing the gap between actual capabilities and advertised claims through weight comparison, KL divergence measurement, 13 benchmark tests, and the HarmBench 400 classic test set.

Uncensored models are model variants that use post-processing techniques to remove refusal behaviors implanted during RLHF (Reinforcement Learning from Human Feedback) and safety fine-tuning stages. Mainstream model vendors typically train models to refuse harmful content requests through techniques like Constitutional AI and DPO (Direct Preference Optimization) before release. The uncensored community attempts to reverse these safety layers, motivated by academic research, creative writing, and philosophical stances on AI free expression. The core technical challenge in this domain is: how to remove refusal behaviors while maintaining the model's reasoning capabilities and knowledge integrity.

This is the most systematic uncensored modification comparison study to date for a single model family. The test results not only exposed inaccurate model card information but also uncovered a previously overlooked issue: at the 27B parameter scale, overly aggressive editing causes models to fall into "thought loops," unable to complete normal reasoning.

Top Eight Ranking: Surgical Editing Beats Brute Force Modification

The HarmBench Attack Success Rate (ASR) ranking shows that editing precision matters more than editing intensity. HarmBench is a standardized red-teaming framework jointly developed by top research institutions, containing approximately 400 carefully designed harmful request samples covering chemical/biological hazards, cyberattacks, harassment, misinformation, copyright infringement, and more. Its core metric ASR (Attack Success Rate) measures the proportion of times a model "cooperates" in answering adversarial requests and has become the de facto standard for evaluating model safety alignment:

orcarouter (82.2%)—the undisputed champion. Uses Arditi-style single-vector editing, modifying only 131 matrices in layer 38, and is the only model where all claims perfectly match actual weights. Copyright content unlock rate reaches 39%, the highest among all variants.

Arditi-style editing originates from Andy Arditi et al.'s 2024 research, with the core finding that refusal behavior in large language models can be localized to a single directional vector in the residual stream. By subtracting this "refusal direction" from weight matrices in specific Transformer layers, safety refusal tendencies can be precisely eliminated with almost no impact on other capabilities. The elegance of this method lies in requiring minimal weight modifications—orcarouter modified only 131 matrices in layer 38, equivalent to "surgical" intervention on an extremely small fraction of hundreds of billions of parameters.

apostate (78.7%)—the best value choice. Uses the novel KCRN method with only 41 actual edits, KL divergence as low as 0.0439 (close to the original model), and nearly perfect capability retention. However, it has packaging issues: after re-saving as a pure text model, it loses visual capabilities and multi-task pretraining data, with precision downgraded to FP16.

KCRN (Knowledge-Conditioned Refusal Neutralization) is a newer uncensored technical approach that differs from traditional abliteration methods by neutralizing refusal behavior through conditioned knowledge editing. KCRN's core concept is identifying knowledge nodes responsible for "refusal decisions" in the model and making targeted minimal interventions. Apostate achieved 78.7% ASR with only 41 actual edits, demonstrating KCRN's significant advantage in precisely locating refusal-related parameters. However, this method is still in its early stages, and its generalization ability across different model architectures remains to be validated.

KL divergence (Kullback-Leibler Divergence) is a classical information theory metric measuring the difference between two probability distributions. In language model evaluation, it's used to measure the deviation in output probability distribution between a modified model and the original. Lower KL divergence indicates more "conservative" modifications and better preservation of the model's original behavior. Apostate's KL divergence of only 0.0439 means its output distribution is nearly identical to the original Qwen model, explaining why it maintains near-perfect capability levels while removing censorship. Note that KL divergence calculations are sensitive to sampling strategy, calibration dataset, and floating-point precision, requiring caution when comparing across environments.

huihui (75.6%)—a reliable representative of classical methods. Achieves clean unlock across all domains except copyright (only 3%).

ultra_heretic (70.5%)—Heretic v2 combined with MPOA method. Functions normally but shows serious decline in truthfulness evaluation (second only to obliteratus) and produces 118 soft refusals.

coder3101 (70.0%)—original Heretic method. The model card claims "weakest removal effect" (33 refusals in 100 tests), but actual measurements show only 5 explicit refusals in 400 tests—the model card description is overly conservative.

blackfrost (68.5%)—closed-source method with serious integrity issues. Weight analysis shows only single-vector editing with 100% rank-1 modifications, completely contradicting its claimed "multi-vector direction bank" (rank-k direction bank). In linear algebra, a rank-1 matrix is the simplest form of non-zero matrix, expressible as the outer product of a column vector and a row vector. When weight analysis shows modifications are "100% rank-1," this means each modified weight matrix was adjusted along only one direction—the characteristic signature of classic single-vector abliteration. If a "multi-vector direction bank" were truly used, the rank of modified matrices should be greater than 1 (rank-k, k>1), as the superposition of multiple directions produces higher-rank modification patterns. This finding constitutes direct technical evidence of the contradiction between blackfrost's method claims and actual implementation.

More alarmingly, the model embeds a 1457-character jailbreak prompt in its chat template, meaning every user input is secretly injected with modified instructions, yet the README mentions nothing about this.

obliteratus (63.9%)—explicitly not recommended. Modified 841 out of 850 tensors, the most aggressive editing among all variants. The cost is severe: 44.8% of HarmBench responses never complete the thinking phase, and it's the only variant showing significant degradation in intelligence tests.

trohrbaugh (57.5%)—ranks last because censorship removal is insufficient. 122 explicit refusals with the highest residual alignment, but has the cleanest capability retention curve. The tester admits "this is the version I use myself, and it performs well"—for scenarios requiring safety boundaries, this is actually an advantage.

base (4.5%)—the base model serves as a reference baseline, with nearly zero cooperation in chemical/biological, harassment, harmful content, and copyright domains.

Key Finding: The Thought Loop Trap

Qwen 3.8 adopts a think-before-answer architecture, which exposes new problems in uncensored modifications. Qwen 3.8's chain-of-thought architecture requires the model to perform explicit reasoning before outputting the final answer, unfolding internal thought processes within <think> tags. This design was originally intended to improve accuracy on complex reasoning tasks but produces unexpected side effects in uncensored scenarios. When safety alignment is partially removed, the reasoning module and residual safety mechanisms may conflict: the model "wants" to answer but residual safety signals try to prevent it, causing the reasoning process to loop repeatedly in contradiction. This "thought loop" phenomenon doesn't occur in traditional non-CoT models and is unique to the Qwen 3 series after introducing reasoning architecture.

Testing found that aggressively edited models fall into infinite thinking loops in up to 45% of responses when handling adversarial questions, failing to output answers even after reaching the 15,360 token budget limit. The 15,360 token budget limit means a failed response may consume significant computational resources without producing any useful content.

The judging system reads the complete thought trajectory, so "cooperation" within loops still counts toward success rate. But a model that only delivers results in unterminated monologues has no practical value.

Interestingly, models that loop over 40% on HarmBench can complete normally on GSM8K math reasoning tests—all variants differ from the base model by within 1.2 percentage points. This indicates that conventional math reasoning capability has converged, but adversarial deep thinking has not yet stabilized. GSM8K is a benchmark test set containing approximately 8,500 elementary school math word problems, commonly used to evaluate language models' basic reasoning capabilities. Math reasoning tasks don't involve safety refusal mechanisms, so modified models' performance on such tasks serves as a reference indicator for "capability retention."

Copyright Becomes the New Universal Barrier

The testing reveals a significant shift in censorship boundaries: no model's copyright content unlock rate exceeds 39%, with 5 of 9 models at 3.2% or below. Conversely, the historically most difficult chemical/biological category has become the easiest domain to breach. Regulatory pressure and changes in training strategies are redefining what constitutes "untouchable" content types.

There are deeper reasons behind this change. As generative AI copyright controversies in creative industries intensify—from Getty Images v. Stability AI to The New York Times v. OpenAI—model vendors have invested more safety training resources in copyright protection. Unlike chemical/biological safety alignment which mainly relies on refusal training during the RLHF phase, copyright protection is often implemented simultaneously in pretraining data filtering, fine-tuning data screening, and multi-layer safety policies, making it extremely difficult to remove copyright restrictions through post-processing weight editing alone.

Chat Templates: An Underestimated Behavior Control Lever

Blackfrost's embedded jailbreak prompts in chat templates prompted the team's first chat template forensic analysis.

Chat templates are the rule definitions by which large language models convert user input into the actual token sequences the model receives, usually stored in Jinja2 template format in the tokenizer configuration. They define separators and formats between system prompts, user messages, and assistant replies. Since chat templates are automatically applied during inference, users typically don't check their contents, making them a covert behavior control point. Blackfrost's practice of injecting jailbreak prompts into templates essentially injects a hidden system-level instruction before each conversation begins, similar to "prompt injection attacks" but proactively implemented by the model publisher. The problem with this approach is not only lack of transparency but also affects all downstream use cases, including normal uses that don't require uncensoring.

Forensic analysis results:

• blackfrost: Injects 1457-character jailbreak prompt
• obliteratus: Disables chain-of-thought functionality
• ultra_heretic: Removes official reasoning effort prompt

The team retested all models after fixing a standard template, because chat templates often have greater impact on behavior than weight modifications. This also explains why some models show huge differences between user experience and benchmark test results.

Model Card Integrity Audit

Completely honest: orcarouter's 4 claims perfectly match measurements; trohrbaugh's KL divergence error is only 9% from claimed value.

Technical errors: Other models have larger KL divergence deviations, but this isn't intentional fraud—KL divergence measurements vary with CUDA version, hardware, and computation methods, only suitable for intra-batch comparison.

Unreproducible: obliteratus honestly disclosed modification methods, but its "0% refusal, 0% evasion" claims cannot be verified. In actual tests, 44.8% of HarmBench tests fell into thought loops, and reasoning analysis found 142 evasive behaviors.

Information concealment: blackfrost claims to use an "internal direction bank" (implying multi-vector modifications), but weight analysis only found single-vector changes. More seriously, the README completely fails to disclose the modified chat template.

Surgical vs Carpet Bombing: Precise Editing Is the Right Direction

This comparison validates a core conclusion once again: in uncensored modifications, precise editing far surpasses brute force modification. The top two models both adopt validated minimal editing strategies, while the most aggressive editing approach (obliteratus) ranks second to last. At the 27B parameter scale, extensive weight modifications only trap models in thought loops.

This conclusion aligns with broader research trends in machine learning. From LoRA (Low-Rank Adaptation) to model editing methods like ROME and MEMIT, increasing evidence suggests that specific behaviors in large language models are often controlled by low-dimensional subspaces in parameter space. Understanding the location and function of these "behavior vectors" is more effective than blindly modifying large numbers of parameters. This also means the future direction of uncensored technology is not broader modifications but more precise localization—finding and removing the key neurons or attention heads that encode refusal behavior.

The complete test report, reasoning trajectories for all responses, and evaluation results are publicly available at abliterlitics.dev. The team is considering introducing new measurement dimensions to more comprehensively assess the impact of chat templates and hyperparameters on model behavior.

Key Takeaways

• Surgical editing (orcarouter, apostate) outperforms aggressive modifications across all metrics
• Thought loop phenomenon is a critical failure mode at 27B scale that renders models practically unusable
• Copyright protection has become the hardest censorship layer to remove, while traditional high-risk categories are now easier to unlock
• Chat templates can have equal or greater impact on behavior than weight modifications
• Model card claims require independent verification—technical specifications often don't match actual implementations