Freeze the Base, Fix the Output: Error Correction Without Capability Loss

The Unavoidable Tradeoff: Correction vs. Capability

When a large language model makes mistakes, there are typically two options: fine-tune the entire model, or attach an external correction module. The first approach fixes errors effectively but often at the cost of the model's existing capabilities. A new arXiv paper (arXiv:2609.16145) addresses this challenge head-on — can we repair output errors using a tiny correction module while keeping the base model completely frozen, without degrading its general capabilities?

The team's solution is called CRN v2 — a lightweight correction module that operates at the logit level, with only approximately 34 million trainable parameters, equivalent to just 0.73% of the 4.65 billion-parameter text module. It sits like a cap on top of the frozen Gemma 4 E2B model: the base weights remain untouched throughout, and only the correction module learns.

How CRN v2 Works

Training follows a two-stage pipeline: supervised fine-tuning (SFT) first, then reference-free DPO, using 83,400 error-correction sample pairs. The design is built around three core principles: frozen base model, logit-level correction, and KL anchoring.

Evaluation was conducted on a 60-question domain exam called CEHRI (Certified Human-Robot Intelligence), covering three task types: factual questions, arithmetic operations, and implicit goal reasoning. Results show that CRN v2 corrects 53.3% of the base model's errors, and still corrects 43.3% on paraphrase variants (reworded questions).

More critically, on capability preservation: CRN v2 showed zero capability degradation on benchmarks including MMLU/BoolQ (N=200) and car-wash (N=8). This is precisely the value proposition it aims to demonstrate — error correction doesn't have to come at the cost of forgetting.

Reference-free DPO is a variant of Direct Preference Optimization. Standard DPO requires a "reference model" to compute the KL penalty for policy deviation, while the reference-free variant replaces this role implicitly — eliminating the need to store and query a full reference model copy during training, significantly reducing memory usage. In the CRN v2 setup, since the base model is frozen, its output logits naturally serve as the reference distribution. The KL anchoring term builds on this foundation, directly constraining the corrected logit distribution from deviating too far from the frozen base's original output. This both reduces the dependency on a separate reference model and embeds the goal of "preserving existing capabilities" directly into the optimization objective.

Head-to-Head Comparison with the LoRA Baseline

To highlight the tradeoff, the paper compares against a LoRA baseline matched to the CRN v1 parameter budget: 6.6 million parameters, rank 19. This LoRA configuration achieves a correction rate of 83.3%, which looks like a commanding lead.

But the cost is striking — it shows 30% to 75% capability loss on the same benchmarks. This is the "correction-capability tradeoff" the researchers repeatedly emphasize: aggressively pursuing high correction rates can easily destroy what the model was originally good at. CRN v2 trades a somewhat lower correction rate for zero capability loss — a far better deal in many real-world deployment scenarios.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method. Instead of directly modifying the model's full weight matrices, it inserts a pair of low-rank matrices alongside the original weights (rank determines the size of these matrices) and only trains those parameters. Higher rank means more trainable parameters and stronger fitting capacity, but also greater perturbation to the original weight distribution. The LoRA baseline in this paper uses rank 19 and 6.6 million parameters — fewer parameters than CRN v2, yet because it directly modifies the effective distribution of base weights, the model's correction ability improves dramatically on the target task while performance on other tasks drifts accordingly. This stands in sharp contrast to CRN v2's design of keeping base weights completely frozen, and is the fundamental reason why the two approaches differ so dramatically in capability preservation.

The KL Preservation Term Is Non-Negotiable

Ablation experiments reveal a decisive factor: the KL preservation term (lambda=0.1) is critical. When this coefficient is reduced to 0.01, the correction rate drops directly from 53.3% to 35.0%. This shows that KL anchoring isn't just a safety belt against capability degradation — it actively supports the effectiveness of the correction itself.

In other words, the elegance of this design is that constraint and performance are not opposites. A moderate KL constraint actually stabilizes the learning direction of the correction module.

KL divergence (Kullback-Leibler Divergence) measures the difference between two probability distributions, commonly used in language model training to prevent excessive policy drift. In alignment methods like RLHF and DPO, the KL penalty term keeps the trained model from "drifting too far" — its output distribution should maintain reasonable similarity to the reference distribution. In CRN v2, the KL anchoring coefficient lambda=0.1 plays a dual role: it prevents the correction module from overfitting to the training set's error-correction pairs (which would collapse the output distribution on other tasks), while also providing a regularization direction for gradient updates, making the correction signal more stable and generalizable. The ablation showing correction rate plummeting when lambda drops from 0.1 to 0.01 indicates that insufficient anchoring pushes the correction module into an unstable optimization landscape — rather than simply "removing a constraint."

Exploring Variants: 53% Looks Like a Ceiling

The research team also tested a series of alternative configurations in an attempt to push past 53%, without success:

• Hidden state injection variant (injecting at earlier layers, 1.6M parameters, SFT only): reached 50.0%/55.8%, but did not surpass logit correction;
• Shallower layer injection (layer 4): dropped to 30.0%/28.3%;
• Multi-depth logit correction (~35M parameters): only 40%;
• Longer training (5,000 SFT steps + 2,000 DPO steps): still stuck at 53.3%.

No alternative configuration outperformed the rank-128 logit correction result. The researchers conclude that ~53% looks more like a best achievable ceiling for the current approach, rather than a floor that can easily be raised.

The Research's Positioning and Value

The authors are explicit that this is a study of design principles (frozen base + logit correction + KL anchoring), not a claim of architectural innovation. This honest positioning actually makes the conclusions more credible — it validates a reusable engineering paradigm rather than presenting yet another flashy new model.

For practical applications, the approach has clear appeal: when you have a base model you cannot or do not want to modify, yet need to fix its errors in a specific domain, attaching a lightweight correction module that accounts for less than 1% of the base model's parameters is far safer than fine-tuning the entire model and risking capability collapse.

Noteably, the team has open-sourced all code, primary result weights, and evaluation scripts (deeper variants provide code only, without trained checkpoints), leaving room for subsequent replication and extension.

Summary

The significance of CRN v2 lies not in pushing correction rates as high as possible, but in using rigorous controlled experiments to map out the "correction-capability tradeoff" curve and identify KL anchoring as the key lever for balancing both ends. For teams who care about model controllability and safe deployment, this is a practical paradigm worth referencing.