CRN v2
A lightweight correction module operating at the logit level that repairs language model output errors without modifying base model weights. It follows three principles: freezing the base model, logit-level correction, and KL anchoring. The module contains approximately 34 million trainable parameters.
Timeline (last 90 days)
CRN v2, proposed in arXiv paper (arXiv:2609.16145), is a lightweight correction module operating at the logit level that fixes output errors while keeping the base model completely frozen.
CRN v2 has approximately 34 million trainable parameters, equivalent to 0.73% of the 4.65 billion-parameter text module.
CRN v2 is placed on top of the frozen Gemma 4 E2B model, with the base weights unchanged throughout.
The training pipeline for CRN v2 consists of two steps: supervised fine-tuning (SFT) first, followed by reference-free DPO, using 83,400 error-correction pairs.
On the CEHRI 60-question domain exam, CRN v2 corrected 53.3% of the base model's errors, and 43.3% on paraphrase variants.
On benchmarks such as MMLU/BoolQ (N=200) and car-wash (N=8), CRN v2 showed no capability degradation.
In reference-free DPO, since the base model is already frozen, its output logits can serve as the reference distribution, and the KL anchoring term directly constrains the corrected logit distribution from deviating excessively from the frozen base model's original output.
The LoRA baseline used for comparison in the paper uses rank 19 with 6.6 million parameters, achieving a correction rate of 83.3%, but exhibited capability loss of 30% to 75%.
Ablation experiments show that when the KL retention coefficient drops from 0.1 to 0.01, the correction rate falls from 53.3% to 35.0%, indicating that KL anchoring also contributes to supporting correction effectiveness.
Multiple alternative configurations tested by the research team (hidden state injection, shallower injection, multi-depth logit correction, longer training) all failed to surpass the rank-128 logit correction result, with approximately 53% judged as the current upper bound of the method.
2 more timeline events
All Facts (12)
CRN v2, proposed in arXiv paper (arXiv:2609.16145), is a lightweight correction module operating at the logit level that fixes output errors while keeping the base model completely frozen.
50%UnverifiedCRN v2 has approximately 34 million trainable parameters, equivalent to 0.73% of the 4.65 billion-parameter text module.
50%UnverifiedCRN v2 is placed on top of the frozen Gemma 4 E2B model, with the base weights unchanged throughout.
50%UnverifiedThe training pipeline for CRN v2 consists of two steps: supervised fine-tuning (SFT) first, followed by reference-free DPO, using 83,400 error-correction pairs.
50%UnverifiedOn the CEHRI 60-question domain exam, CRN v2 corrected 53.3% of the base model's errors, and 43.3% on paraphrase variants.
50%UnverifiedOn benchmarks such as MMLU/BoolQ (N=200) and car-wash (N=8), CRN v2 showed no capability degradation.
50%UnverifiedIn reference-free DPO, since the base model is already frozen, its output logits can serve as the reference distribution, and the KL anchoring term directly constrains the corrected logit distribution from deviating excessively from the frozen base model's original output.
50%UnverifiedThe LoRA baseline used for comparison in the paper uses rank 19 with 6.6 million parameters, achieving a correction rate of 83.3%, but exhibited capability loss of 30% to 75%.
50%UnverifiedAblation experiments show that when the KL retention coefficient drops from 0.1 to 0.01, the correction rate falls from 53.3% to 35.0%, indicating that KL anchoring also contributes to supporting correction effectiveness.
50%UnverifiedMultiple alternative configurations tested by the research team (hidden state injection, shallower injection, multi-depth logit correction, longer training) all failed to surpass the rank-128 logit correction result, with approximately 53% judged as the current upper bound of the method.
50%UnverifiedThe authors explicitly state that this is a study on design principles (frozen base + logit correction + KL anchoring), rather than a claim of architectural innovation.
50%UnverifiedThe research team made all code, main result weights, and evaluation scripts publicly available; deep variants provide only code without trained checkpoints.
50%