Unverified50% confidenceFactExact time
The masks learned by Deletion Tuning are hundreds to thousands of times smaller than the corresponding full fine-tuning checkpoints.
1
Sources
50%
Confidence
Long-term
Relevance
11/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn reinforcement learning post-training, Deletion Tuning's prior-task accuracy remains almost at the base model level, whereas full fine-tuning, LoRA, and random sparse fine-tuning lose up to 5.4 percentage points.69% similarUnverifiedThe model is trained in two stages: first masked language model pre-training on about 20 million sentences from FineWeb-Edu, then correction fine-tuning on 12.02 million (noisy, clean) pairs, of which 30% are identity pairs.67% similarUnverifiedThe research team trained a series of decoder-only architecture models from scratch, with parameter scales ranging from 47 million to 349 million.66% similarUnverifiedUnder the most permissive setting of each relaxed verification rule, relaxed verification accepted an average of 1.34 times as many tokens per round as lossless verification, but the generated completion length inflated to 1.54 times.66% similarUnverifiedDistillation training used 600 prompts (500 training / 100 validation), achieving a representation cosine similarity of 0.9959, a semantic correction cosine similarity of 0.9836, and a mixed-condition cosine similarity of 0.999958 relative to the teacher bridge at alpha=0.10.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1153749API
curl https://kongchang.com/api/v1/knowledge/claims/1153749MCP
get_claim(id=1153749)