Unverified50% confidenceFactExact time
Latent Compaction training is divided into two stages: the first stage fits the attention output and log normalizer of the full cache, and the second stage jointly optimizes the entire cache through end-to-end distillation.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMethods such as freezing LLM backbone parameters during the visual alignment training phase, staged training, data ratio replay, and LoRA can mitigate catastrophic forgetting in multimodal training.74% similarUnverifiedUpdating parametric memory requires fine-tuning or continual pre-training, both of which require additional training processes and computational resources.73% similarUnverifiedLatent Compaction treats the KV cache itself as a trainable neural network and optimizes it through test-time training.71% similarUnverifiedMiniMind covers the complete pipeline including Tokenizer training, pre-training, supervised fine-tuning (SFT), LoRA fine-tuning, DPO reinforcement learning, and model distillation.70% similarUnverified延迟解耦(Delayed Untying)的核心思想是：训练前2/3阶段将embed与lm_head绑定共享，到达某个节点后复制权重和优化器状态，然后让二者作为独立参数分别训练69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037896API
curl https://kongchang.com/api/v1/knowledge/claims/1037896MCP
get_claim(id=1037896)