PLE
An external memory structure that pre-encodes statistical co-occurrence information of high-frequency n-grams into retrievable vectors. During pretraining, n-gram-level conditional probabilities are distilled into fixed parameters. Once frozen, the module can be mounted as a standalone component onto other language models.
Timeline (last 90 days)
Transferring the PLE (n-gram memory) pre-trained by Qwen3.8 Flash-Next to the Qwen3.5-0.8B model achieved a 5.05% perplexity reduction on the frozen validation set.
In the experiment, the Qwen3.5-0.8B backbone and approximately 51 billion parameter PLE memory module were frozen, with only R=1 readers deployed at decoder layers 3 and 9 being trained.
Using real pre-trained PLE performs significantly better than control groups using random memory and shuffled memory.
The GGUF packages the Qwen3.5 backbone together with reader and arbitration tensors, while the large PLE memory exists as an external quantized sidecar and is not packed into the model GGUF.
All Facts (4)
Transferring the PLE (n-gram memory) pre-trained by Qwen3.8 Flash-Next to the Qwen3.5-0.8B model achieved a 5.05% perplexity reduction on the frozen validation set.
50%UnverifiedIn the experiment, the Qwen3.5-0.8B backbone and approximately 51 billion parameter PLE memory module were frozen, with only R=1 readers deployed at decoder layers 3 and 9 being trained.
50%UnverifiedUsing real pre-trained PLE performs significantly better than control groups using random memory and shuffled memory.
50%UnverifiedThe GGUF packages the Qwen3.5 backbone together with reader and arbitration tensors, while the large PLE memory exists as an external quantized sidecar and is not packed into the model GGUF.
50%