Qwen3.5-0.8B
A compact 0.8-billion-parameter language model in Alibaba's Qwen series. Its language modeling capabilities can be enhanced through an externally attached memory module combined with a dynamic gating mechanism.
Timeline (last 90 days)
On 194 Yelp text judgments, B1 (0.8B) got 184 correct and B2 (4B) got 192 correct; the average NLLs were 0.09620 and 0.04024, respectively.
On 1,184 historical multi-task questions, B1 got 951 correct and B2 got 1,033 correct, with NLLs of 0.60133 and 0.46749, respectively.
B1 made 181 predictions with confidence no lower than 0.9, of which 1 was wrong; B2 made 188, of which 2 were wrong; a better average does not automatically mean fewer high-confidence errors, and 0.9 is not a ready-made production threshold.
Transferring the PLE (n-gram memory) pre-trained by Qwen3.8 Flash-Next to the Qwen3.5-0.8B model achieved a 5.05% perplexity reduction on the frozen validation set.
In the experiment, the Qwen3.5-0.8B backbone and approximately 51 billion parameter PLE memory module were frozen, with only R=1 readers deployed at decoder layers 3 and 9 being trained.
The original Qwen3.5-0.8B has NLL of 2.905585 and PPL of 18.2759; Qwengram-0.8B has NLL of 2.853786 and PPL of 17.3534.
All Facts (6)
On 194 Yelp text judgments, B1 (0.8B) got 184 correct and B2 (4B) got 192 correct; the average NLLs were 0.09620 and 0.04024, respectively.
50%UnverifiedOn 1,184 historical multi-task questions, B1 got 951 correct and B2 got 1,033 correct, with NLLs of 0.60133 and 0.46749, respectively.
50%UnverifiedB1 made 181 predictions with confidence no lower than 0.9, of which 1 was wrong; B2 made 188, of which 2 were wrong; a better average does not automatically mean fewer high-confidence errors, and 0.9 is not a ready-made production threshold.
50%UnverifiedTransferring the PLE (n-gram memory) pre-trained by Qwen3.8 Flash-Next to the Qwen3.5-0.8B model achieved a 5.05% perplexity reduction on the frozen validation set.
50%UnverifiedIn the experiment, the Qwen3.5-0.8B backbone and approximately 51 billion parameter PLE memory module were frozen, with only R=1 readers deployed at decoder layers 3 and 9 being trained.
50%UnverifiedThe original Qwen3.5-0.8B has NLL of 2.905585 and PPL of 18.2759; Qwengram-0.8B has NLL of 2.853786 and PPL of 17.3534.
50%