SHADOW-50M
An ultra-small language model trained from scratch by Reddit developer QLNI, with an actual parameter count of 44M. It employs ternary weights and a frozen fingerprint vocabulary. The model file is only 19.8 MB and achieves approximately 1,900 tokens per second on CPU inference, optimized for extreme compression and local execution.
Timeline (last 90 days)
SHADOW-50M is a complete language model trained from scratch that has digested 45 billion tokens, actually has 44M parameters, and has a model size of 19.8 MB.
SHADOW-50M runs at approximately 1,900 tok/s on a laptop CPU with a memory footprint of only 41 MB.
SHADOW-50M uses ternary weights, with all parameters constrained to three values {-1, 0, +1}, theoretically requiring only 1.58 bits per parameter.
SHADOW does not use a trained embedding layer; instead, it represents a vocabulary of 73,880 tokens using fixed 512-bit fingerprints, the frozen table is only 4.7 MB, and new tokens can be added at any time without retraining.
The author fixed missing word pieces by directly adding 8,600 rows to the frozen fingerprint table without any training or weight changes, and all 34 previously published answers remained unchanged.
The frozen fingerprint table achieved a Spearman correlation of 0.594 on a human word similarity benchmark, compared to -0.057 for random encoding, indicating that fixed fingerprints carry meaningful semantic structure.
SHADOW triggers fixed circuits to take over computation during the readout phase by outputting markers such as [calc]347*86[eq], without using calculator APIs, tool calls, or prompt pasting, with circuits implemented for arithmetic, percentages, dates, and unit conversions.
SHADOW writes attention states to disk at 1-bit precision, consuming 288 bytes per token, with the index occupying only 22 bytes per token, record lookup taking approximately 1 microsecond, and attention states being reinjected into the model in approximately 0.03 ms, without using a vector database or embedding model.
At the 100 million token scale, the archive occupies 28.8 GB plus a 2.2 GB index on disk, while in-process memory via memory-map is only approximately 28 MB.
On repeated questions, index reinforcement raised the top-1 hit rate from 0.571 to 0.743 without any model training throughout the process.
4 more timeline events
All Facts (14)
SHADOW-50M is a complete language model trained from scratch that has digested 45 billion tokens, actually has 44M parameters, and has a model size of 19.8 MB.
50%UnverifiedSHADOW-50M runs at approximately 1,900 tok/s on a laptop CPU with a memory footprint of only 41 MB.
50%UnverifiedSHADOW-50M uses ternary weights, with all parameters constrained to three values {-1, 0, +1}, theoretically requiring only 1.58 bits per parameter.
50%UnverifiedSHADOW does not use a trained embedding layer; instead, it represents a vocabulary of 73,880 tokens using fixed 512-bit fingerprints, the frozen table is only 4.7 MB, and new tokens can be added at any time without retraining.
50%UnverifiedThe author fixed missing word pieces by directly adding 8,600 rows to the frozen fingerprint table without any training or weight changes, and all 34 previously published answers remained unchanged.
50%UnverifiedThe frozen fingerprint table achieved a Spearman correlation of 0.594 on a human word similarity benchmark, compared to -0.057 for random encoding, indicating that fixed fingerprints carry meaningful semantic structure.
50%UnverifiedSHADOW triggers fixed circuits to take over computation during the readout phase by outputting markers such as [calc]347*86[eq], without using calculator APIs, tool calls, or prompt pasting, with circuits implemented for arithmetic, percentages, dates, and unit conversions.
50%UnverifiedSHADOW writes attention states to disk at 1-bit precision, consuming 288 bytes per token, with the index occupying only 22 bytes per token, record lookup taking approximately 1 microsecond, and attention states being reinjected into the model in approximately 0.03 ms, without using a vector database or embedding model.
50%UnverifiedAt the 100 million token scale, the archive occupies 28.8 GB plus a 2.2 GB index on disk, while in-process memory via memory-map is only approximately 28 MB.
50%UnverifiedOn repeated questions, index reinforcement raised the top-1 hit rate from 0.571 to 0.743 without any model training throughout the process.
50%UnverifiedSupra-50M leads SHADOW on standard benchmarks: ARC-Easy 0.435 vs. 0.307, PIQA 0.600 vs. 0.570, and WikiText-2 perplexity 165 vs. 186.
50%UnverifiedIn a video memory experiment, Gemma 3 4B watched a ten-minute short film, captured a frame every two seconds, and wrote 298 descriptions; SHADOW retained these moments as disk memory and scored 55–56/60 on a laptop with approximately 42 MB of memory.
50%UnverifiedAs a draft model for Qwen3-32B, SHADOW boosted llama.cpp generation speed from 19.7 to 28.5 tok/s through speculative decoding.
50%UnverifiedAfter compiling the SHADOW kernel to WebAssembly, it can run in a browser tab at approximately 500 tok/s, completely offline.
50%