Neural 3
An ultra-compact on-device language model developed by Cactus Compute (a YC-backed company), designed for structured output and tool calling. The full model weighs only 29 MB and is intended to run on edge devices such as smartphones, smartwatches, and embedded development boards.
Timeline (last 90 days)
Neural 3 (also known as Needle) is an on-device model launched by San Francisco startup Cactus Compute, incubated by YC; it requires no internet connection and can run on phones, watches, and small development boards.
The full version of Neural 3 is only 29 MB, the smallest usable version is approximately 8 MB, and the weights are compressed to approximately 2 bits each.
Neural 3 compiles function schemas into byte-level grammars via grammar-constrained decoding, ensuring every generated token conforms to the grammar so that JSON can always be parsed successfully.
Cactus replaces the feed-forward layer of the traditional Transformer with a headband MLP, containing only approximately 25,600 parameters per layer instead of 4.7 million, with most knowledge stored in an engram table that retrieves short 2-to-3-word fragments via hash lookup.
The full Neural 3 model with 121 million parameters has an actual computational load roughly equivalent to a 50-million-parameter model.
Neural 3's intelligent slider design gives the model 20 layers in total, with depths 2 through 4 each capable of running as independent models; the 4-layer version has 29 million parameters, and each additional layer consumes approximately 1 MB more memory.
Neural 3 reports a decode speed of approximately 400 to 4,000 tokens per second on small development boards.
Neural 3's three-way routing mode is: confidence ≥ 0.7 executes directly, the middle range asks the user for confirmation, and no return results in a reply that the task cannot be completed; the engine's default confidence lower limit is 0.5.
Every field filled in by Neural 3's extraction function is a span copied directly from the source text; if a value is not explicitly stated in the text the model cannot produce it, and text with scattered information is harder to process than clean text.
The Neural 3 base model's 4-layer version scores only 11.7 on the mobile action benchmark and 21 on droid call, while the full 20-layer version scores 86 and 47 respectively, and one developer's self-test yielded only 32.2%.
3 more timeline events
All Facts (13)
Neural 3 (also known as Needle) is an on-device model launched by San Francisco startup Cactus Compute, incubated by YC; it requires no internet connection and can run on phones, watches, and small development boards.
50%UnverifiedThe full version of Neural 3 is only 29 MB, the smallest usable version is approximately 8 MB, and the weights are compressed to approximately 2 bits each.
50%UnverifiedNeural 3 compiles function schemas into byte-level grammars via grammar-constrained decoding, ensuring every generated token conforms to the grammar so that JSON can always be parsed successfully.
50%UnverifiedCactus replaces the feed-forward layer of the traditional Transformer with a headband MLP, containing only approximately 25,600 parameters per layer instead of 4.7 million, with most knowledge stored in an engram table that retrieves short 2-to-3-word fragments via hash lookup.
50%UnverifiedThe full Neural 3 model with 121 million parameters has an actual computational load roughly equivalent to a 50-million-parameter model.
50%UnverifiedNeural 3's intelligent slider design gives the model 20 layers in total, with depths 2 through 4 each capable of running as independent models; the 4-layer version has 29 million parameters, and each additional layer consumes approximately 1 MB more memory.
50%UnverifiedNeural 3 reports a decode speed of approximately 400 to 4,000 tokens per second on small development boards.
50%UnverifiedNeural 3's three-way routing mode is: confidence ≥ 0.7 executes directly, the middle range asks the user for confirmation, and no return results in a reply that the task cannot be completed; the engine's default confidence lower limit is 0.5.
50%UnverifiedEvery field filled in by Neural 3's extraction function is a span copied directly from the source text; if a value is not explicitly stated in the text the model cannot produce it, and text with scattered information is harder to process than clean text.
50%UnverifiedThe Neural 3 base model's 4-layer version scores only 11.7 on the mobile action benchmark and 21 on droid call, while the full 20-layer version scores 86 and 47 respectively, and one developer's self-test yielded only 32.2%.
50%UnverifiedCactus officially states that Needle is a task-specific model that should be fine-tuned before going to production, and that the base model is a starting point rather than a finished assistant.
50%UnverifiedNeural 3 fine-tuning defaults to 10 epochs, training LoRA adapters on a frozen base, and supports exporting pre-compiled engines under 1 MB for macOS, Windows, Linux, Android, iOS, watchOS, and WebAssembly browsers.
50%UnverifiedNeural 3 uses the Apache 2.0 open-source license, has zero cost per call, and data never leaves the device.
50%