BitRun
A zero-compilation, single-file 1-bit LLM inference runner with statically linked built-in NEON DotProd and AVX2 native SIMD kernels. It runs ternary-quantized models on pure CPU without build dependencies.
Timeline (last 90 days)
BitRun is a statically linked standalone runner with built-in native SIMD kernels (NEON DotProd on Mac, AVX2 on Windows/Linux) and requires no build dependencies.
On an Apple Silicon M4 Pro (12 cores, ARM NEON), Falcon3 1B Instruct reaches 137.5 t/s prompt processing and 98.6 t/s generation speed, with 1.9 GB memory usage.
On Windows 10/11 x86_64 (Intel Core i5-12600K, AVX2), BitRun runs Falcon3 1B at a generation speed of 40-45 t/s.
On Linux x86_64 (WSL2/Ubuntu, AVX2), BitRun has a first-token latency of about 130 ms and a generation speed of 40-45 t/s.
Cloud APIs typically have a first-token latency of 200-500 ms, and BitRun's roughly 130 ms on WSL2 is already usable.
BitRun can be installed with a one-line curl command and supports macOS and Linux; there is also a one-line install command for Windows PowerShell, and after installation you can pull the model with bitrun pull falcon3-1b and run it with bitrun run falcon3-1b.
BitRun can start an OpenAI-compatible REST interface with the command bitrun serve falcon3-1b --port 11435.
BitRun's GitHub repository is IshekKhal/bitrun, with released binaries under the v0.1.0-alpha tag; the related upstream PRs #642 and #644 are in microsoft/BitNet.
All Facts (8)
BitRun is a statically linked standalone runner with built-in native SIMD kernels (NEON DotProd on Mac, AVX2 on Windows/Linux) and requires no build dependencies.
50%UnverifiedOn an Apple Silicon M4 Pro (12 cores, ARM NEON), Falcon3 1B Instruct reaches 137.5 t/s prompt processing and 98.6 t/s generation speed, with 1.9 GB memory usage.
50%UnverifiedOn Windows 10/11 x86_64 (Intel Core i5-12600K, AVX2), BitRun runs Falcon3 1B at a generation speed of 40-45 t/s.
50%UnverifiedOn Linux x86_64 (WSL2/Ubuntu, AVX2), BitRun has a first-token latency of about 130 ms and a generation speed of 40-45 t/s.
50%UnverifiedCloud APIs typically have a first-token latency of 200-500 ms, and BitRun's roughly 130 ms on WSL2 is already usable.
50%UnverifiedBitRun can be installed with a one-line curl command and supports macOS and Linux; there is also a one-line install command for Windows PowerShell, and after installation you can pull the model with bitrun pull falcon3-1b and run it with bitrun run falcon3-1b.
50%UnverifiedBitRun can start an OpenAI-compatible REST interface with the command bitrun serve falcon3-1b --port 11435.
50%UnverifiedBitRun's GitHub repository is IshekKhal/bitrun, with released binaries under the v0.1.0-alpha tag; the related upstream PRs #642 and #644 are in microsoft/BitNet.
50%