BitRun v0.2 Hands-On: A 1-bit LLM Terminal Cockpit with Milliwatt-Level Power Telemetry

Microsoft Research's BitNet b1.58 paper proposed training language models from scratch with ternary weights {-1, 0, +1}, completely eliminating floating-point matrix multiplication at inference time in favor of integer addition for tensor operations. The paper promised 70% to 80% energy reduction while running at very high speed on ordinary laptop CPUs.

But anyone who actually runs models locally knows the reality: battery percentages visibly ticking down, the terminal spewing unformatted text, and no idea how many watts the CPU is actually consuming. The initial alpha of BitRun, released by author IshekKhal, drew feedback from thousands of developers—they wanted real power data (active wattage and joules per token), models with more than 2 billion parameters, and an interface that felt like a command center rather than a scrolling terminal.

v0.2.0-beta is a direct response to that feedback.

BitRun Cockpit: A Real-Time Command Center in the Terminal

The centerpiece of this update is BitRun Cockpit. Running bitrun cockpit bonsai-1.7b launches a dual-panel terminal interface purpose-built for low-power inference.

The left panel is an interactive conversation, using local Server-Sent Events (SSE) for character-by-character token streaming, with independent scrolling—you can revisit earlier parts of a long conversation without interrupting generation. The right panel is a real-time telemetry dashboard monitoring hardware metrics including:

• Generation speed and prompt processing speed (tokens/sec)
• Time to first token (milliseconds)
• CPU performance-core and efficiency-core load percentages
• CPU package instantaneous power (watts)
• Cumulative energy per token (joules)
• Battery discharge rate estimate

All background logs are automatically written to ~/.bitrun/engine.log, keeping the terminal interface clean with no screen tearing. For a tool that centers on energy efficiency, putting power data directly in front of the user is more persuasive than any benchmark.

Server-Sent Events (SSE) is an HTTP-based one-way streaming technology that lets a server continuously push text data to a client without closing the connection. Unlike WebSocket's bidirectional communication, SSE is more lightweight and naturally suited to LLM token-by-token output: the server pushes an event for each token generated, and the client renders it immediately. SSE uses the text/event-stream content type, is parseable by both browsers and terminal clients, and supports automatic reconnection. In BitRun Cockpit, SSE is exactly what enables character-by-character streaming in the conversation panel without repeatedly issuing requests for each response segment.

Can 1-bit Compute Handle 8 Billion Parameters?

One of the community's biggest questions is whether 1-bit compute breaks down at scale.

v0.2's answer: BitRun can run PrismML's 8-billion-parameter model bonsai-8b entirely in CPU memory, using about 2.08 GB of RAM. No discrete GPU, no cloud API, zero compilation.

The principle: 1.58-bit models replace floating-point matrix multiplication with integer addition, cutting memory footprint by roughly 80% compared to a 16-bit baseline, allowing a full-scale inference model to sit comfortably on an ordinary laptop.

Ternary weights and where 1.58-bit comes from: Each weight can only take three discrete values {-1, 0, +1}, which in binary representation requires exactly log₂(3) ≈ 1.58 bits—hence the name "1.58-bit." Unlike traditional 2-bit quantization, ternary weights reduce matrix multiplication to "signed integer accumulation"—multiplying by +1 is addition, multiplying by -1 is subtraction, multiplying by 0 is a skip, and the floating-point multiply units sit completely idle. This form delivers two benefits simultaneously: greatly compressed memory footprint, and operations that can be done on integer ALUs with far lower power than FP32/FP16 matmul. The trade-off is limited expressiveness, which is why the BitNet family emphasizes "training from scratch" rather than post-training quantization—so the model can genuinely adapt to the ternary constraint.

Rebuilding the Inference Engine

To get the 8B model running stably, the author rebuilt the inference runtime from the ground up, solving engineering problems at four levels.

Decoupling from the Old Submodule

Rather than patching Microsoft's now-frozen research submodule, the native engine was rebuilt on the modern upstream llama.cpp architecture. This independent engine branch gives the team full control over SIMD optimization and tensor layout.

Fixing the 8B Tensor Offset Crash

Early attempts to load the 8B ternary checkpoint crashed due to a GGUF Type 42 tensor conflict. The author implemented dynamic legacy type remapping in the reader, allowing the 8-billion-parameter weights to load without memory faults.

Matrix Memory Boundaries and Stride Safety

In the packed 2-bit kernel, stride mismatches previously caused memory corruption beyond 32 context tokens. The fix was to unify row-stride computation across the entire tensor graph (.blck_size = 4, .type_size = sizeof(int8_t)), ensuring stability across multi-turn conversations.

Stride is the number of bytes a tensor must skip to advance one row along a given dimension in memory. Low-bit quantization often packs multiple weights into a single byte (e.g., four 2-bit values stuffed into an int8), so the logical row width and the physical byte row width no longer correspond one-to-one. If the reader and the kernel compute stride inconsistently—say the kernel counts pre-packing elements while the reader counts post-packing bytes—the pointer will read out of bounds into an adjacent tensor's data, manifesting as sudden garbled output or crashes after a few tokens. BitRun eliminated this ambiguity in the packed kernel by unifying row-stride computation under .blck_size = 4 and .type_size = sizeof(int8_t), addressing a very typical class of "data mismatch" bugs in low-bit inference engines.

Architecture-Specific SIMD Acceleration

On Apple Silicon, ARM NEON dot-product kernels and squared ReLU activation are integrated, while Windows and Linux PCs get an AVX2 fallback.

SIMD (Single Instruction, Multiple Data) allows a single CPU instruction to process multiple data elements at once, and is the core source of CPU inference performance. On ARM platforms, the counterpart is the NEON instruction set, whose dot-product instructions such as vdotq_s32 can complete the multiply-accumulate of 16 int8 values in one operation; on x86 platforms, AVX2 handles 256-bit-wide data, with AVX-512 extending to 512 bits. BitNet's dot product is essentially a "weighted sum," which aligns naturally with SIMD dot-product kernels—so whether a dedicated kernel exists essentially determines the performance ceiling. BitRun enables ARM NEON dot-product kernels on Apple Silicon, while on x86 the 8B model still runs through the generic portable loop in arch-fallback.h—which is precisely the root cause of the steep performance drop for large models on x86.

Real-Hardware Data: Power and Speed Across Three Platforms

All data below was recorded on physical hardware, with GPU offloading fully disabled.

Apple Silicon (MacBook Pro, M4 Pro CPU, 12 cores, 24 GB RAM)

Model	Params	Format	Disk Size	Generation Speed	Active Power	Energy per Token
bonsai-1.7b	1.7B	i2_s ternary	~237 MB	72.8 tok/s	11.7 W	0.316 J
falcon3-1b	1.0B	i2_s packed	~1.30 GB	73.2 tok/s	11.7 W	0.345 J
bitnet-2b	2.4B	i2_s ternary	~1.13 GB	31.6 tok/s	11.7 W	0.484 J
bonsai-8b	8.0B	i2_s ternary	~2.08 GB	21.0 tok/s	11.7 W	0.886 J
bitnet-70m	70M	Q8_0	~79 MB	1,619.8 tok/s	11.7 W	0.043 J

The standout figure on the M4 Pro is the 8B model: just over 2 GB on disk, 21 tok/s, under 1 joule per token. That's a remarkable efficiency ratio for a laptop.

Windows 10 Native (Intel Core i5-12600K, MSVC AVX2, 32 GB RAM)

Model	Params	Format	Disk Size	Generation Speed	Active Power	Energy per Token
bonsai-1.7b	1.7B	i2_s ternary	~237 MB	85.6 tok/s	22.6 W	0.264 J
falcon3-1b	1.0B	i2_s packed	~1.30 GB	11.1 tok/s	22.6 W	2.036 J
bitnet-2b	2.4B	i2_s ternary	~1.13 GB	6.7 tok/s	22.6 W	3.373 J
bonsai-8b	8.0B	i2_s ternary	~2.08 GB	4.0 tok/s	22.6 W	5.650 J
bitnet-70m	70M	Q8_0	~79 MB	597.8 tok/s	22.6 W	0.038 J

The x86 table shows a clear bifurcation: small, simple models perform reasonably well (bonsai-1.7b reaches 85.6 tok/s, even beating the M4 Pro), but packed formats and larger parameter counts fall off sharply—bonsai-8b manages only 4.0 tok/s.

Linux WSL2 (Intel Core i5-12600K, GCC AVX2, 2 vCPU)

Model	Params	Format	Disk Size	Generation Speed	Active Power	Energy per Token
bonsai-1.7b	1.7B	i2_s ternary	~237 MB	16.8 tok/s	8.5 W	0.505 J
falcon3-1b	1.0B	i2_s packed	~1.30 GB	6.7 tok/s	8.5 W	1.261 J
bitnet-2b	2.4B	i2_s ternary	~1.13 GB	4.4 tok/s	8.5 W	1.938 J
bonsai-8b	8.0B	i2_s ternary	~2.08 GB	0.5 tok/s	8.5 W	16.058 J
bitnet-70m	70M	Q8_0	~79 MB	134.1 tok/s	8.5 W	0.063 J

WSL2 is limited by the number of virtual CPUs, so its performance numbers have limited reference value, but the 8B model's 0.5 tok/s is essentially unusable in practice.

The author's explanation for the architectural throughput differences: Apple Silicon benefits from dedicated ARM NEON dot-product inline functions in arch/arm/quants.c; on x86_64, bonsai-8b and packed formats currently go through the generic portable loop in arch-fallback.h to ensure stability on older CPUs, and dedicated AVX2/AVX-512 kernels have not yet been added.

This data actually answers a key question: 1-bit LLMs already deliver practical performance on ARM platforms, but running large models on x86 will have to wait for dedicated SIMD kernels to be completed.

Zero-Compilation Setup

Precompiled standalone binaries are available for macOS (Apple Silicon), Windows (x86_64), and Linux—no CMake, no Python virtual environment, no Docker.

macOS / Linux install:

curl -fsSL https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.sh | sh

Windows PowerShell install:

irm https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.ps1 | iex

Launch the Cockpit:

bitrun cockpit bonsai-1.7b

Run an OpenAI-compatible REST service on localhost:8080:

bitrun serve bonsai-1.7b

You can point Cursor, Continue.dev, or Open WebUI directly at the local endpoint—no cloud dependency.

Open Source and Availability

All code and precompiled release binaries are open source under the MIT license. The project is at github.com/IshekKhal/bitrun, and the v0.2.0-beta release assets are available on the repository's Releases page.

BitRun Cockpit demonstrates a pragmatic direction: embedding power telemetry directly into the inference terminal so developers can see their electricity costs at a glance. For anyone who needs to run models offline or on battery power, that's more useful than benchmark numbers alone. And the weak spot in x86 large-model performance clearly marks where the next round of optimization needs to go.

One-line Summary

BitRun v0.2 launches a Cockpit dual-panel terminal that embeds CPU power telemetry into local 1-bit inference, with an 8B model running on a laptop CPU.

Paragraph Summary

BitRun v0.2.0-beta responds to thousands of developer feedback items from the early alpha with BitRun Cockpit—a dual-panel terminal interface featuring streaming conversation on the left and real-time telemetry on the right, showing tokens/s, first-token latency, CPU package power, and joules per token. Built on BitNet b1.58's ternary weight scheme, the 8B bonsai-8b model takes only ~2.08 GB of memory and runs on CPU with no GPU or cloud API. The author rebuilt the inference engine from the ground up, fixing GGUF Type 42 tensor conflicts and 2-bit kernel stride memory corruption, and integrating ARM NEON acceleration. Real-hardware data shows: on M4 Pro, bonsai-8b hits 21 tok/s at 0.886 joules per token; on x86 platforms, the same model manages only 4 tok/s due to missing dedicated AVX2/AVX-512 kernels—a clear direction for the next optimization effort.