#BitNet
14 related articles
·4 min
Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop
Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.
Read more →
·3 min
Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression
PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.
Read more →
·2 min
On-Device LLMs, Custom Silicon, and 100K-GPU Clusters: AI Industry Enters Full-Spectrum Competition
Apple's 1-bit compression fits 27B models on iPhone, Meta builds custom chip Iris, China's 100K-GPU cluster goes live, Samsung enters AI PC — a deep dive into AI's new full-spectrum competition.
Read more →
Bonsai 27B: The First 1-bit LLM That R…
·3 min
Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI
Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.
Read more →
·2 min
Training a CPU-Native LLM on Zero Budget: The Architectural Gamble of Ternary Quantization × Sparse MoE
An indie dev attempts to train a CPU-native LLM on $0 budget using ternary quantization, sparsity, and fine-grained MoE — with pre-registered success criteria and full public reporting.
Read more →
·2 min
Apple in Talks to Acquire PrismML: Can Extreme Quantization Really Run AI on an iPhone?
Apple is reportedly in talks to acquire AI startup PrismML, whose 1-bit extreme quantization could run large models on iPhone. Community tests reveal tool-calling failures and high hallucination rates.
Read more →
·3 min
Bonsai-27B Extreme Quantization: Running 1-bit LLMs on Consumer Hardware
Bonsai-27B supports binary/ternary extreme quantization for 27B LLMs running on 8GB VRAM. Covers llama.cpp upstream progress, RTX 4060 benchmarks (30 t/s), and real-world limitations.
Read more →
·3 min
AI Daily: Alibaba Open-Sources Dance Model, World Model Launches, First AI-Native Phone Debuts
Alibaba open-sources 14B dance model Wan-Dancer, AutoNavi launches World Studio, Stepfun debuts AI-native phone STEPS NEO; GPT-5.6 file deletion and AI companion shutdowns spark safety and regulation debates.
Read more →
·3 min
CPU-Native Language Models: Breaking Through Inference Bottlenecks with Memory Bandwidth Co-Design
SiliconLLM builds a CPU-native LLM architecture from scratch, combining selective SSM, ternary (1.58-bit) LUT MLP, and granular MoE, co-designed around the L3 cache bandwidth cliff. Ternary kernels show 4-5x speedup over fp32.
Read more →
·5 min
GLM-5.2 Hands-On Review: A 753B-Parameter Open Model That Beats GPT and Claude on Value
In-depth hands-on review of Zhipu AI's flagship GLM-5.2: a 1M-token context window and API pricing just one-fifth of GPT/Claude. Covers website building, Chrome extensions, 3D game cloning, and agentic workflows.
Read more →
·6 min
AMD Ryzen AI Halo Dev Kit: A $4,000 Local LLM Powerhouse Taking on Apple and NVIDIA
AMD officially unveils the Ryzen AI Halo local AI dev kit, priced around $4,000 with 128GB unified memory, capable of running 70B LLMs locally. An in-depth look at its specs, pricing, and market competition.
Read more →
·6 min
The Complete Guide to GLM-5.2: Hands-On Review of an Open-Source Long-Context Model
An in-depth hands-on review of Zhipu AI's flagship GLM-5.2: 1M-token context, strong coding, mature agent workflows—at one-fifth the price of top frontier models. Covers website testing, Cursor integration, MCP tooling, and production migration.
Read more →
·7 min
Electron + LangGraph in Practice: Building an AI Agent Visual Orchestration App from Scratch
A hands-on guide to building an enterprise-grade AI Agent workflow orchestration app with Electron Forge and LangGraph, covering local LLM deployment (Qwen3-0.6B), node-based visual canvas design, and full Function Calling integration.
Read more →
·4 min
Ternlight: A 7MB Browser-Side Embedding Model for Server-Free Semantic Search
Ternlight is a 7MB WebAssembly-based browser-side text embedding model requiring no server or GPU. Explore its tech, use cases, and tradeoffs for private, offline semantic search.
Read more →