#ternary quantization
10 related articles
·2 min
Training a CPU-Native LLM on Zero Budget: The Architectural Gamble of Ternary Quantization × Sparse MoE
An indie dev attempts to train a CPU-native LLM on $0 budget using ternary quantization, sparsity, and fine-grained MoE — with pre-registered success criteria and full public reporting.
Read more →
·4 min
Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop
Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.
Read more →
·7 min
DIY Companion Robot: Building Your AI Desktop Buddy with Jetson Orin
A maker builds a DIY companion robot with NVIDIA Jetson Orin and 4S LiPo battery. Explore the full development journey from first power-up to AI interaction, including edge computing, power design, and companion robot trends.
Read more →
·3 min
Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression
PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.
Read more →
Bonsai 27B: The First 1-bit LLM That R…
·3 min
Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI
Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.
Read more →
·2 min
Apple in Talks to Acquire PrismML: Can Extreme Quantization Really Run AI on an iPhone?
Apple is reportedly in talks to acquire AI startup PrismML, whose 1-bit extreme quantization could run large models on iPhone. Community tests reveal tool-calling failures and high hallucination rates.
Read more →
·3 min
Bonsai-27B Extreme Quantization: Running 1-bit LLMs on Consumer Hardware
Bonsai-27B supports binary/ternary extreme quantization for 27B LLMs running on 8GB VRAM. Covers llama.cpp upstream progress, RTX 4060 benchmarks (30 t/s), and real-world limitations.
Read more →
·3 min
CPU-Native Language Models: Breaking Through Inference Bottlenecks with Memory Bandwidth Co-Design
SiliconLLM builds a CPU-native LLM architecture from scratch, combining selective SSM, ternary (1.58-bit) LUT MLP, and granular MoE, co-designed around the L3 cache bandwidth cliff. Ternary kernels show 4-5x speedup over fp32.
Read more →
·7 min
Electron + LangGraph in Practice: Building an AI Agent Visual Orchestration App from Scratch
A hands-on guide to building an enterprise-grade AI Agent workflow orchestration app with Electron Forge and LangGraph, covering local LLM deployment (Qwen3-0.6B), node-based visual canvas design, and full Function Calling integration.
Read more →
·4 min
Ternlight: A 7MB Browser-Side Embedding Model for Server-Free Semantic Search
Ternlight is a 7MB WebAssembly-based browser-side text embedding model requiring no server or GPU. Explore its tech, use cases, and tradeoffs for private, offline semantic search.
Read more →