#1-bit quantization
15 related articles
·3 min
Unsloth Teams Up with Thinking Machines: Dynamic 1-bit Quantization Achieves 86% Compression, Running Multimodal LLMs in 270GB
Unsloth and Thinking Machines release dynamic 1-bit GGUF quantization for Inkling, compressing the model from 1.9TB to 270GB (86% reduction) while retaining 74.2% accuracy and adding vision/audio multimodal support.
Read more →
·3 min
Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression
PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.
Read more →
·3 min
Bonsai-27B Extreme Quantization: Running 1-bit LLMs on Consumer Hardware
Bonsai-27B supports binary/ternary extreme quantization for 27B LLMs running on 8GB VRAM. Covers llama.cpp upstream progress, RTX 4060 benchmarks (30 t/s), and real-world limitations.
Read more →
·4 min
Quantization Illustrated: A Deep Dive into Core Compression Techniques for Large Language Models
A deep dive into LLM quantization techniques covering symmetric/asymmetric quantization, PTQ, QAT, GPTQ, AWQ, and outlier solutions for efficient model deployment.
Read more →
·4 min
Running an 80B Large Model on Mac with Only 4.3GB of Memory: Extreme Compression Techniques Explained
Deep dive into how an 80B-parameter LLM runs on Mac with only 4.3GB memory, covering ultra-low-bit quantization, sparsity, memory mapping, and implications for privacy and edge AI.
Read more →
·2 min
A 14-Byte Neural Network Solves Mazes from Scratch: An Experiment in Extreme AI Compression
How can a 14-byte neural network solve 96.5% of unseen mazes? Explore extreme model compression, the relationship between model size and task complexity, and small models' potential in edge computing.
Read more →
·2 min
On-Device LLMs, Custom Silicon, and 100K-GPU Clusters: AI Industry Enters Full-Spectrum Competition
Apple's 1-bit compression fits 27B models on iPhone, Meta builds custom chip Iris, China's 100K-GPU cluster goes live, Samsung enters AI PC — a deep dive into AI's new full-spectrum competition.
Read more →
Bonsai 27B: The First 1-bit LLM That R…
·3 min
Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI
Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.
Read more →
·2 min
Apple in Talks to Acquire PrismML: Can Extreme Quantization Really Run AI on an iPhone?
Apple is reportedly in talks to acquire AI startup PrismML, whose 1-bit extreme quantization could run large models on iPhone. Community tests reveal tool-calling failures and high hallucination rates.
Read more →
·3 min
AI Daily: Alibaba Open-Sources Dance Model, World Model Launches, First AI-Native Phone Debuts
Alibaba open-sources 14B dance model Wan-Dancer, AutoNavi launches World Studio, Stepfun debuts AI-native phone STEPS NEO; GPT-5.6 file deletion and AI companion shutdowns spark safety and regulation debates.
Read more →
NVFP4 in Reinforcement Learning Traini…
·2 min
NVFP4 in Reinforcement Learning Training: Stability Challenges and Engineering Practices for 4-Bit Quantization
A deep dive into the stability challenges of NVIDIA NVFP4 (4-bit float) in RL training — covering precision evolution, numerical instability root causes, mixed precision strategies, and dynamic scaling solutions.
Read more →
·3 min
CPU-Native Language Models: Breaking Through Inference Bottlenecks with Memory Bandwidth Co-Design
SiliconLLM builds a CPU-native LLM architecture from scratch, combining selective SSM, ternary (1.58-bit) LUT MLP, and granular MoE, co-designed around the L3 cache bandwidth cliff. Ternary kernels show 4-5x speedup over fp32.
Read more →
·5 min
GLM-5.2 Hands-On Review: A 753B-Parameter Open Model That Beats GPT and Claude on Value
In-depth hands-on review of Zhipu AI's flagship GLM-5.2: a 1M-token context window and API pricing just one-fifth of GPT/Claude. Covers website building, Chrome extensions, 3D game cloning, and agentic workflows.
Read more →
·6 min
The Complete Guide to GLM-5.2: Hands-On Review of an Open-Source Long-Context Model
An in-depth hands-on review of Zhipu AI's flagship GLM-5.2: 1M-token context, strong coding, mature agent workflows—at one-fifth the price of top frontier models. Covers website testing, Cursor integration, MCP tooling, and production migration.
Read more →
·7 min
Electron + LangGraph in Practice: Building an AI Agent Visual Orchestration App from Scratch
A hands-on guide to building an enterprise-grade AI Agent workflow orchestration app with Electron Forge and LangGraph, covering local LLM deployment (Qwen3-0.6B), node-based visual canvas design, and full Function Calling integration.
Read more →