#layer-by-layer inference
15 related articles
·6 min
How the Creator of Redis Squeezed a 284B-Parameter DeepSeek into an Ordinary Laptop
Redis creator antirez built DS4 "Dwarf Star," a pure-C inference engine, using asymmetric quantization to compress DeepSeek V4 Flash from 500GB to 80.8GB for local 128GB unified-memory deployment at 26.7 tokens/sec.
Read more →
·6 min
How the Creator of Redis Squeezed a 284-Billion-Parameter DeepSeek Into an Ordinary Laptop
Redis author antirez built the pure-C inference engine DS4 "Dwarf Star," compressing DeepSeek V4 Flash from 500GB to 80.8GB via asymmetric quantization for local deployment on 128GB unified memory at 26.7 tokens/sec.
Read more →
·6 min
Vue3 from Scratch: A Hands-On Journey to Building an AI Health Monitoring System
A Vue3 beginner tutorial centered on "learn just enough, apply immediately." A three-stage path covers reactivity, Composition API, Element Plus, and data visualization, culminating in an enterprise-grade AI health monitoring system with blood sugar management, RAG consultation, and doctor-patient collaboration.
Read more →
AirLLM: How a 4GB GPU Can Run a 70B Mo…
·2 min
AirLLM: How a 4GB GPU Can Run a 70B Model — Core Principles and Use Cases
AirLLM is an open-source project that uses layer-by-layer inference to run 70B LLMs on a single 4GB GPU. Learn how it works, its tradeoffs, and ideal use cases.
Read more →
Bonsai 27B: The First 1-bit LLM That R…
·3 min
Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI
Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.
Read more →
·2 min
Distributed AI Systems: A Practical Engineering Guide to Training, Inference, and Production Deployment
A deep dive into distributed AI systems engineering: data/model/tensor parallelism for training, KV cache, quantization, elastic scaling for inference, and cloud deployment with Kubernetes, Ray, and DeepSpeed.
Read more →
Mesh LLM: A Practical Exploration of B…
·3 min
Mesh LLM: A Practical Exploration of Building a P2P Distributed AI Inference Network with iroh
Mesh LLM leverages the Rust P2P framework iroh to integrate compute from scattered nodes, exploring a viable path for decentralized LLM inference. This article analyzes its architecture, challenges, and prospects.
Read more →
·4 min
Unsloth Releases Qwen3.6 NVFP4 Quantization: 2.5x Faster with Zero Accuracy Loss
Unsloth releases NVFP4 quantization for Qwen3.6 using W4A4 true 4-bit Tensor Core computation, delivering up to 2.5x inference speedup over NVIDIA's official implementation with accuracy matching or exceeding BF16 on benchmarks like MMLU-Pro.
Read more →
·6 min
Home Self-Hosting Server Setup Guide: Building a Private Cloud Ecosystem from Scratch
One 32GB server, 8 hours to build a complete home self-hosting ecosystem. This article details the selection logic behind core services like Jellyfin, Forgejo, Immich, and Ollama, plus infrastructure setup essentials for Nginx Proxy Manager and Podman Compose.
Read more →
·7 min
Krea2 Tested: Does the FP8 vs BF16 Quality Gap Still Exist?
A Reddit user compared FP8 and BF16 precision on the Krea2 model and found almost no difference in image quality. This article analyzes the technical reasons behind the shrinking quantization gap and its real-world impact on VRAM usage and inference speed.
Read more →
·6 min
Samsung's Chip Profit in a Single Year Exceeds 40 Years Combined: The AI Wave Triggers a Memory Price Crisis
Samsung's chip division is set to earn more profit in one year than in its past 40 years combined, with quarterly profit surging 19x YoY, surpassing NVIDIA. AI data centers are seizing HBM and DRAM capacity, driving up DDR5 and SSD prices.
Read more →
·7 min
Block-Sparse Featurizers: Opening the Black Box of Vision Model Neural Geometry
Block-sparse featurizers remap dense vision model activations into block-sparse representations, making the internal feature spaces of ViT, CNN, and other models readable and interpretable. This article explores their core principles, links to mechanistic interpretability, and applications.
Read more →
·7 min
Dispersion Loss: A Key Technique for Solving Embedding Condensation in Small Language Models
Embedding condensation is a hidden bottleneck in small language model training. Dispersion Loss combats this by enforcing representation spread during training at zero inference cost.
Read more →
·6 min
Codex Configuration in Practice: A Complete Guide to Accessing GPT-5.5 at Low Cost from China
A detailed guide for Chinese developers on configuring the Codex CLI AI coding tool with GPT-5.5 via API proxies, covering setup steps, efficiency gains, and security risks.
Read more →
Tutorials·3 min
Xingliu Agent Tutorial: Precision Editing, Smart Layer Splitting, and Batch Image Generation — A Complete Hands-On Guide
A complete Xingliu Agent tutorial covering click-to-edit precision, one-click layer splitting, smart Mockup fitting, and batch style generation — boost your design output 10x.
Read more →