#memory-constrained GPU
11 related articles
Block Low-Rank Compression: A Guide to…
·2 min
Block Low-Rank Compression: A Guide to Accelerating Large Model Inference on Memory-Constrained GPUs
Learn how Block Low-Rank (BLR) decomposition compresses large model memory usage and accelerates GPU inference, including CUDA kernel optimization and combination with quantization and pruning.
Read more →
·3 min
DeepSeek Open-Sources DeepSpec: A Full-Stack Speculative Decoding Library with 60–85% Inference Speedup
DeepSeek open-sources DeepSpec, a full-stack speculative decoding library delivering 60–85% inference speedup and up to 661% throughput gains in production. MIT license, 9 checkpoints, 9 benchmarks.
Read more →
·3 min
Complete Guide to Deploying LLMs Locally on RobotCore Mini: From Ollama Installation to Web Interface Setup
A complete guide to deploying LLMs locally on RobotCore Mini using Ollama — covering model pulling, CLI verification, Web backend setup, and LAN access.
Read more →
·4 min
Two Paths to Better AI Reasoning: Test-Time Scaling and Formal Verification
This article synthesizes two MSR India Summit talks, exploring two key paths to better AI reasoning: test-time scaling with variable granularity search, and a formal verification framework for trustworthy agent execution.
Read more →
·7 min
Building a Published App with Zero Code in Four Days: A Complete HarmonyOS Development Journey
A tech blogger with zero programming knowledge built a retro DV app in four days using AI tools like Cursor and Codex, and got it published on Huawei App Gallery. A full vibe coding walkthrough.
Read more →
·3 min
CPU-Native Language Models: Breaking Through Inference Bottlenecks with Memory Bandwidth Co-Design
SiliconLLM builds a CPU-native LLM architecture from scratch, combining selective SSM, ternary (1.58-bit) LUT MLP, and granular MoE, co-designed around the L3 cache bandwidth cliff. Ternary kernels show 4-5x speedup over fp32.
Read more →
·4 min
AMD Ryzen AI Halo Hands-On: Can x86 Architecture Challenge NVIDIA's DGX Spark?
In-depth review of the AMD Ryzen AI Halo mini AI box: powered by the Ryzen AI Max Plus 395 (Strix Halo) chip with 128GB unified memory, priced at $4,000. Compared against NVIDIA's DGX Spark across token generation, prefill speed, and x86 advantages.
Read more →
·4 min
Unsloth v0.1.463-beta: Fixes Studio Crash and llama-server Permission Issue
Unsloth v0.1.463-beta fixes a Studio crash caused by access-denied errors during llama-server service discovery. Improves stability for multi-user servers and Windows environments.
Read more →
·4 min
ByteDance Open-Sources Bernini Video Editing: Local Deployment Guide for Character Replacement
ByteDance open-sources Bernini, a video editing model supporting character replacement, outfit swapping, and video blending via ComfyUI. Full local deployment guide.
Read more →
·5 min
Defending the Right to Run Local AI: A Battle for Privacy, Freedom, and Digital Sovereignty
Local AI faces a triple threat from tightening regulation, hardware lock-downs, and commercial pressure. A deep analysis of why running open-source LLMs on your own device is a digital right worth defending.
Read more →
·3 min
Needle: Fine-Tuning a 26M Ultralight Tool-Calling Model Locally — Full Walkthrough
Needle is a 26M-parameter tool-calling model. Learn how to replace Gemini with Ollama for local training data generation and fine-tune on a single GPU, achieving 96.7% F1 — ideal for edge AI deployment.
Read more →