#vector quantization
290 related articles
llmfit: One Command to Check Which Loc…
·2 min
llmfit: One Command to Check Which Local LLMs Your Hardware Can Run
llmfit is an open-source Rust CLI tool that predicts whether your local hardware can run a given LLM — no model download required. Covers hundreds of models and backends like Ollama and llama.cpp.
Read more →
·3 min
Dify 1.8 Complete Beginner's Guide: Build 20+ AI Apps Fast
A complete Dify 1.8 guide covering 3 deployment methods (Docker/cloud/source), MySQL integration, 5 app types (Chatbot/Agent/Workflow), model selection, and publishing strategies.
Read more →
·3 min
Frontend Developers Transitioning to AI: A Complete Three-Phase Minimalist Roadmap
A structured 3-phase roadmap for frontend developers transitioning to AI: master Transformer fundamentals, build RAG & Agent skills, then advance to model fine-tuning.
Read more →
·2 min
Qwen3 Fine-Tuning in Practice: A Complete Guide to Local Deployment and Domain Training
A hands-on guide to LLM fine-tuning: from understanding model weights to local Qwen3 deployment, dataset preparation, and domain-specific training. Build a complete AI engineering skill set.
Read more →
·3 min
Ornith 9B/35B Hands-On: A New Local Deployment Option for Code Agents
Ornith 1.0 by Deep Reinforce reinforces Qwen 3.5 for code agents. We test Ornith 9B & 35B MoE on Chinese writing, logic, and invoice OCR, with full llama.cpp deployment guide.
Read more →
Goku: A Browser-Based LLM Inference En…
·2 min
Goku: A Browser-Based LLM Inference Engine and Model Manager Built on WebAssembly
Goku is a browser-based LLM inference engine built on WebAssembly and wllama, enabling local AI inference, model management, and offline use — no cloud API needed.
Read more →
·3 min
Bernini Tile-Based Generation: An Engineered Approach to Video Super-Resolution in ComfyUI
Bernini is a ComfyUI video super-resolution node package using Tile Split/Select/Merge to solve seams, drift, and VRAM overflow. Benchmarked at 325s for 39 frames at 1920×1080.
Read more →
·2 min
Abliteration Explained: Building a Custom Uncensored Text Encoder for Krea 2
How a developer ran 4,000 multi-seed abliteration trials to build a 100% HarmBench-compliant uncensored text encoder for Krea 2's Qwen3-VL-4B, with five quantization formats and ComfyUI deployment guide.
Read more →
·4 min
The Local AI Inflection Point: Edge Deployment, Multi-Model Collaboration, and the Open-Source Ecosystem
AI Engineer Summit deep dive: Local AI hits a real inflection point, driven by privacy and cost. Multi-model collaboration goes mainstream, NVIDIA + ExoLabs achieve 10x gains, open-source ecosystem accelerates.
Read more →
·3 min
Google Gemma 4 On-Device Model: Optimized for Pixel 10 TPU — Offline Multimodal Capabilities Tested and Analyzed
Google's Gemma 4 E2B for TPU runs offline on Pixel 10's Tensor G5 chip, enabling local AI chat, image recognition, and audio transcription. We break down the features and real-world test results.
Read more →
NVIDIA DeepStream 9.1 Multi-Camera 3D …
·2 min
NVIDIA DeepStream 9.1 Multi-Camera 3D Tracking: Principles, Architecture, and Applications
A deep dive into NVIDIA DeepStream 9.1 multi-camera 3D tracking: cross-camera Re-ID, 3D coordinate fusion, GPU-accelerated pipelines, and real-world deployments in retail, warehousing, and security.
Read more →
The Guardian Angels Framework: How LLM…
·3 min
The Guardian Angels Framework: How LLM Personalization Can Achieve Both Efficiency and Data Security
The Guardian Angels framework shows how LLM personalization can achieve both productivity and data security through local deployment, differential privacy, and tiered permissions.
Read more →
TurboVec: A Deep Dive into the Rust-Po…
·3 min
TurboVec: A Deep Dive into the Rust-Powered High-Performance Vector Index Library
TurboVec is a Rust-based vector index library powered by TurboQuant, with Python bindings for RAG, semantic search, and AI applications. A deep-dive into its architecture.
Read more →
·3 min
Cherry Studio + MCP: A Practical Guide to Building AI Agents with Local Deployment
Learn how to build an automated AI agent using Cherry Studio, MCP protocol, and locally deployed models — covering DeepSeek integration, web scraping, and private knowledge base setup.
Read more →
·5 min
A Deep Dive into Vector Search for AI Agents: The Technical Foundations of RAG and Long-Term Memory
An in-depth look at the core tech behind AI Agents: how the HNSW, IVF, and PQ vector search algorithms power RAG and long-term memory. Understand where a model's "memory" and "knowledge" come from.
Read more →
·3 min
Cherry Studio + MCP + Ollama: The Complete Guide to Building a Local AI Agent and Private Knowledge Base
A hands-on guide to building a local AI agent and private knowledge base using Cherry Studio, MCP, and Ollama — with web scraping, report generation, and terminal control.
Read more →
GitHub Daily · July 17: AI Coding Infr…
·3 min
GitHub Daily · July 17: AI Coding Infrastructure Explodes — From Context Graphs to Vector Indexes
AI coding infrastructure explodes on GitHub: context management, code graphs, and vector indexes become the new battleground as the community shifts from apps to underlying capabilities.
Read more →
·3 min
The AI Privacy Debate: Why Local Models and Open-Source Frameworks Are Essential
When cloud AI privacy terms hide data-sharing risks, local model deployment and open-source frameworks offer developers a path to true data control. Analyzing xAI, OpenAI, GLM, Kimi, and Qwen.
Read more →
·3 min
Bonsai-27B Extreme Quantization: Running 1-bit LLMs on Consumer Hardware
Bonsai-27B supports binary/ternary extreme quantization for 27B LLMs running on 8GB VRAM. Covers llama.cpp upstream progress, RTX 4060 benchmarks (30 t/s), and real-world limitations.
Read more →
·3 min
Can Small Local Models Become Software Domain Experts? A RAG Architecture Deep Dive
Can small local models (1.5B–3B) become software domain experts? This article breaks down CPT, SFT, RAG, and Agent architectures, with a layered RAG-centric design for CPU-only local deployment.
Read more →