#Ollama llama.cpp vLLM
52 related articles
·3 分钟
Self-Hosting a Code Review AI Agent: From Architecture Design to Engineering Implementation
A deep dive into building and self-hosting a code review AI Agent from scratch, covering architecture design, context management, model selection, and noise control.
阅读全文 →
·4 分钟
Heretic Uncensoring Technology Explained: Practical Applications on Jamba2 and Qwen3.5 Open-Source Models
Deep dive into Heretic uncensoring technology applied to Jamba2-Mini, Qwen3.5-9B, and 27B open-source models, exploring how refusal rates dropped from 97% to 4% and the safety debates involved.
阅读全文 →
·2 分钟
DeepSeek V4 Flash Weights Open-Sourced: A New Option for Lightweight LLM Deployment
DeepSeek V4 Flash model weights reportedly open-sourced. This article analyzes its lightweight positioning, open-weight value, comparisons with closed-source models, and deployment guidance.
阅读全文 →
·3 分钟
Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide
Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.
阅读全文 →
·3 分钟
Ollama's Extra Charges for Kimi 3 Spark Community Controversy: Should Open-Source Models Be Free to Use?
Ollama lists Kimi 3 with extra pay-per-use fees, breaking subscription expectations and sparking debate over open-weight models and AI service pricing tiers.
阅读全文 →
·3 分钟
Kimi K3 Local Deployment Guide: Storage Requirements for q4 and q8 Quantization Explained
Detailed analysis of Kimi K3 quantization deployment options, comparing q4 vs q8 storage requirements, precision trade-offs, and hardware configurations for local self-hosting.
阅读全文 →
·3 分钟
HuggingHack Update: S3 Storage, Dual-Engine Scheduling, and GGUF Inspection Get Major Upgrades
HuggingHack releases major updates with S3/MinIO storage, Ollama + vLLM dual-engine scheduling, GGUF inspection, and local accounts for enterprise-grade local LLM management.
阅读全文 →
·3 分钟
Laguna S 2.1: The Open-Source Coding Model That's 14x Smaller Yet Stronger
Poolside releases Laguna S 2.1, an open-source agentic coding model: 118B total params with only 8B active, scoring 40.4% on DeepSWE — ~4.5x DeepSeek V4 Pro Max. Supports 1M-token context, deployable on a single workstation.
阅读全文 →
·3 分钟
LangChain v1.3 Hands-On Tutorial: Deep Dive into Agents, RAG, and Local Model Deployment
Deep dive into LangChain v1.3: compare LangChain, LangGraph, and DeepAgent paradigms, explore RAG pipelines, multi-agent systems, and local LLM deployment for enterprise AI apps.
阅读全文 →
llmfit: One Command to Check Which Loc…
·2 分钟
llmfit: One Command to Check Which Local LLMs Your Hardware Can Run
llmfit is an open-source Rust CLI tool that predicts whether your local hardware can run a given LLM — no model download required. Covers hundreds of models and backends like Ollama and llama.cpp.
阅读全文 →
·2 分钟
Forge: Adding Reliability Guardrails to Local Model Tool Calling
Forge is an open-source Python middleware for local models (Ollama, llama.cpp, vLLM) that boosts tool-calling reliability via three-layer guardrails: validation, rescue parsing, and retry.
阅读全文 →
·2 分钟
vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment
A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.
阅读全文 →
·3 分钟
The AI Privacy Debate: Why Local Models and Open-Source Frameworks Are Essential
When cloud AI privacy terms hide data-sharing risks, local model deployment and open-source frameworks offer developers a path to true data control. Analyzing xAI, OpenAI, GLM, Kimi, and Qwen.
阅读全文 →
·4 分钟
Running Qwen3.6 27B Locally on a Single RTX 3090: A Complete Guide to Cloud-Rivaling Performance
One used RTX 3090, one 16.8GB GGUF file, and Qwen3.6 27B runs locally offline. SWE-bench score of 77 rivals Claude Sonnet. MTP boosts speed to 59 tok/s. Full local AI coding assistant deployment guide.
阅读全文 →
·2 分钟
QuantaMind: An Open-Source Tool for Testing AI Agent Reliability Locally
QuantaMind is a free, open-source local AI Agent reliability testing tool using pass^k scoring and deterministic evaluation, supporting Ollama, llama.cpp, vLLM, and more.
阅读全文 →
·2 分钟
Production LLM Deployment Guide: Model Selection, GPU Configuration, and Inference Engine Breakdown
A comprehensive decision framework for production LLM deployment: model selection (open-source vs. API), GPU VRAM configuration, and inference engine comparison including vLLM and TensorRT-LLM.
阅读全文 →
·2 分钟
Local AI Agent Reads arXiv So You Don't Have To: Only the Papers That Actually Matter
Research Radar is an open-source local AI agent that fetches arXiv papers daily, scores and filters them in batches, deep-reads summaries, and pushes truly relevant content via Telegram. Supports local models, keeps data on your machine, free and self-hostable.
阅读全文 →
Local Coding Agents in Practice: A Com…
·6 分钟
Local Coding Agents in Practice: A Complete Guide to Replacing Your Claude Subscription with Open-Source Models
An in-depth look at local coding agents—core concepts, advantages, and real challenges. Compare against Claude Code and learn to build a zero-subscription, private AI coding workflow with open-weight models.
阅读全文 →
·5 分钟
Ollama Raises $65M Series B: 85% of Fortune 500 Have Already Deployed Local LLMs
Local LLM tool Ollama closes a $65M Series B, bringing total funding to $88M. With 9M developers and 85% of Fortune 500 having deployed internally, this deep dive explores why enterprises embrace local LLMs: compliance, Agent cost savings, and open-source ecosystem.
阅读全文 →
·5 分钟
Real-World Comparison of 6 Open-Source LLMs for On-Premise Deployment: A Selection Guide to Avoid Pitfalls
A hands-on comparison of 6 open-source LLMs (DeepSeek, Qwen3, Zhipu GLM, Kimi K2, MiniMax M3, Tencent Hunyuan 3) for on-premise deployment—covering hardware cost, inference efficiency, and deployment difficulty.
阅读全文 →