#Georgi Gerganov

65 related articles

Aug 1, 2026·4 min

Heretic Uncensoring Technology Explained: Practical Applications on Jamba2 and Qwen3.5 Open-Source Models

Deep dive into Heretic uncensoring technology applied to Jamba2-Mini, Qwen3.5-9B, and 27B open-source models, exploring how refusal rates dropped from 97% to 4% and the safety debates involved.

Jul 31, 2026·3 min

Practical Guide to Setting Up a Local AI Coding Environment on MacBook Pro M4

Complete guide to setting up a local AI coding environment on MacBook Pro M4, covering Ollama, MLX, Continue, Qwen3-Coder 30B configuration, and performance optimization strategies for 32GB RAM.

Jul 30, 2026·3 min

No Python Required: A Unified Multimodal Local AI Generation CLI Tool

A CLI tool that enables local text, image, video, music, and 3D generation without Python. Explore its technical approach, advantages, limitations, and the growing trend of local AI tooling.

Jul 30, 2026·3 min

No Python Required: A Unified Multimodal Local AI Generation CLI Tool

A CLI tool requiring no Python that supports local text, image, video, music, and 3D generation. Explore its technical approach, advantages, limitations, and the growing trend of local AI toolification.

Jul 29, 2026·2 min

Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon

Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, running fully offline with Qwen models. Explore its technical features, privacy benefits, and comparison with Whisper.

Jul 29, 2026·2 min

Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon

Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, powered by Qwen models for fully offline use. Explore its technical features, privacy benefits, and comparison with Whisper.

Jul 29, 2026·3 min

Local LLM Agent Lagging? Lowering num_gpu Is the Key

Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.

Jul 29, 2026·3 min

Ollama Template Silently Overridden After GGUF Import: A Troubleshooting Guide for Custom TEMPLATE Being Ignored

Analysis of Ollama silently overriding custom TEMPLATE with GGUF-embedded templates after import. Learn to verify active templates via /api/show to prevent evaluation failures.

Jul 28, 2026·3 min

Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac

In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.

Jul 27, 2026·3 min

HuggingHack Update: S3 Storage, Dual-Engine Scheduling, and GGUF Inspection Get Major Upgrades

HuggingHack releases major updates with S3/MinIO storage, Ollama + vLLM dual-engine scheduling, GGUF inspection, and local accounts for enterprise-grade local LLM management.

Jul 25, 2026·3 min

Laguna S 2.1: The Open-Source Coding Model That's 14x Smaller Yet Stronger

Poolside releases Laguna S 2.1, an open-source agentic coding model: 118B total params with only 8B active, scoring 40.4% on DeepSWE — ~4.5x DeepSeek V4 Pro Max. Supports 1M-token context, deployable on a single workstation.

Jul 24, 2026·4 min

jlens-gguf: Bringing Jacobian Lens Visualization and Steering to llama.cpp Local LLMs

jlens-gguf is an open-source tool bringing Anthropic's Jacobian Lens interpretability method to GGUF and llama.cpp, enabling internal observation, real-time steering, and abliteration for both dense and MoE models.

Jul 24, 2026·4 min

Dify Local Deployment Guide: One-Click Private AI Platform with Docker

Step-by-step guide to deploying Dify AI platform locally with Docker. Covers Linux, Windows, macOS setup, docker compose launch, and first-time initialization in under 30 minutes.

llmfit: One Command to Check Which Loc…

Jul 22, 2026·2 min

llmfit: One Command to Check Which Local LLMs Your Hardware Can Run

llmfit is an open-source Rust CLI tool that predicts whether your local hardware can run a given LLM — no model download required. Covers hundreds of models and backends like Ollama and llama.cpp.

transcribe.cpp: A Unified Speech Recog…

Jul 21, 2026·2 min

transcribe.cpp: A Unified Speech Recognition Engine Built on ggml, Supporting 16+ Model Families

transcribe.cpp is an open-source ggml-based speech recognition engine supporting 16+ model families in a single C++ codebase — lightweight, cross-platform, and quantization-ready for local STT.

Jul 20, 2026·3 min

lx: 72 Local CLI Tools That Bring Ollama Models Into Your Terminal

lx is a set of 72 single-purpose CLI tools running on local Ollama models — no API key, fully offline. Supports git commit generation, log debugging, and more. Rust binaries with <15ms cold start; 7–8B models work great.

Goku: A Browser-Based LLM Inference En…

Jul 20, 2026·2 min

Goku: A Browser-Based LLM Inference Engine and Model Manager Built on WebAssembly

Goku is a browser-based LLM inference engine built on WebAssembly and wllama, enabling local AI inference, model management, and offline use — no cloud API needed.

Jul 19, 2026·2 min

Ollama Modelfile in Practice: Deploy Your Own Custom DeepSeek-R1 Model Locally

Step-by-step guide to wrapping DeepSeek-R1 with an Ollama Modelfile — set temperature, system prompts, and run fully offline for privacy and flexibility.

Jul 19, 2026·3 min

The Complete Ollama Guide: The Simplest Way to Run LLMs Locally

Ollama has 9M active developers and 170K GitHub stars. This guide covers its features, architecture, business model, and use cases for running LLMs locally.

Jul 17, 2026·2 min

Running Qwen 122B on 8GB VRAM: A Complete Guide to Quantized Local Deployment

A Bilibili creator ran Qwen 122B with 256K context on just 8GB VRAM + 64GB RAM using llama.cpp. Full breakdown of quantization, deployment params, performance, and cost-effective alternatives.