Practical Guide to Setting Up a Local AI Coding Environment on MacBook Pro M4

Introduction: The Real Struggles of Local AI Coding

As large language models continue to improve rapidly, more and more developers are exploring running AI coding assistants locally to break free from cloud subscription services like Cursor, Claude Code, and GitHub Copilot. However, as one Reddit developer's real experience shows, this path is far from smooth.

This developer, using a MacBook Pro M4 with 32GB of RAM, spent several days trying various tool combinations including Ollama, Continue, Claude Code, Gemini CLI, and OpenRouter, only to conclude: "I feel like I've spent more time configuring tools than actually writing code." This statement perfectly captures the core pain point of the current local AI coding ecosystem—severe tool fragmentation with no mature, out-of-the-box solutions.

Reddit original post discussion

Technical Options for Local Inference on Apple Silicon

There are currently four main technical approaches for running local large models on Apple Silicon, each with its own pros and cons. To understand the differences between these approaches, you first need to understand Apple Silicon's hardware characteristics: Apple Silicon uses a Unified Memory Architecture (UMA), where the CPU, GPU, and Neural Engine share the same physical memory, eliminating the need to shuttle data between CPU memory and GPU VRAM as in traditional PCs. This architectural feature gives Apple Silicon a unique advantage for running large language models—32GB of unified memory can fully load models that would need to be heavily compressed on consumer NVIDIA GPUs due to limited VRAM (typically 8-24GB). The M4's memory bandwidth is approximately 100-120GB/s, which is lower than the HBM used by data center GPUs (which can reach several TB/s), but is practical enough for the token-by-token generation in inference tasks.

Ollama: The Most Beginner-Friendly Option

Ollama has become the go-to choice for most people thanks to its minimalist installation and model management experience. A single command downloads and runs a model, with native macOS support. Its downside is that it doesn't fully exploit Apple Silicon's performance—while it's based on llama.cpp under the hood, the default configuration may not leverage the full potential of M-series chips.

MLX: Apple's Official Performance Ace

MLX is Apple's own machine learning framework, optimized specifically for Apple Silicon's unified memory architecture. Under the same hardware conditions, MLX typically outperforms other solutions in speed and memory efficiency. MLX's core advantage lies in its deep utilization of Apple Silicon's Metal GPU compute capabilities and the zero-copy nature of unified memory—model weights exist only once in memory, and both CPU and GPU can access them directly, avoiding the data transfer overhead common in traditional frameworks. For developers pursuing peak local performance, MLX paired with mlx-lm is worth the learning investment, though its ecosystem maturity and ease of use are slightly behind Ollama.

LM Studio and llama.cpp

LM Studio provides a graphical interface, suitable for users who prefer visual operations, and comes with a built-in OpenAI-compatible API server. Meanwhile, llama.cpp, as the lowest-level inference engine, offers maximum flexibility but also the highest configuration barrier, suited for tech enthusiasts willing to dive deep into tuning.

It's worth noting that llama.cpp holds a pivotal position in the entire local LLM ecosystem. Open-sourced by Georgi Gerganov in March 2023, its core contribution was proving that large language models can run at acceptable speeds outside the NVIDIA CUDA ecosystem, on pure CPU or non-NVIDIA hardware. The project is written in pure C/C++, with manual optimizations for various hardware instruction sets including ARM NEON (Apple Silicon), AVX2 (x86), and Metal (Apple GPU). Nearly all mainstream local inference tools—Ollama, LM Studio, GPT4All, etc.—rely on llama.cpp as their underlying inference engine, and GGUF (GPT-Generated Unified Format) is its defined model file format that has become the de facto standard for local deployment. Understanding this hierarchy helps developers know which layer to optimize when encountering performance issues.

Model Selection: Realistic Constraints with 32GB RAM

The original poster downloaded three models locally: Gemma, Qwen 3.5 9B, and Qwen3-Coder 30B. With only 32GB of memory, model selection requires careful calculation.

This involves a key technical concept—model quantization. Quantization is the technique of compressing model weights from high-precision floating-point numbers (such as FP16, where each parameter takes 2 bytes) to lower-precision representations (such as INT4, where each parameter takes approximately 0.5 bytes). Taking Qwen3-Coder 30B as an example, at FP16 precision it would require about 60GB of memory, far exceeding the 32GB limit. After 4-bit quantization, memory usage drops to approximately 17-20GB (including quantization metadata and KV cache overhead), making it run smoothly on a 32GB Mac. Common quantization methods include GPTQ (offline quantization based on layer-by-layer calibration), AWQ (activation-aware quantization), and the k-quant series used by the GGUF format. Quantization inevitably loses some precision, but modern quantization algorithms can keep the performance loss of 4-bit quantization within a small range—for relatively structured tasks like code generation, the output quality difference between 4-bit and FP16 models typically doesn't affect practical use.

The Golden Choice for Coding Scenarios

For modern web development workflows involving Python, FastAPI, React, and TypeScript, Qwen3-Coder 30B is currently a well-regarded choice among local coding models. With appropriate quantization (such as 4-bit), a 30B parameter model runs smoothly on the M4 with 32GB unified memory and provides sufficient context understanding for code completion and refactoring.

For scenarios requiring faster response and lower latency (such as real-time code completion), 9B-class models like Qwen 3.5 9B are more appropriate—they sacrifice some capability in exchange for a more agile interactive experience. In terms of actual inference speed, 9B models typically achieve 30-50 tokens/s generation speed on the M4, while 30B models run at around 10-20 tokens/s. This difference is very noticeable in interactive code completion scenarios.

Dual-Model Strategy: Balancing Speed and Quality

In practice, many developers adopt a dual-model strategy: using a small model (7B-9B) for instant completion and a large model (30B) for complex code generation and problem analysis. This layered approach achieves a balance between performance and quality. This strategy is actually inspired by the architecture of commercial tools like Cursor—they use a fast small model in the background for Tab completion, and only invoke the large model when the user actively initiates a conversation or complex edit. Local developers can achieve the same effect by configuring separate autocomplete and chat models in tools like Continue.

Frontend Integration: Making AI Truly Part of Your Development Workflow

With an inference engine and model in place, you still need a good frontend to integrate AI capabilities into your daily development environment.

Comparison of Mainstream IDE Integration Solutions

Continue: An open-source plugin for VS Code and JetBrains that supports connecting to local Ollama or any OpenAI-compatible API, with flexible configuration. It's a popular choice for replacing Copilot. Continue's design philosophy completely decouples the AI backend from the frontend interaction, supporting three interaction modes: Tab autocomplete (inline suggestions similar to Copilot), Chat (discussing code issues with AI in the sidebar), and Edit (selecting code and letting AI modify it directly). It builds context by reading the current file, related files, and project structure, then sends it to the configured model backend. Since it's fully compatible with the OpenAI API protocol, virtually any local inference service that provides this protocol can be seamlessly integrated.
Aider: A command-line tool that excels at Git-based multi-file editing, very friendly for developers who prefer terminal workflows. Aider's core difference from other AI coding tools is its Git-first design philosophy—after each AI modification to code, Aider automatically creates a Git commit with a meaningful commit message. This means developers can easily review, revert, or cherry-pick each AI modification. Aider supports both "whole file" and "diff" editing modes, with the latter having the AI output only the unified diff format patch of changes, which is more efficient when handling large files. It also features a "repo map" capability that analyzes the project's file structure and symbol definitions, helping the model understand the overall architecture of the codebase.
Zed: An emerging high-performance editor with native AI support and excellent Apple Silicon optimization. Written in Rust with a GPU-accelerated rendering layer, Zed's startup and response speed far exceed the Electron-based VS Code, making it particularly suitable for developers pursuing peak performance.
Goose: More oriented toward Agent-style automated task execution, capable of autonomously planning multi-step operations. It's suitable for scenarios requiring AI to complete relatively complex automation workflows (such as scaffolding generation and batch refactoring).

For the tech stack mentioned in the original post, VS Code + Continue + local Ollama/MLX is currently the mainstream combination that balances ease of use and functionality.

Can Local Solutions Truly Replace Cursor?

This is the question every developer exploring local AI coding cares about most. The honest answer is: not yet as a complete replacement, but already practical enough.

Behind cloud services like Claude Code and Cursor are massive-scale models (such as the Claude 3.5/4 series, possibly hundreds of billions of parameters), which still hold advantages in complex reasoning and large codebase understanding that local 30B models cannot match. Specifically, this gap manifests in three dimensions: First, context window—cloud models typically support 128K or even 200K token contexts, while local 30B models are usually limited to 8K-32K effective context due to memory constraints; Second, reasoning ability—the parameter gap directly impacts multi-step logical reasoning, cross-file dependency analysis, and other complex tasks; Third, training data freshness and breadth—cloud services can update their models more frequently.

However, for daily tasks like function writing, code completion, unit test generation, and documentation, local models are fully capable, and they have three irreplaceable advantages:

Privacy and Security: Code never leaves your local device, which is crucial for sensitive projects. In regulated industries like finance, healthcare, and government, the compliance risks of code leakage make cloud AI coding tools difficult to pass security audits, making local solutions the only option.
Zero Marginal Cost: No token billing or subscription fees. Considering Cursor Pro at $20/month and Claude Code potentially costing $50-200/month based on token usage, the hardware investment for a local solution could pay for itself within a year.
Offline Availability: Works without any network connection, including during flights, in secure environments, or with unstable connectivity.

Practical Setup Recommendations for M4 Users

Overall, if you're an Apple Silicon developer looking to build a local AI coding environment that "just feels right," consider the following paths:

Prioritizing Simplicity: Ollama + Qwen3-Coder 30B (Q4_K_M quantization) + VS Code/Continue
Prioritizing Performance: MLX + mlx-lm + Zed or Continue
Prioritizing Terminal Workflow: Ollama + Aider

Some practical tuning tips: Ensure you set a sufficient num_ctx (context length) in Ollama—at least 8192 is recommended for coding scenarios; if using Continue, configure the autocomplete model as a small model and the chat model as a large model to implement the dual-model strategy mentioned above; keep an eye on model updates, as open-source coding models iterate extremely fast, with better options potentially emerging every 1-2 months.

The key is to get it running first, then optimize. Don't get trapped in endless tool comparisons and benchmark numbers—as the original poster reflected, tools serve the purpose of writing code, not the other way around. Settle on a solution that meets 80% of your needs, and invest the remaining energy into actual development work. That's the right way to approach local AI coding.