Building a Local AI Coding Agent on Mac: A Complete Guide with Ollama, OpenCode, and MCP

Why Run a Coding Agent Locally

Cloud-based AI coding assistants are undeniably powerful, but data privacy, API costs, and network latency remain persistent pain points. One developer shared a complete, hands-on approach: running Gemma 4 or Qwen 3 as a local coding agent on a Mac with 32GB of RAM, using a combination of Ollama + OpenCode + MCP persistent memory.

The core value of this setup is threefold: your code never leaves your machine, there's no subscription fee, and the agent can remember your project state across sessions. For developers who prioritize privacy or need to work offline, this is a solution worth taking seriously.

Choosing an Inference Engine: Ollama vs LM Studio

When building a local AI coding agent, the first decision is which inference engine to use. Ollama and LM Studio are the two most popular options for running local large language models today, each with its own focus.

The Advantages of Ollama

Ollama is command-line-centric, lightweight, easy to script, and provides an OpenAI-compatible API interface. This last point is crucial—because it exposes standard API endpoints, tools like OpenCode can connect to it directly, redirecting requests that would otherwise go to the cloud toward local inference instead.

From a technical standpoint, Ollama is essentially a local model server that wraps the llama.cpp inference engine. Developed by Georgi Gerganov in 2023, llama.cpp enables large language models to run on consumer-grade CPUs/GPUs through an efficient C++ implementation and the GGUF quantization format. On top of this, Ollama provides a Docker-like model management experience—pull and run a model with a single command—while exposing an OpenAI-compatible REST API locally (on port 11434 by default). This API compatibility means that nearly any tool originally built for OpenAI can seamlessly switch to local inference simply by changing the base URL, with no changes to business logic required.

Where LM Studio Fits In

LM Studio, on the other hand, is more oriented toward GUI users, offering a friendlier experience for model management and parameter tuning. It's well suited for scenarios where you want to visually inspect model behavior and manually adjust inference parameters. But if your goal is to integrate a local model into an automated coding workflow, Ollama is the more convenient choice.

Model Selection: Finding Balance Within 32GB of RAM

32GB of RAM is a hard constraint for this setup. The model can't consume all your memory—otherwise your browser, IDE, and other everyday applications will suffer.

Quantization: Fitting Large Models onto Consumer Hardware

To understand memory management, you first need to understand quantization—the key technique that makes large models runnable on consumer hardware. Quantization compresses model weights from 32-bit or 16-bit floating-point numbers into lower-precision integers such as 4-bit or 8-bit, shrinking the model size to between 1/4 and 1/8 of the original. Common quantization levels include Q4_K_M, Q5_K_M, and Q8_0—the higher the number, the greater the precision but also the higher the memory footprint. Take a 7-billion-parameter model as an example: in FP16 format it requires about 14GB of RAM, whereas after Q4_K_M quantization it needs only about 4GB, with a precision loss that's typically within an acceptable range.

The practical recommendation is to prioritize quantized mid-sized models (such as Q4 or Q5 quantized versions in the 7B–14B parameter range), striking a reasonable balance between model capability and memory usage.

The Trade-offs of Context Length

The num_ctx parameter directly affects memory usage. The larger the context window, the more code the model can remember, but memory consumption rises sharply as well—this extra overhead comes from the KV Cache (Key-Value Cache), which needs to store the intermediate states of attention computations for every context token. A 32K context window on a Q4 quantized model may consume an additional 2–4GB of RAM. Under the 32GB constraint, you need to find a balance between "context capacity" and "system smoothness."

The practical recommendation is to set num_ctx to a range that can accommodate typical project context without crowding out system resources, and to test and adjust repeatedly based on the specific model and actual tasks.

Gemma 4's Thinking Mode

The thinking mode offered by Gemma 4 deserves special attention. When handling more complex debugging tasks, enabling thinking mode allows the model to perform deeper reasoning, significantly improving the quality of solutions to complex problems and achieving a reasoning depth on-device that approaches that of cloud models.

This feature draws on the design philosophy of Chain-of-Thought and "letting the model think slowly." It belongs to the same technical lineage as the test-time compute scaling approach used by DeepSeek^Ad-R1 and OpenAI's o1 series: before generating a final answer, the model first produces internal "thinking process" tokens, decomposing the problem, testing hypotheses, and reasoning step by step. This approach is particularly effective for tasks that require multi-step analysis, such as mathematical reasoning, code debugging, and architectural decisions. It's worth noting that the cost of thinking mode is longer inference time and higher token consumption, which is especially pronounced on local hardware—for simple code completion tasks, disabling thinking mode can significantly improve response speed.

Connecting OpenCode to Local Inference

The coding agent layer of this setup is handled by OpenCode. Unlike simple code completion tools, OpenCode is an AI coding agent that runs in the terminal and is capable of autonomously executing multi-step tasks: reading and writing files, executing shell commands, running tests, parsing errors, and iteratively fixing them—forming a closed "perceive–plan–act" loop. It interacts with the operating system through the Tool Use / Function Calling mechanism, which is also why the underlying model should preferably be an instruction-tuned version rather than a base pretrained model.

OpenCode was originally designed to connect to cloud APIs, but by pointing its configuration to the local endpoint provided by Ollama, you can achieve "cloud-grade agent experience with locally executed inference." This step is the critical connection point of the entire architecture: the everyday coding assistant workflow barely changes at all—only the compute behind it migrates from remote servers back to your local Mac. All code, context, and interaction records stay on your machine and never pass through any external server.

MCP Persistent Memory: Letting the Agent Remember Projects Across Sessions

The most common shortcoming of local coding agents is that they "lose their memory" with each new session—developers are forced to repeatedly explain the project structure and past decisions. Introducing an MCP (Model Context Protocol) memory server effectively solves this problem.

The MCP Protocol: A "USB-C Standard" for AI Tool Integration

MCP is an open standard protocol released by Anthropic in November 2024, aimed at addressing the fragmentation of integration between AI models and external tools and data sources. Its design philosophy is similar to USB-C—providing a unified "connector" that lets models access file systems, databases, API services, and memory storage in a standardized way. MCP adopts a client-server architecture: the model host (such as OpenCode) acts as the client, while various tools and data sources expose their capabilities as MCP servers. The memory server is an important component of the MCP ecosystem, typically implemented on top of a vector database (such as SQLite or Qdrant) or key-value storage to achieve cross-session state persistence.

Maintaining Context Across Sessions

By connecting to an MCP memory service, the agent can preserve project state across multiple sessions: remembering prior work progress, code conventions, and contextual information, rather than starting from scratch each time. For developers who maintain the same project over the long term, this is a substantial improvement in experience.

OAuth Authentication and Solution Flexibility

Connecting to an MCP memory service requires completing an OAuth authentication flow to ensure the memory service is secure and controllable. What you might not have noticed is that the MCP memory solution isn't tied to a single tool—this component can be freely swapped out for any memory system compatible with the MCP protocol. Because the protocol is fully open, developers can even build their own local MCP memory server, keeping the memory data entirely on-device as well—highly consistent with the privacy-protection philosophy of the overall solution. This gives the setup excellent extensibility.

Summary: The State and Prospects of Local AI Coding Agents

The combination of Ollama + OpenCode + MCP represents a typical path toward making local AI coding agents practical. The three modules each play their part:

Local inference (Ollama): Wraps the llama.cpp engine, addressing data privacy and API cost concerns.
Agent framework (OpenCode): Provides a complete coding workflow experience with tool-calling capabilities.
Persistent memory (MCP): Uses an open protocol standard to remedy the core weakness of session amnesia.

The 32GB RAM constraint means that local models still lag behind top-tier cloud models on very large contexts and the most complex reasoning tasks. But for everyday coding, privacy-conscious work, or scenarios requiring offline operation, this setup already offers considerable practical value. As open-source model capabilities continue to iterate, quantization techniques keep improving, and consumer hardware specs keep advancing, the feasibility of local AI coding agents will only grow stronger.