Local Deployment of the Flint Model: A Guide to Running on Ollama and Integrating with Claude Code

Why Flint Is Worth Noticing

Recursive's Flint is a 40B-parameter open-source coding model that activates only 2.5B parameters per inference. This sparse architecture means it can run on consumer-grade hardware while maintaining coding capabilities close to large dense models. For developers who want to get a Claude Code-like experience locally, Flint combined with Ollama provides an alternative that requires no cloud API.

Sparse Architecture and MoE: The "40B total parameters, 2.5B activated parameters" mentioned in the article usually refers to a Mixture-of-Experts (MoE) architecture. Unlike dense models that compute all parameters on every inference, an MoE model contains multiple "expert" subnetworks and uses a router to select only a few experts per token for computation. Therefore, the total parameter count determines model capacity and the VRAM needed for loading, while the activated parameter count determines the compute required for a single inference. This design lets Flint maintain a large knowledge capacity while significantly reducing inference compute requirements, but it also brings challenges such as uneven expert load and complex routing training. Understanding the difference between total and activated parameters is key to judging its hardware requirements and actual speed.

Running Flint on Ollama

Ollama's model library already supports Flint, and installation follows the same process as other models. First make sure your Ollama version is new enough, then run the pull and run commands. Since the model has 40B total parameters, even with fewer activated parameters you still need sufficient VRAM or memory to load the weights. It is recommended to have at least 24GB of unified memory or VRAM; otherwise CPU offloading may be triggered, causing a drop in speed.

After running, you can interact with Flint through Ollama's API interface, which listens on a local port by default. For coding tasks, it is recommended to explicitly specify the language and framework in the prompt, because local models are more sensitive to context length than cloud services.

Quantization and VRAM Estimation: Models pulled by Ollama by default are often quantized (such as 4-bit or 5-bit) to reduce VRAM usage. 40B parameters require about 80GB of VRAM in FP16, while after 4-bit quantization they require about 20-25GB, which explains the article's recommendation of "at least 24GB of unified memory or VRAM." Quantization loses a small amount of precision, which may affect complex code generation and the stability of tool calls. If hardware is insufficient, Ollama will offload some layers to the CPU, causing a significant drop in inference speed. Therefore, actual experience depends not only on activated parameters but also on quantization level, context length, and batch settings. It is recommended to choose an appropriate quantized version based on available VRAM.

Pointing Claude Code at Flint

Claude Code is a terminal coding assistant launched by Anthropic that uses cloud-based Claude models by default. By modifying environment variables or configuration files, you can forward requests to a local Ollama instance. Specifically, set the Anthropic-compatible API base address to point to Ollama's endpoint, and specify Flint as the model name.

It is worth noting that Claude Code relies on a specific tool-calling format and system prompt, and Flint, as an open-source model, has varying degrees of support for these conventions. In actual testing, simple code generation and file editing work fine, but complex multi-step agent tasks may produce format errors or tool-call failures.

Anthropic-Compatible API and Tool Calling: Claude Code interacts with models through the Anthropic Messages API and relies on a structured tool use protocol. When pointing the API base address to Ollama, you need to ensure that Ollama provides a compatible endpoint or converts request formats through a proxy. Open-source models vary in how well they follow the tool-calling JSON schema, and are prone to missing parameters, format errors, or early termination. In addition, Claude Code's system prompt is tuned for Claude models, and using it directly with Flint may lead to behavioral deviations. In actual deployment, an intermediate layer is usually needed to convert requests/responses and to validate and retry tool-call results in order to improve the reliability of multi-step tasks.

Caveats About Benchmarks

Recursive's published benchmark data should be interpreted with caution. The configuration of 40B total parameters and 2.5B activated parameters has a clear advantage in inference cost, but benchmarks usually target specific task sets and differ from real-world development scenarios. Evaluation results for open-source models are also affected by quantization precision, prompt templates, and the inference backend, and scores may fluctuate considerably across different configurations.

In addition, when using Flint together with Claude Code, the performance bottleneck is often not the model itself but the reliability of tool calls. It is recommended to first validate the workflow on small-scale tasks before gradually expanding usage.

Who It Suits

Flint on Ollama suits developers who have data privacy requirements or want to work in an offline environment. If daily tasks mainly involve code completion, single-file refactoring, and document generation, a local solution is already sufficient. However, for scenarios requiring long-context understanding and complex agent orchestration, cloud models still have clear advantages.

One-line Summary

Flint is an open-source coding model with 40B total parameters and 2.5B activated parameters that can run locally via Ollama and be integrated with Claude Code.

Paragraph Summary

The article introduces Flint, an open-source coding model launched by Recursive. It uses a sparse architecture with 40B total parameters but only 2.5B activated per inference, allowing it to run on consumer-grade hardware. Flint can be pulled and run through Ollama, but at least 24GB of VRAM or unified memory is needed; otherwise CPU offloading is triggered and speed drops. By modifying the API endpoint, Claude Code can be pointed at a local Flint instance, but Flint has limited support for tool-calling formats and system prompts—simple tasks are feasible, while complex multi-step agent tasks are error-prone. The article cautions that benchmarks are affected by quantization precision and prompt templates and should be interpreted carefully, and notes that this solution suits privacy-sensitive and offline scenarios, while cloud models are still recommended for complex tasks.