Ollama Tutorial: Run DeepSeek and Other LLMs Locally with a Single Command

What Is Ollama? A One-Minute Overview of This Local LLM Runner

Ollama is an open-source project designed to let users easily run various large language models (LLMs) locally. The project has earned over 170,000 Stars on GitHub with nearly 16,000 Forks, making it one of the most popular local LLM tools available today. Written in Go, it has won widespread recognition from the developer community for its clean design and excellent usability.

Put simply, Ollama is like Docker for large models—a single command pulls and runs a model locally, without wrestling with environment setup or dependency conflicts. This analogy is intentional: just as Docker uses containerization to package applications together with their runtime environments, Ollama bundles model weights, runtime configurations, and the inference engine into a unified management unit. Ollama even features a Modelfile similar to a Dockerfile, where users can define the base image, system prompts, temperature parameters, and other settings through declarative syntax, enabling reproducible and shareable model runtime environments.

What Is a Large Language Model (LLM)?

Large language models are deep learning models built on the Transformer architecture and trained on massive text datasets. These models typically range from billions to hundreds of billions of parameters—for example, Llama 3 comes in multiple versions from 8B (8 billion) to 405B (405 billion) parameters. More parameters generally mean stronger comprehension and generation capabilities, but also greater hardware demands. Traditionally, running these models required expensive GPU servers or cloud API services (like OpenAI or Anthropic). Ollama changes this by enabling ordinary developers to run optimized large language models locally on a personal computer with a mid-range GPU or even just a CPU.

What Models Does Ollama Support? Full Coverage of Mainstream Open-Source Models

A Continuously Updated Model Library

Ollama currently supports an extremely rich lineup of models, covering the most cutting-edge open-source models in the AI field:

Model Name	Developer	Highlights
DeepSeek	DeepSeek	Powerful reasoning, outstanding coding performance
Qwen	Alibaba	Excellent Chinese language capabilities, multiple sizes available
Gemma	Google	Lightweight, suitable for resource-constrained devices
Kimi-K2.5	Moonshot AI	Strong performance across multiple benchmarks
GLM-5	Zhipu AI	Latest generation general-purpose language model
MiniMax	MiniMax	High performance, long context support
Llama	Meta	Most active model series in the open-source community

This model list continues to expand—almost every significant open-source model release is supported by Ollama shortly after launch. Notably, these models typically run in quantized versions within Ollama. Quantization is a model compression technique that converts model weights from high-precision floating-point numbers (e.g., FP16, 2 bytes per parameter) to low-precision integers (e.g., INT4, only 0.5 bytes per parameter), shrinking model size to one-quarter or less while drastically reducing runtime memory requirements. Ollama's underlying inference engine, llama.cpp, uses the GGUF (GPT-Generated Unified Format) model format—currently the most mainstream quantized model format for local inference—supporting quantization levels from Q2 to Q8, allowing users to flexibly balance model quality and inference speed based on their hardware.

A Unified Management Framework: Say Goodbye to Environment Configuration Nightmares

In today's rapidly iterating AI landscape, new models emerge constantly. Ollama's value lies not just in providing a runtime environment, but in building a unified model management and execution framework. Users don't need to configure separate Python environments or handle CUDA dependencies for each model—simple commands let you switch between and run different models:

# Run the DeepSeek model
ollama run deepseek-r1

# Switch to the Qwen model
ollama run qwen2.5

# View downloaded models
ollama list

Anyone with AI development experience knows the pain of environment configuration: different models may depend on different versions of PyTorch, different CUDA toolkit versions, or even specific Python versions. An environment that works for one model might throw errors for another. Ollama sidesteps this Python dependency hell entirely by compiling its inference engine into a native binary—it directly calls llama.cpp's C/C++ inference kernels at the low level, bridged through Go's CGo mechanism. Users never need to install Python, pip, or any deep learning framework.

Three Core Advantages of Ollama

Minimal Installation and Usage: One Command Does It All

Ollama's design philosophy is "Get up and running"—getting users started as quickly as possible. Its command-line interface is clean and intuitive, typically requiring just one command to download and run a model. This extremely low barrier to entry means that even users without deep technical backgrounds can get their first local LLM running within minutes.

On macOS, users simply download and double-click the installer. On Linux, a single curl -fsSL https://ollama.com/install.sh | sh completes the deployment. Windows users also have a native installer available. After installation, Ollama launches a lightweight HTTP service in the background (listening on localhost:11434 by default), enabling both direct command-line interaction and REST API calls from other applications.

Data Privacy Through Local Execution

Unlike cloud API calls, Ollama keeps all inference computation local. This is crucial for enterprise users and privacy-conscious individual developers:

• Data stays local: Sensitive information never needs to be uploaded to third-party servers, meeting GDPR and other compliance requirements
• No network dependency: Works normally in offline and intranet environments
• Zero API costs: Beyond hardware costs, there are no usage fees—long-term costs are far lower than cloud solutions

GDPR (General Data Protection Regulation) is the EU's data privacy regulation implemented in 2018, imposing strict restrictions on how companies handle personal data, with fines of up to 4% of global annual revenue for violations. Similarly, China's Personal Information Protection Law (PIPL) and California's CCPA impose clear requirements on cross-border data transfers and third-party processing. When companies use cloud AI services, users' prompts and conversation content are sent to the service provider's servers, posing both legal and security risks—especially in industries like finance, healthcare, and law that handle large amounts of sensitive data. Ollama's local execution model fundamentally eliminates the possibility of data leakage, with all computation occurring on hardware under the user's own control.

Active Community and Rich Ecosystem

Behind the 170,000+ Stars and nearly 16,000 Forks is an extremely active developer community. Numerous third-party tools, plugins, and integration solutions are built around Ollama, forming a rich ecosystem:

• Web UI: Visual chat interfaces like Open WebUI and Chatbox
• IDE Plugins: AI coding assistant integrations for VS Code and JetBrains IDEs
• API Compatibility: Compatible with the OpenAI API format, enabling near-zero-modification migration of existing code
• Development Frameworks: Native support from mainstream frameworks like LangChain and LlamaIndex

The OpenAI API format compatibility deserves special attention. OpenAI's Chat Completions API has become the de facto standard interface in AI application development—the vast majority of AI applications, development frameworks, and toolchains are built on this specification. By providing a locally hosted HTTP endpoint fully compatible with the OpenAI API format, Ollama allows developers to seamlessly switch existing AI applications to local models simply by changing the API address from https://api.openai.com to http://localhost:11434, without modifying any business logic code. This dramatically lowers the technical barrier to migrating from cloud to local deployment.

LangChain and LlamaIndex are the most mainstream LLM application development frameworks today. LangChain provides abstraction layers for chain-of-thought calls, Agents, and memory management, helping developers rapidly build complex AI applications. LlamaIndex focuses on connecting external data (documents, databases, etc.) with large models to enable Retrieval-Augmented Generation (RAG). Both frameworks have natively integrated Ollama as a backend inference engine, allowing developers to specify local Ollama models directly in their code while enjoying a development experience consistent with cloud APIs.

Ollama Technical Architecture Explained

Ollama is developed in Go, a technical choice that brings several significant advantages:

• High performance: Go's compiled nature and efficient concurrency model ensure strong runtime performance
• Cross-platform deployment: Native support for macOS, Linux, and Windows with ready-to-use installers
• Single binary distribution: No complex dependency management needed—one executable file is all it takes

Go (also known as Golang) was released by Google in 2009, designed specifically for building high-performance system software. Its core strengths include: compiling to statically-linked native binaries that don't depend on any runtime environment (unlike Java which needs a JVM, or Python which needs an interpreter); a built-in goroutine concurrency model that easily handles thousands of concurrent connections—critical for Ollama serving multiple API requests simultaneously; and cross-compilation capabilities that make building binaries for all major operating systems from a single platform extremely simple. Well-known infrastructure tools like Docker, Kubernetes, and Terraform are also written in Go—Ollama's choice of Go follows best practices in the systems tooling space.

At the inference layer, Ollama doesn't implement model inference from scratch. Instead, it wraps llama.cpp, a high-performance C/C++ inference engine created by developer Georgi Gerganov. It's currently the most mature local LLM inference solution, supporting CPU inference (accelerated via AVX, AVX2, AVX-512 and other SIMD instruction sets), NVIDIA GPU acceleration (via CUDA), Apple Silicon GPU acceleration (via Metal), and AMD GPU acceleration (via ROCm). Ollama calls llama.cpp's inference kernels through the CGo mechanism, while implementing model management, API services, and concurrency scheduling in Go at the upper layer, forming a cleanly layered architecture.

For model management, Ollama borrows Docker's design philosophy, using an image-layer approach to manage model files for efficient storage and version management. Specifically, a model in Ollama is split into multiple layers, each uniquely identified by a SHA256 hash. When multiple models share the same base weights (e.g., different quantization versions of the same model, or different fine-tuned versions based on the same base model), identical layers are stored only once and shared by reference between models, effectively saving disk space. This design is particularly advantageous when users manage multiple models simultaneously.

Ollama Use Cases: Who Should Use It?

Ollama is particularly well-suited for the following scenarios:

1. Local development and debugging: Quickly test different models like DeepSeek and Qwen to find the best fit for your business needs. In practice, developers often need to compare multiple models' performance on specific tasks—DeepSeek might be better for code generation, while Qwen might excel at Chinese copywriting. Ollama makes this comparative testing extremely convenient, eliminating the need to apply for different API keys or set up different runtime environments for each model.

2. Enterprise private deployment: Deploy AI capabilities in intranet environments to meet data compliance requirements in finance, healthcare, and other industries. Many enterprise production networks are completely isolated from the internet ("air-gapped networks"), making cloud AI services impossible to use. Ollama's offline capability makes it ideal for these scenarios—simply transfer the model files and Ollama binary into the intranet via secure media to provide AI services in a completely disconnected environment.

3. AI learning and research: Experience and compare the capabilities of various open-source models at low cost—perfect for students and researchers. Compared to cloud APIs (GPT-4-level models cost several to tens of dollars per million tokens), the marginal cost of running open-source models locally is virtually zero, allowing researchers to conduct extensive experiments without budget constraints.

4. Rapid prototype validation: Verify the feasibility of AI applications locally before committing cloud resources, reducing trial-and-error costs. Combined with frameworks like LangChain, developers can quickly build prototypes of complex applications such as RAG (Retrieval-Augmented Generation) systems and AI Agents locally, validating the technical approach before deciding whether to migrate to the cloud for scaled deployment.

Conclusion: Why Ollama Is Worth Trying

Ollama's success demonstrates a clear trend: AI democratization is accelerating. As open-source model capabilities continue to improve and hardware costs decline, running large models locally is evolving from a geek's toy into a genuine productivity tool.

Several key forces drive this trend. First, the capability gap between open-source and closed-source models is narrowing rapidly—open-source models like DeepSeek-R1, Llama 3.1, and Qwen 2.5 have approached or even surpassed GPT-4-level closed-source models on multiple benchmarks. Second, AI computing power in consumer hardware continues to grow—Apple Silicon's unified memory architecture lets MacBooks run models with billions of parameters, and NVIDIA's RTX 40/50 series GPUs provide powerful local inference capabilities for PC users. Finally, the maturity of quantization techniques (such as multi-level quantization supported by the GGUF format) allows models that originally required hundreds of GB of VRAM to be compressed to sizes manageable by ordinary laptops, with minimal loss in inference quality.

As one of the most important infrastructure projects for local LLM execution, Ollama's continuously growing Star count—backed by its minimal user experience, rich model support, and active community ecosystem—confirms the market's strong demand for local AI deployment solutions.

For any developer hoping to explore large model capabilities locally, Ollama is currently the most worthwhile tool to try. Open your terminal, type one command, and your local AI journey begins.

Key Takeaways

• Ollama has over 170,000 Stars on GitHub, making it one of the most popular local LLM tools
• Supports mainstream open-source models including Kimi-K2.5, GLM-5, DeepSeek, Qwen, and Gemma with comprehensive ecosystem coverage
• Built on the llama.cpp inference engine with GGUF quantization format, enabling efficient LLM execution on consumer hardware
• Local execution ensures data privacy with no network dependency or API fees, meeting GDPR and other compliance requirements
• Developed in Go with cross-platform support, high performance, and simple deployment
• Compatible with the OpenAI API format, seamlessly integrating with mainstream frameworks like LangChain and LlamaIndex
• Suitable for development/debugging, enterprise private deployment, AI research, and prototype validation