Ollama Tutorial: One-Click Local Deployment of DeepSeek and Other LLMs

What Is Ollama? A One-Minute Overview of This Local LLM Powerhouse

Ollama is an open-source tool for running large language models (LLMs) locally, enabling you to run mainstream models like DeepSeek, Qwen, and Gemma right on your own computer. The project has earned over 170,000 Stars on GitHub, making it one of the hottest open-source projects in the local AI deployment space.

Large Language Models (LLMs) are deep learning models built on the Transformer architecture, trained on massive amounts of text data. The Transformer was introduced by a Google team in their 2017 paper Attention Is All You Need, with its core innovation being the Self-Attention mechanism — it allows the model to "attend to" all other tokens in the input sequence when processing each word, thereby capturing long-range semantic dependencies. Compared to the previously dominant Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs), Transformers not only dramatically improve semantic understanding capabilities but, more critically, their computation process inherently supports parallelization, making large-scale training across thousands of GPUs possible. This architecture laid the technical foundation for all modern large models, from the GPT series to DeepSeek and Qwen.

A model's capability generally correlates with its parameter count — ranging from billions to hundreds of billions of parameters. Parameter count can be roughly understood as the model's "memory" capacity: more parameters mean the model can encode richer language patterns and world knowledge, but correspondingly require more computational resources and storage space. In recent years, with the release of high-quality open-source models like Meta's LLaMA, Alibaba's Qwen, and DeepSeek, large models are no longer exclusive to a handful of tech giants — ordinary developers now have the opportunity to run and fine-tune these models on local hardware.

It's worth noting that the flourishing of the open-source LLM ecosystem had a key turning point: in February 2023, Meta released the LLaMA series, proving for the first time that carefully trained smaller-parameter models (7B-65B) could match or even surpass larger closed-source models on multiple benchmarks. This event triggered an "arms race" in the open-source community, with teams like Mistral, Alibaba's Qwen, and DeepSeek subsequently releasing high-quality open-source models, creating a vibrant landscape. Chinese teams have been particularly outstanding in this wave — DeepSeek attracted global attention with its exceptional reasoning capabilities and training efficiency, while the Qwen series has maintained a leading edge in bilingual Chinese-English capabilities. It's precisely this rich supply of open-source models that created enormous demand for local runtime tools like Ollama.

Ollama is developed in Go, has nearly 16,000 Forks, and enjoys extremely high community activity. Go (also known as Golang) is a programming language released by Google in 2009, renowned for its fast compilation, simple deployment, and excellent concurrency performance. Go compiles into statically linked single binary files with no external runtime dependencies, meaning users can execute the program directly without installing additional language runtimes or libraries. Go's built-in goroutine and channel mechanisms provide a lightweight concurrency model, making it ideal for building high-performance network services and API servers. Docker, Kubernetes, and other cloud-native infrastructure are also developed in Go, which further validates Go's reliability in the systems tooling domain.

Ollama's core philosophy is simple — run a large model with a single command, eliminating the hassle of manually configuring environments, downloading weights, and resolving dependency conflicts.

What Models Does Ollama Support?

Full Coverage of Mainstream Open-Source LLMs

Ollama currently boasts an impressive model lineup, covering essentially all the cutting-edge open-source large models:

Model	Source	Highlights
DeepSeek	DeepSeek	Outstanding reasoning capabilities, excellent cost-efficiency
Qwen	Alibaba Tongyi Qianwen	Strong Chinese language ability, multiple sizes available
Gemma	Google	Lightweight and efficient, suitable for consumer hardware
Kimi-K2.5	Moonshot AI	Recently released, excels at long-text processing
GLM-5	Zhipu AI	Next-generation general language model
MiniMax	MiniMax	Rising star in open-source models

Through Ollama's unified interface, you can freely switch between models from different providers on the same machine without setting up separate runtime environments for each one. This unified management capability is extremely valuable in practice — different models excel at different tasks. For example, DeepSeek performs exceptionally well in mathematical reasoning and code generation, Qwen has advantages in Chinese comprehension and multimodal tasks, while Gemma delivers surprisingly good basic conversational ability in a very small package. Ollama lets you choose the model best suited for your current task as easily as switching tools.

Continuously Updated Model Library

The Ollama team is very quick to add support for new models. Recently released models like Kimi-K2.5 and GLM-5 have already been included in the support list at the earliest opportunity. This means you rarely need to worry about "the model I want to use isn't available." Ollama's Model Library uses a registry mechanism similar to Docker Hub, where community members can also upload and share their customized model versions, further enriching the diversity of available models.

Ollama's Three Core Advantages

Simple Installation, Ready to Use Immediately

Traditionally, running large models locally often involves wrestling with Python environments, CUDA drivers, model format conversions, and a host of other issues. CUDA (Compute Unified Device Architecture) is NVIDIA's parallel computing platform and programming model that allows developers to leverage the massive parallel computing power of GPUs for general-purpose computing tasks. Large model inference involves extensive matrix operations — essentially billions of multiplication and addition operations — and the GPU's thousands of compute cores (for example, the RTX 4090 has over 16,000 CUDA cores) can process these operations simultaneously, delivering performance improvements of tens of times compared to CPUs. With traditional model deployment, users need to manually install matching CUDA versions, cuDNN (CUDA Deep Neural Network acceleration library), and corresponding PyTorch/TensorFlow frameworks. Version compatibility issues are a constant headache — for example, PyTorch 2.x might require CUDA 11.8 or 12.1, but the driver version already installed on the system might be incompatible. This class of problems is jokingly called "CUDA Hell" in the deep learning community.

Ollama packages all of this up, automatically detecting hardware and selecting the optimal inference backend (including NVIDIA GPU, Apple Silicon's Metal, and pure CPU mode). After installation, a single command pulls and runs a model:

ollama run deepseek-r1

For beginners, this experience is as intuitive as pulling a Docker image.

About Apple Silicon's Metal Acceleration: Apple's custom M-series chips (M1/M2/M3/M4) employ a Unified Memory Architecture (UMA), where CPU and GPU share the same high-bandwidth memory pool, eliminating the need to copy data back and forth between system memory and VRAM as in traditional PCs. This design offers a unique advantage for LLM inference — the model's weight parameters can be directly accessed by the GPU, avoiding data transfer bottlenecks. Metal is Apple's GPU programming framework, and Ollama leverages this hardware feature through llama.cpp's Metal backend. This means that even a MacBook with 16GB or 24GB of unified memory can smoothly run quantized models with 7B or even 13B parameters, making Mac users one of Ollama's most active user groups.

It's worth noting that Ollama's ability to run large models smoothly on consumer hardware is largely thanks to model quantization technology. Quantization refers to the process of converting model weights from high-precision floating-point numbers (such as FP32, FP16) to low-precision integers (such as INT8, INT4). Take a 7B parameter model as an example: storing it in FP16 precision requires about 14GB of VRAM, but after 4-bit quantization, it only needs about 4GB. This can compress the model size to 1/4 or even 1/8 of the original while significantly reducing VRAM and memory usage, enabling models that originally required data-center-grade GPUs to run on ordinary laptops.

Internally, Ollama integrates the llama.cpp inference engine, an open-source project initiated by Georgi Gerganov with the original goal of efficiently running LLaMA models in pure CPU environments. llama.cpp is written in C/C++, doesn't depend on heavy frameworks like Python and PyTorch, and optimizes memory management and computational efficiency from the ground up. The engine supports quantized models in GGUF (GPT-Generated Unified Format), a model storage format defined by the llama.cpp community. Compared to the earlier GGML format, GGUF adopts a key-value metadata structure, supports more flexible model information storage (such as tokenizer configuration, model architecture parameters, etc.), and offers better forward compatibility. With controllable precision loss, llama.cpp achieves extremely high inference efficiency. Common quantization levels like Q4_K_M and Q5_K_M follow this naming convention: Q stands for Quantization, the number indicates the quantization bit depth, K indicates the use of the k-quant grouped quantization algorithm, and M represents medium precision configuration (with S for small and L for large). Higher numbers mean higher precision but also more resource consumption — users can choose the appropriate version based on their hardware capabilities. Generally speaking, Q4_K_M offers the best balance between performance and quality, suitable for most use cases.

Data Privacy Guaranteed

The biggest benefit of running large models locally is data security:

• Data never leaves your machine: All conversation content is processed locally and never uploaded to any cloud server. This stands in stark contrast to using cloud services like ChatGPT or Claude — cloud service requests travel over the internet to the provider's servers for processing. Even if the provider promises not to retain data, the transmission process itself carries the risk of interception
• Works offline: Once the model is downloaded, it runs without an internet connection. Model weight files are stored on local disk, and inference computation is performed entirely by the local CPU/GPU, requiring no network connection
• Suitable for sensitive scenarios: Enterprise internal documents, customer data, code reviews, and similar scenarios can all be handled with confidence. In heavily regulated industries like healthcare, finance, and law, local data processing isn't just a security preference — it's a compliance requirement. For example, GDPR (EU General Data Protection Regulation) and China's Data Security Law both impose strict restrictions on cross-border transfer of sensitive data

Cross-Platform Compatible with Stable Performance

Go compiles into a single binary file that runs directly on macOS, Windows, and Linux without dependency hell issues. "Dependency Hell" is a classic software development problem — when a program depends on multiple libraries, and those libraries each depend on different versions of other libraries, version conflicts become nearly inevitable. This problem is particularly severe in the Python ecosystem, where different projects may require different versions of PyTorch, NumPy, and other libraries. Virtual environment management tools (like conda, venv) can alleviate but never fully solve the issue. Ollama completely sidesteps this problem through Go's static compilation. Go's native concurrency capabilities also provide solid performance for model inference services, especially when handling multiple user requests simultaneously — goroutine's lightweight scheduling mechanism is more efficient than traditional thread models.

The Ollama Ecosystem: More Than Just a Command Line

Behind those 170,000+ Stars is a quite mature tool ecosystem:

• Graphical Interfaces: GUI frontends like Open WebUI and Chatbox make it easy for users unfamiliar with the command line. Open WebUI is the most popular choice, offering a ChatGPT-like web chat interface with support for multi-turn conversations, conversation history management, model switching, and more — deployable with a single Docker command
• IDE Integration: Mainstream editors like VS Code and JetBrains have corresponding plugins for calling local models directly while coding. For example, the Continue plugin can use Ollama as a backend to provide code completion, code explanation, refactoring suggestions, and other AI-assisted programming features within the editor — an experience similar to GitHub Copilot but running entirely locally
• API Compatibility: Ollama provides an OpenAI-compatible API interface, making migration costs for existing projects extremely low
• RAG Solutions: Combined with frameworks like LangChain and LlamaIndex, you can quickly build local knowledge base Q&A systems

The Deep Value of OpenAI-Compatible API

OpenAI's Chat Completions API has become the de facto standard interface format for LLM application development. This API defines a structured message format — a list of messages with three roles: system (system prompt), user (user input), and assistant (model response) — along with parameters like temperature (controls output randomness) and max_tokens (limits output length). Virtually all mainstream AI application frameworks, chat interfaces, and development tools prioritize support for this interface specification. Ollama's provision of a locally compatible OpenAI-format API means developers only need to change the API address from OpenAI's cloud endpoint to the local http://localhost:11434, and existing code can switch to local models with almost no modifications. This compatibility dramatically reduces migration costs and allows Ollama to seamlessly integrate into existing AI toolchain ecosystems. In practice, many developers adopt a hybrid strategy of "using Ollama's local models during development to save costs, then switching to cloud APIs in production for stronger performance."

RAG: Making Local Models Understand Your Data Better

RAG (Retrieval-Augmented Generation) is a technical architecture that combines external knowledge bases with large models. The core idea is: when a user asks a question, the system first retrieves the most relevant document fragments from the knowledge base, then feeds these fragments as context along with the question into the large model, allowing it to generate answers based on real materials. This approach effectively mitigates the "hallucination" problem of large models (where models confidently fabricate non-existent information — such as inventing paper citations or making up legal provisions), while enabling the model to answer domain-specific questions not covered in its training data.

The technical pipeline of a RAG system typically includes these key steps: First, document content is converted into high-dimensional vector representations using a text embedding model — these vectors capture the semantic information of the text, with semantically similar texts being closer together in vector space. These vectors are then stored in a vector database (such as ChromaDB, FAISS, Milvus, etc.) that supports efficient similarity retrieval. When a user asks a question, the question is similarly converted to a vector, and the system finds the most relevant document fragments using metrics like cosine similarity or Euclidean distance. Finally, these fragments are concatenated into the prompt and fed to the large model to generate the final answer. Ollama itself also supports running embedding models (such as nomic-embed-text), meaning the entire RAG pipeline — from text vectorization to model inference — can be completed locally.

LangChain and LlamaIndex are currently the two most popular RAG development frameworks. LangChain provides highly modular "Chain" abstractions suitable for building complex multi-step AI workflows; LlamaIndex focuses more on data indexing and retrieval optimization, offering rich data connectors (supporting multiple data sources including PDFs, web pages, databases, etc.). Combined with Ollama's local model inference capabilities, you can quickly build a fully privatized intelligent Q&A system, particularly suitable for scenarios like enterprise internal knowledge management, technical documentation retrieval, and customer service automation.

Who Is Ollama For?

• Individual Developers: Quickly validate ideas, assist with programming, generate documentation — without spending money on API calls. Taking GPT-4-level cloud APIs as an example, costs range from several to tens of dollars per million tokens, while the marginal cost of running local models with Ollama is essentially zero (only electricity costs). For developers who need extensive experimentation and iteration, the savings are substantial
• Enterprise Teams: Build internal knowledge base Q&A systems and data analysis assistants while meeting data compliance requirements. Enterprises can deploy Ollama on intranet servers to provide unified AI services to their teams, ensuring both data security and avoiding the per-call costs of cloud APIs
• AI Researchers: Compare different models side by side and provide a foundational environment for fine-tuning experiments. Regarding fine-tuning, while Ollama itself primarily focuses on inference, researchers can use parameter-efficient fine-tuning techniques like LoRA (Low-Rank Adaptation) or QLoRA (Quantized LoRA) to customize models locally. The core idea of LoRA is to freeze most of the original model's parameters and only train a small set of low-rank matrices to adapt to specific tasks, making it possible to fine-tune 7B or even 13B parameter models on a single consumer-grade GPU. Fine-tuned models can be converted to GGUF format for deployment and inference through Ollama
• Privacy-Sensitive Industries: The go-to solution for localized AI deployment in healthcare, legal, finance, and other fields. These industries handle highly sensitive data (such as patient medical records, litigation documents, transaction records), and sending data to third-party cloud services may violate industry regulations. Ollama's local deployment model allows these industries to safely enjoy the efficiency gains that large models bring

Conclusion: Why You Should Try Ollama

What Ollama does sounds simple — it lets everyone run large models on their own computer. But it's precisely this ability to "make complex things simple" that has made it one of the most popular AI tools on GitHub.

As high-quality open-source models like DeepSeek and Qwen continue to emerge, the barrier to local LLM deployment will only keep getting lower. And as the de facto standard tool in this space, Ollama is worth trying for anyone interested in AI.

Whether you want to use large models to assist with daily work or build a complete AI application locally, Ollama is currently the most hassle-free starting point.

Key Takeaways

• Ollama has earned over 170,000 Stars on GitHub, making it the most popular open-source tool for running LLMs locally
• Supports mainstream open-source models including Kimi-K2.5, GLM-5, DeepSeek, Qwen, and Gemma with broad coverage
• Developed in Go, providing a minimalist command-line experience that dramatically lowers the barrier to local AI deployment
• Built-in llama.cpp inference engine and model quantization technology enable smooth operation on consumer hardware
• Local runtime mode ensures data privacy, suitable for enterprise and sensitive use cases
• Provides an OpenAI-compatible API for seamless integration with existing AI development toolchains
• A rich ecosystem has formed around Ollama, including GUI frontends, IDE plugins, and RAG framework integrations