Local Deployment of DeepSeek + RAG Knowledge Base Tutorial: A Step-by-Step Guide with Ollama + RAGFlow

Why Deploy DeepSeek Locally and Build a RAG Knowledge Base

While the web version of DeepSeek is convenient, it has obvious limitations in practical applications. Consider this scenario: you want an LLM to answer employee questions based on company policies, or predict this year's exam questions based on a dozen past final exam papers. Without providing these materials, the LLM can only "hallucinate" based on its training data.

The web version of LLMs faces three main pain points:

1. Data privacy risks: All conversations and files are uploaded to cloud servers, making the security of sensitive data impossible to guarantee
2. File upload restrictions: The web version imposes strict limits on the number and size of uploaded files, and parsing files requires additional computing power, which usually costs money
3. Cumbersome and inflexible operations: You need to re-upload attachments every time you start a new conversation, and adding or modifying existing attachments is very inconvenient

Therefore, we need to address two core requirements: absolute data privacy protection and personalized knowledge base construction. The former is achieved by deploying the DeepSeek model locally via Ollama, while the latter is achieved through RAG technology.

RAG Technology and Embedding Model Principles Explained

What is RAG (Retrieval Augmented Generation)

RAG (Retrieval Augmented Generation) is one of the mainstream solutions for addressing the LLM "hallucination problem." The so-called "Hallucination" refers to when a model generates content that appears reasonable but is actually incorrect, with extremely high confidence, when it lacks real knowledge support.

This problem stems from the training mechanism of LLMs: large language models represented by GPT and DeepSeek are essentially based on the Transformer architecture, learning language patterns by predicting "the probability distribution of the next token" on massive text corpora. This training approach gives the model the ability to generate fluent, coherent text, but cannot guarantee factual accuracy—the model learns "what expressions are statistically more reasonable" rather than "what content is true in reality." When the model encounters knowledge gaps not covered in its training data, it won't say "I don't know" but instead tends to generate an answer that "looks correct" at the linguistic level—this is the source of hallucinations. RAG introduces an external knowledge base as a real-time reference, decoupling the model's "memory" from "retrieval," fundamentally reducing the probability of hallucinations.

Unlike model fine-tuning, RAG doesn't require retraining the model. Instead, it retrieves relevant content from an external knowledge base before generating answers, using real materials to enhance response accuracy.

Here's an analogy: Fine-tuning is like studying before an exam—the model memorizes knowledge through training before answering; RAG is like an open-book exam—the model first looks through the knowledge base you provide after seeing the question, then formulates its answer.

The RAG workflow consists of three steps:

• Retrieval: When a user asks a question, the system retrieves content fragments related to the question from the external knowledge base
• Augmented: The retrieved information is combined with the user's original question to expand the context available to the model
• Generation: The LLM generates a well-grounded final answer based on the augmented input

The Critical Role of Embedding Models in RAG

Embedding technology maps discrete text symbols into a continuous high-dimensional vector space, so that semantically similar words or sentences are closer together in this space. This distance is typically measured using Cosine Similarity—the smaller the angle between two vectors, the closer their semantics.

Modern Embedding models (such as BERT, text-embedding series) are pre-trained on massive corpora and can capture contextual semantics of words, rather than relying solely on literal matching. The key breakthrough behind this is "context awareness": the same word in different contexts is mapped to different vector positions. For example, "apple" in "eating an apple" and "Apple phone" corresponds to different semantic vectors. This is precisely why RAG can achieve "semantic retrieval" rather than "keyword retrieval"—even if the user's question wording is completely different from the original text in the knowledge base, the system can still accurately match relevant content as long as the semantics are similar.

A machine cannot directly understand the relationship between two words like "SpongeBob" and "Bikini Bottom," but after Embedding processing, semantically related words are mapped to nearby positions in the vector space, enabling semantic-level matching.

The entire process can be understood as follows: after uploading knowledge base files, the Embedding model generates a "semantic fingerprint" for each text segment; when a user asks a question, the question also gets a "semantic fingerprint"; the RAG system takes the user's "fingerprint" and matches it against the most similar content in the knowledge base, then passes the results to DeepSeek to generate an answer.

These "semantic fingerprints" (i.e., vectors) are stored in specialized Vector Databases. Unlike traditional relational databases (such as MySQL) that match by exact fields, the core capability of vector databases is Approximate Nearest Neighbor (ANN) search—through specialized index structures like HNSW (Hierarchical Navigable Small World graphs) and IVF (Inverted File Index), they can find the most similar results to a query vector in milliseconds, even when storing hundreds of millions or billions of high-dimensional vectors. Common vector databases include open-source solutions like Milvus, Qdrant, and Weaviate, as well as cloud services like Pinecone. RAGFlow integrates Elasticsearch as the vector storage backend by default in local deployment, which is why the subsequent "parsing" step is the key to making the knowledge base effective—only after parsing is complete will documents be chunked, vectorized, and written to the database, allowing retrieval to actually occur.

This also explains why web-based LLM chat is free, but file uploads often cost money—because file parsing, Embedding vectorization, and vector database storage all require additional computing power and storage space.

Complete Steps for Local Deployment of DeepSeek + RAG Knowledge Base

The entire deployment process is divided into three steps: deploying DeepSeek with Ollama, deploying RAGFlow via Docker, and building a personal knowledge base to start Q&A.

Step 1: Install Ollama and Deploy the DeepSeek Model

Ollama is an open-source tool specifically designed for running and managing large language models locally. It's easy to use and supports multiple models. Under the hood, it implements efficient CPU/GPU hybrid inference based on llama.cpp and supports quantization compression techniques such as 4-bit and 8-bit—quantization significantly reduces VRAM usage and computation by lowering the numerical precision of model weights (e.g., compressing from 32-bit floating point to 4-bit integers), enabling consumer-grade hardware to run larger models at the cost of slightly reduced answer quality. Visit ollama.com to download and install it. The most critical step is configuring environment variables.

Required environment variables:

• OLLAMA_HOST: Set to 0.0.0.0:11434 so that RAGFlow in the Docker container can access the Ollama service on the host machine
• OLLAMA_MODELS (recommended): Customize the storage path for model files, since LLM files can easily be tens of GBs and the default C drive can fill up quickly

Configuration method (Windows): Settings → System → About → Advanced system settings → Environment Variables → New System Variable

Important reminder: You must restart your computer after configuring environment variables! Otherwise, the new configuration won't take effect and subsequent steps will fail.

After restarting, open the command line (CMD or PowerShell) and execute the following command to download and run DeepSeek:

ollama run deepseek-r1:1.5b

If the model responds to your messages normally, the local DeepSeek deployment is successful.

The model parameter count (the B in 1.5B, 14B, 32B stands for Billion) directly determines model capability and hardware requirements. Parameter count essentially represents the number of learnable weights in the model—more parameters mean the model can encode richer knowledge and reasoning patterns, but the required VRAM and computing resources grow proportionally. Specifically: the 1.5B model requires about 2-3GB of VRAM and can run on ordinary integrated graphics or CPU; the 14B model requires about 10-12GB of VRAM and needs a mid-to-high-end discrete GPU (such as RTX 3080/4070); the 32B model requires 20GB+ of VRAM and typically needs professional-grade GPUs or multi-GPU parallel setups. If VRAM is insufficient, Ollama will automatically offload some layers to RAM or even disk, but speed will drop significantly. It's recommended to start testing with the smaller 1.5B model to confirm the workflow works, then users with discrete GPUs can try the 14B or 32B versions for better answer quality.

Step 2: Deploy RAGFlow via Docker

RAGFlow is an open-source RAG engine that provides complete knowledge base management and intelligent Q&A functionality. Deploying RAGFlow requires two things: the RAGFlow source code and Docker Desktop.

Download RAGFlow source code: Visit GitHub and search for "RAGFlow