RAG Getting Started in Practice: From Principles to Building an Enterprise Knowledge Base

Large models are powerful, but they only know the data they saw during training. Business documents, product manuals, and customer records inside an enterprise — this "private domain knowledge" is completely unknown to large models. To make large models answer questions based on internal enterprise materials, the mainstream technical approach today is Retrieval-Augmented Generation (RAG). This step-by-step course on Bilibili starts from zero and breaks down RAG's underlying logic, workflow, and practical boundaries quite clearly.

What Problem Does RAG Actually Solve

RAG stands for Retrieval-Augmented Generation. Its core idea is straightforward: after the user asks a question, the system first retrieves content snippets relevant to the question from the enterprise knowledge base, then puts these snippets together with the original question into the large model's prompt, letting the model answer based on these "reference materials."

Compared with another common approach — using tools (Function Calling) to query a database directly and then inject the results into the context — RAG's advantage lies in precise matching. Tool-based retrieval returns large amounts of content, and in multi-turn conversations it easily blows up the context window, causing answer quality to drop sharply afterward. RAG, by contrast, precisely recalls relevant snippets for the user's question, keeping context usage much more controllable.

However, the course also reminds us that even with well-implemented RAG, response accuracy is hard to make 100%. Large models are essentially probabilistic models, and the RAG pipeline has many stages — a problem in any one of them affects the final answer.

How RAG Works: A Complete Pipeline

The course uses a flowchart to connect RAG's working mechanism. Although rough, it is enough for beginners to build an overall understanding.

Knowledge Ingestion Stage

Internal enterprise documents (PDF, Word, Markdown, HTML, Text, etc.) first go through format conversion and preprocessing, then are split into small blocks called "chunks" in the course. The chunking granularity is usually by paragraph, but how exactly to split involves chunking strategy — too large and retrieval is imprecise; too small and semantics easily break apart.

Each chunk is converted into a mathematical vector through an embedding model and stored in a vector database. The course mentions that Milvus is currently the most commonly used in the large model field, though ES, PGVector, Victoria, and others also have their own use cases.

Embedding Model is a neural network model that maps text to high-dimensional vectors. Its core capability is making semantically similar texts closer in vector space, thereby supporting similarity-based retrieval. Mainstream embedding models include OpenAI's text-embedding-3 series, and open-source ones like BGE, M3E, and Sentence-BERT. When choosing an embedding model, pay attention to its supported vector dimensions, maximum input length, multilingual capability, and performance in specific domains (such as legal and medical). It is usually advisable to run small-scale evaluations on actual business data rather than blindly chasing high leaderboard scores.

Vector Database is a system specifically designed for efficiently storing and retrieving vectors, with the core capability of approximate nearest neighbor search (ANN). Milvus is one of the most popular open-source choices, supporting billion-scale vectors; PGVector integrates vector capabilities into PostgreSQL, suitable for teams already using PG; Elasticsearch and Victoria have advantages in full-text search and lightweight scenarios, respectively. When selecting, consider data scale, query latency, operational cost, and ecosystem integration.

Retrieval and Generation Stage

After the user asks a question, the same embedding model converts the question into a vector, similarity computation is performed in the vector database (most commonly cosine similarity), and the chunks closest to the question vector are found. These chunks, combined with the prompt template and the user's original question, are passed to the large model for inference and response generation.

The key to this process lies in the quality of the "retrieval" stage — whether the recalled content is truly relevant directly determines whether the large model can answer correctly.

Cosine Similarity is a metric measuring the directional consistency of two vectors, ranging from -1 to 1, with larger values indicating greater similarity. In RAG, since embedding models usually normalize text vectors, cosine similarity is equivalent to inner product computation and is relatively efficient. Besides cosine similarity, dot product and Euclidean distance (L2) are also common choices, depending on how the embedding model was trained. In actual retrieval, vector databases use approximate nearest neighbor algorithms (such as HNSW, IVF) to trade off between recall and speed, rather than brute-force computing all vectors.

Five Key Factors Affecting RAG Accuracy

The course spends considerable time breaking down why RAG responses are inaccurate — these are also high-frequency interview questions.

Quality of source material processing. If documents contain images or tables that are not parsed and converted, this information is lost at the retrieval stage. If what the user wants to look up happens to be data in a table, the system simply cannot recall it.

Text chunking strategy. If strongly semantically related content is split into two different chunks, only half is recalled during retrieval, and the answer is naturally incomplete. The design of chunking boundaries needs to be based on business semantics.

Embedding model capability. Vectorization quality directly affects the accuracy of similarity computation. The stronger the model's semantic understanding, the higher the relevance of recall.

Retrieval strategy. Should only the single most similar snippet be recalled, or Top-3, Top-5? Should adjacent chunks be included to avoid semantic loss? These strategic choices significantly affect recall performance.

Prompt and generative model capability. Even if retrieval is correct, poorly organized prompts or insufficient reasoning capability of the model itself will still lead to wrong answers. The course specifically mentions that evaluation metrics can distinguish whether the problem lies in the retrieval stage or the generation stage — very practical for engineering troubleshooting.

When Data Volume Is Extremely Large, RAG May Not Be Optimal

The course offers a pragmatic judgment: RAG is suitable for knowledge bases ranging from a few documents to a few hundred documents. If enterprise data reaches TB or even PB scale, RAG's retrieval cost and effectiveness will face challenges. At that point, fine-tuning is more worth considering — letting the model internalize domain knowledge directly into its parameters.

This viewpoint deserves attention. Many teams rush to pile on RAG without thinking clearly about whether data scale and business scenarios match. For scenarios with small data volume, frequent updates, and a need for traceable citations, RAG is the first choice; for scenarios with extremely large data volume, relatively fixed patterns, and sensitivity to inference latency, fine-tuning may be more suitable. The two are not mutually exclusive — they are often combined in real projects.

Fine-tuning refers to continuing to train a pretrained large model with domain data so that the model parameters adapt to a specific task or knowledge. Unlike RAG's "external knowledge base" approach, fine-tuning internalizes knowledge into model weights, requiring no additional retrieval at inference time and offering lower latency, but updating knowledge requires retraining, which is costly and hard to trace. Common fine-tuning methods include full-parameter fine-tuning and parameter-efficient fine-tuning (such as LoRA, QLoRA). RAG and fine-tuning are not opposed: RAG excels at dynamic, traceable knowledge queries, while fine-tuning excels at shaping fixed style, format, and reasoning patterns. Real projects often adopt a combined approach of "fine-tuned model + RAG retrieval."

Practical Advice for Beginners

The value of this course lies in thoroughly explaining the RAG chain from concept to implementation. For developers who want to build a private knowledge base hands-on, it is recommended to proceed in the following order: first get the minimal closed loop working (document parsing → chunking → vectorization → retrieval → generation), then optimize each stage. Chunking strategy and retrieval strategy are the easiest tuning points to see results, while the choice of embedding model and generative model depends on budget and scenario requirements. Do not skip the evaluation stage — without evaluation metrics, optimization is like a blind man trying to understand an elephant.

One-line Summary

A Bilibili course breaks down RAG principles, workflow, accuracy factors, and applicable boundaries, guiding enterprise knowledge base construction.

Paragraph Summary

Based on a Bilibili RAG introductory course, this article systematically organizes the core logic of Retrieval-Augmented Generation (RAG). RAG answers private-domain questions by retrieving relevant snippets from an enterprise knowledge base and injecting them into large model prompts, offering precise context control compared to tool-calling approaches. The article details RAG's two-stage pipeline: knowledge ingestion (document parsing, chunking, vectorized storage) and retrieval generation (question vectorization, similarity retrieval, prompt assembly and generation). It also summarizes five key factors affecting accuracy: material processing quality, chunking strategy, embedding model capability, retrieval strategy, and generative model capability. The article further points out that RAG suits knowledge bases of a few to a few hundred documents, while fine-tuning should be considered when data volume is extremely large, and provides practical advice for beginners.