RAG and Agent Interview Essentials: A Full Walkthrough of Retrieval-Augmented Generation

A large model's knowledge is fixed once training is complete, leaving it helpless when faced with private data or the latest information. RAG (Retrieval-Augmented Generation) exists precisely to solve this pain point, and it is also a high-frequency core topic in current large-model job interviews. A Bilibili creator systematically broke down the RAG workflow in a recent tutorial. Based on that explanation and combined with consensus from multiple technical sources, this article lays out a complete RAG knowledge framework for you.

Why RAG Is Needed: The Dilemma of Frozen Knowledge and Fine-Tuning

According to the Bilibili creator's analysis, a large model's knowledge is "frozen" after training—you can ask any model "when is your knowledge cutoff date?" and it will basically give a point in the past. The model knows nothing about anything that happened after its training data cutoff. Although some models can obtain real-time information through web-connected tools, they still cannot directly answer questions about a company's internal documents or private data.

Fine-tuning seems like a way out, but the creator points out that it is costly and time-consuming, and more critically, once knowledge is written into the model's parameters, updating it requires retraining—like "making you memorize a piece of news requires you to go through college all over again." RAG takes a completely different path: instead of stuffing knowledge into parameters, it retrieves external knowledge in real time when the user asks a question, puts the relevant content into the prompt, and hands it to the large model to read. The large model itself has strong reading ability; as long as the content is in the context, it can answer based on it.

Offline Stage: Building the Knowledge Base

The complete RAG workflow can be divided into two major stages: offline knowledge base construction and online retrieval and generation. According to the system architecture diagram shown by the creator, the offline stage includes four key steps.

Document Loading and Chunking

The first step is document loading, reading in raw data in various formats such as PDF, Word, Markdown, web pages, and even database records. The second step is document chunking, splitting documents into small chunks. Why not just store the entire document directly? The creator explains two reasons: embedding models have input length limits; and even without limits, compressing an entire document into a single vector would "average out" detailed information, leading to inaccurate retrieval.

Chunking strategy is an important interview topic. Splitting by fixed character count easily cuts through semantics, so the creator suggests keeping chunk size between 500 and 1000 as a reasonable range—too large and the information is too mixed, making it easy to recall irrelevant content during retrieval; too small and the semantics are incomplete. At the same time, overlap should be set during chunking to avoid complete semantics being cut in the middle.

Vectorization and Storage

The third step is vectorization, the most core part of the offline process. The embedding model converts text into high-dimensional numerical vectors (such as a 1536-dimensional list of floats), which can be understood as a "semantic coordinate system"—texts with similar semantics are located closer together in space. The creator emphasizes that vector retrieval matches semantics rather than keywords, which is the essential difference from traditional search.

Vectorization can represent relevance because of training: the embedding model adjusts weights through contrastive learning on massive amounts of text, pulling texts with similar meanings closer and pushing irrelevant ones farther apart. Once this spatial structure is formed, similar content can be found during querying by calculating vector distance, commonly measured with cosine similarity.

The fourth step is storage, persisting vectors together with the original text into a vector database (such as Milvus, Chroma, etc.). Such databases are specially optimized for storing high-dimensional vectors and similarity search, supporting fast retrieval of the most similar results among tens of millions of vectors.

Embedding Models and Contrastive Learning

Embedding models are usually based on the Transformer architecture (such as BERT, BGE, text-embedding-ada-002, etc.), and their training objective is to make semantically similar texts close to each other in vector space. Contrastive Learning is the core method for achieving this goal: the model receives positive sample pairs (such as two sentences with the same meaning) and negative sample pairs (unrelated sentences), and through a loss function (such as InfoNCE) pulls positive samples closer and pushes negative samples farther apart. The larger and higher-quality the training data, the stronger the semantic discrimination ability of the vector space. In actual model selection, you need to pay attention to the model's maximum input length, dimensions, inference speed, and multilingual support, and ensure that queries and documents are encoded with the same model; otherwise, inconsistent vector spaces will cause retrieval to fail.

Online Stage: Retrieval and Generation

The online stage provides real-time retrieval and response for user questions and includes four core steps.

Query Rewriting and Coarse Ranking

User input is often colloquial and vague, making direct retrieval ineffective. The creator suggests first rewriting and normalizing the question into a form more suitable for retrieval, so that vectorization and matching are more accurate.

Then vector retrieval is performed: vectorize the question, perform similarity matching in the vector database, and return the top-k chunks with the closest distances. This step is extremely fast; even a million-level vector database can return results quickly. But the creator reminds us that vector retrieval is essentially comparing vector distances and does not deeply understand the semantic relationship between the query and the document, so the recalled results inevitably include content that looks close but is actually irrelevant—this step is usually called "coarse ranking."

Reranking and Generation

Reranking (Rerank) is precisely meant to make up for the shortcomings of coarse ranking: the Rerank model concatenates the user question with each candidate chunk, deeply understands relevance, and reorders them to filter out irrelevant content. The creator also mentions that in actual production environments, multi-route recall is usually adopted—in addition to vector retrieval, keyword matching and other methods are also performed, and all recalled results are uniformly reranked during reranking to ensure the returned content is the most relevant.

Finally comes generation: put the user question and the reranked chunks into the prompt and hand it to the large model to generate an answer. The creator emphasizes that the prompt should clearly require the model to answer only based on the provided materials and to say it does not know if the materials do not contain the answer, which can effectively suppress hallucinations. He also reminds us that it is not rare for large models to output logically fluent but factually incorrect content, and users need the ability to judge whether an answer is correct.

Rerank Models and Multi-Route Recall

Rerank models are usually cross-encoders, which concatenate the query and candidate document and feed them into a Transformer to directly output a relevance score. Unlike the bi-encoder used in vector retrieval, a cross-encoder can model fine-grained interactions between the query and the document, so it has higher accuracy, but its computational overhead is large and it cannot pre-build an index, making it suitable only for reranking a small number of candidates (such as top-50). Multi-route recall refers to simultaneously using multiple methods such as vector retrieval, keyword retrieval (such as BM25), and even knowledge graph queries to obtain candidate sets, then merging and deduplicating them before unified reranking. This combination can complement the blind spots of different recall methods; for example, BM25 is good at exact matching of terms, while vector retrieval is good at semantic generalization.

The Core Value of RAG and Interview Key Points

According to the creator's summary, the value of RAG is mainly reflected in two points: first, knowledge can be hot-updated at any time by updating documents in the knowledge base without retraining the model, at low cost; second, answers are traceable, with each answer traceable to a specific chunk, so even if an error occurs, it is possible to locate which piece of knowledge caused the problem.

In interviews, candidates need to completely describe the design workflow and operational steps of RAG. The creator suggests that it is best to show a RAG project in project experience. In addition, the knowledge scope of RAG is still expanding: Graph RAG, which combines RAG with graph databases, can better maintain the associations between knowledge; after the system is built, evaluation and optimization are also needed, focusing on metrics such as recall accuracy, recall rate, and faithfulness, and tools such as RAGAS can be used for evaluation.

It should be clear that a RAG system is not something that is done once and for all after it is built; it requires continuous adjustment and optimization in practice. Only by thoroughly understanding the division of labor between offline knowledge base construction and online retrieval and generation, and then mastering details such as chunking strategy, multi-route recall, reranking, and hallucination suppression, can you truly establish a knowledge framework for RAG.

The RAGAS Evaluation Framework and Core Metrics

RAGAS (Retrieval-Augmented Generation Assessment) is an open-source framework specifically for evaluating RAG systems. It mainly focuses on three dimensions: faithfulness (whether the answer is entirely based on the retrieved context), answer relevance (whether the answer is on topic), and context relevance (whether the retrieved content is relevant to the question). In addition, recall measures whether the retrieval stage found all relevant chunks, and precision measures how many of the recalled results are truly relevant. These metrics are usually automatically calculated with a large model as an LLM-as-a-Judge, without manual annotation, making them suitable for rapid feedback during iterative optimization.

Basic Principles of Graph RAG

Graph RAG combines knowledge graphs with RAG. In the offline stage, it not only chunks and vectorizes text but also uses a large model to extract entities and relationships to build a graph structure (nodes represent entities, edges represent relationships). During online retrieval, in addition to vector similarity, multi-hop reasoning can be performed along relationships in the graph, thereby answering questions that require cross-document associations. For example, when asking "Which university did the CEO of Company A attend?", traditional RAG may fail because the information is scattered across different documents, while Graph RAG can find the answer through entity linking and relationship paths. The cost of Graph RAG is the high cost of building and maintaining the graph, and it relies heavily on the accuracy of entity extraction.

One-line Summary

RAG solves the problem of frozen knowledge in large models by retrieving external knowledge in real time and injecting it into the prompt, covering the full workflow from offline knowledge base construction to online retrieval and generation.

Paragraph Summary

This article systematically organizes the core knowledge framework of RAG (Retrieval-Augmented Generation). It first explains why RAG is needed: large-model knowledge is frozen, and fine-tuning is costly and hard to update, while RAG retrieves external knowledge in real time and puts the results into the prompt so the model answers based on context. It then breaks down RAG's two-stage workflow in detail: the offline stage includes document loading, chunking (chunk size 500-1000 with overlap), vectorization (the embedding model converts text into semantic vectors), and storage (vector database); the online stage includes query rewriting, vector retrieval coarse ranking, Rerank fine ranking, and generation (prompt constraints to suppress hallucinations). Finally, it summarizes RAG's core value—hot knowledge updates and traceable answers—and points out interview key points and advanced directions such as Graph RAG and RAGAS evaluation.