EduRAG Intelligent Q&A System: A Hands-On Breakdown of a Heima AI Agent Project

In the practical deployment of large AI models, RAG (Retrieval-Augmented Generation) is shifting from a "nice-to-have" to a "must-have." The EduRAG Intelligent Q&A System project launched by Heima Programmers is precisely a hands-on agent development course aimed at this trend. According to the official course video on Bilibili, instructor Xing Bu (a core member of the current AI large model development team, formerly at Didi, ByteDance, and TAL) provides a detailed breakdown of the project's architecture, tech stack, and learning path.

Project Positioning: Vertical Q&A for the IT Domain, Solving Factual Hallucinations

EduRAG is an intelligent Q&A system designed for the IT domain. Users input professional questions, and the system provides answers based on an internal knowledge base. According to Xing Bu, the core difference between this project and the web version of ChatGPT is that ChatGPT relies on web searches to generate answers, while EduRAG's knowledge comes from a pre-built professional knowledge base, making it more suitable for enterprise-level consulting scenarios.

Among the four approaches to addressing large model hallucinations (prompting, RAG, fine-tuning, and agents), RAG primarily targets factual hallucinations, while faithfulness hallucinations still require model fine-tuning. This classification provides a theoretical basis for subsequent technology selection.

The difference between factual hallucinations and faithfulness hallucinations: Factual hallucinations refer to the model generating content that does not match objective facts—for example, writing an incorrect parameter for an API or fabricating a non-existent library function. Such issues can be corrected by introducing external authoritative knowledge bases through RAG. Faithfulness hallucinations refer to the model's answer appearing reasonable but deviating from user instructions or input context—for example, the user asks for "a summary in three sentences," but the model outputs a lengthy essay. Such issues stem from the model's instruction-following capability and need to be improved through fine-tuning or RLHF. Understanding the difference in causes between these two types of hallucinations is the key criterion for deciding between RAG and fine-tuning.

RAG (Retrieval-Augmented Generation): Proposed by Facebook AI Research in 2020. Its core idea is to first retrieve relevant document fragments from an external knowledge base before the large model generates an answer, then feed them into the model as context, so that the model's answer is grounded in evidence. RAG's advantages include real-time knowledge updates, no need to retrain the model, and the ability to provide citation sources, making it a mainstream solution for enterprise-level AI applications.

Dual-Module Architecture: RAG Core + FAQ Acceleration

The project is divided into two modules. The RAG module is the core, containing three standard processes: retrieval, augmentation, and generation. According to analysis from Bilibili videos and YouTube tech channels, this architecture is the mainstream practice for current enterprise-level RAG systems.

The FAQ module is used for accelerating high-frequency questions. Xing Bu explains that in consulting scenarios, questions like "large model course fees" appear frequently. If every such question went through the full RAG pipeline, the cost would be too high. The FAQ module significantly reduces response latency and token consumption by caching high-frequency Q&A pairs. This design is consistently recognized in multi-source analyses as a common optimization for industrial-grade RAG.

Key Tech Stack: LangChain, Milvus, and Reranking

The core technical points covered in the course include:

• LangChain: The project's most core framework, running through the entire RAG workflow.
• Hybrid Search: A combination of full-text search + semantic search. Foundations were covered in the Define phase; here it is changed to a code implementation.
• Vector Database Milvus: The most widely used vector database in China, chosen by the course as a teaching focus. Xing Bu mentions that after mastering Milvus, similar databases like Pinecone and Weaviate can be quickly picked up.
• Reranking: A second-stage fine ranking of hybrid search results. Hybrid search is considered "coarse ranking," while the reranking model serves as "fine ranking," improving final context relevance.

One detail: model embedding no longer relies on one-click platform conversion. Instead, the original model files are loaded through the Transformers framework and run on local compute.

Hybrid Search: A single retrieval method has its limitations—full-text search (such as BM25) excels at exact keyword matching but cannot handle synonymous expressions; semantic search (vector similarity) excels at understanding semantics but may miss documents containing key terms. Hybrid search fuses the results of both, usually combined with algorithms like RRF (Reciprocal Rank Fusion) for score merging, and can significantly improve recall. In enterprise knowledge base scenarios, user questions often contain both technical terms and colloquial expressions, making hybrid search a pragmatic choice that balances both.

Reranking: Hybrid search typically returns dozens of candidate documents, of which only a few may be truly relevant. Reranking models (such as BGE-Reranker, Cohere Rerank) use cross-encoders to score each "query-document" pair individually. Their accuracy is far higher than vector similarity, but computational cost is also higher, so fine ranking is only performed on a small number of candidates after coarse ranking. This two-stage "coarse ranking + fine ranking" architecture is the standard paradigm for current RAG systems.

Milvus: An open-source distributed vector database that supports billion-scale vector retrieval and provides multiple index types (such as IVF, HNSW) to balance speed and accuracy. It supports hybrid queries combining scalar filtering and vector retrieval, suitable for complex retrieval needs in enterprise knowledge bases.

Deep Learning and Model Fine-Tuning: Binary Classification to Filter Irrelevant Questions

Although the project is mainly RAG-based, it involves a key model fine-tuning step: performing binary classification on user input to determine whether it is a professional question or an irrelevant question (such as "How do you make scrambled eggs with tomatoes?"). Only professional questions enter the RAG system, avoiding wasted tokens. This design reflects engineering thinking—not all queries are worth going through an expensive pipeline.

According to Xing Bu, the course also includes RAG system evaluation, using metrics similar to F1-score, but requiring a large model to participate in judgment, which differs from traditional model evaluation.

The engineering significance of binary classification filtering: In large model applications, every API call consumes tokens and incurs latency. If a user asks something unrelated to the knowledge base, such as "What's the weather today?", going through the full RAG pipeline not only wastes resources but may also generate low-quality answers because no relevant content can be retrieved. By fine-tuning a small classification model (such as BERT-base) to quickly determine at the entry point whether a question belongs to the professional domain, irrelevant queries can be intercepted outside the RAG system. This practice is known in industry as "query routing" or "intent classification" and is a common means of controlling costs and improving experience.

LLM-as-Judge in RAG evaluation: Traditional NLP evaluation relies on standard answers to calculate F1-score, but RAG answers are natural language and difficult to measure with exact matching. The current mainstream approach is to let a large model act as a judge, scoring dimensions such as faithfulness (whether the answer is based on retrieved content), relevance (whether it is on topic), and completeness. Although this approach introduces bias from the judge model itself, it is the most cost-effective evaluation solution in the absence of manual annotation.

Course Structure and Job Market Conditions

The originally planned 10-day course was compressed into 8 days. Days 1-2 cover background and tools, Day 3 covers Milvus and LangChain, Days 4-5 cover the FAQ module (compressed), Days 6-8 cover the RAG core, and finally project integration and Docker deployment. Later, the EDU Agent project will follow as an upgrade.

Regarding the job market, Xing Bu clearly states in the video: Last year RAG was a bonus; this year it has become a "must-know." The same applies to Agents. These two projects are regarded as the most core hands-on experience for applying to AI positions.

Learning Advice and AI Programming Assistance

The project has a large codebase, and the course focuses on "running + explaining" rather than typing out every line. Xing Bu advises students to focus on understanding the workflow and logic, and recommends using AI programming tools such as Claude Code to assist in understanding the code. He emphasizes that with AI tools, the worry about "not being able to do the job" after joining a company can basically be dismissed, but it takes time to adapt to colleagues' technical language.

For engineers who want to go deep into RAG development, mastering Milvus, LangChain, hybrid search, and reranking are the core capabilities for current enterprise-level projects. EduRAG provides a complete hands-on path from theory to deployment.

One-line Summary

A breakdown of Heima Programmers' EduRAG project: a hands-on course on an enterprise-level Q&A system with a dual-module RAG + FAQ architecture.

Paragraph Summary

This article introduces the EduRAG Intelligent Q&A System course launched by Heima Programmers. The project targets the IT domain and uses a dual-module RAG + FAQ architecture to solve factual hallucination issues. The core tech stack includes the LangChain framework, Milvus vector database, hybrid search, and reranking models, and also includes binary classification model fine-tuning to filter irrelevant questions and optimize token consumption. The course was originally planned for 10 days but compressed into 8, covering background tools, Milvus and LangChain, the FAQ module, the RAG core, and Docker deployment. Instructor Xing Bu points out that RAG has shifted from last year's bonus skill to this year's must-know for job seekers, and mastering Milvus, LangChain, hybrid search, and reranking are the core capabilities for current enterprise-level RAG development.