Agent LLM Primer: From RAG to Agent Tuning Explained

Large language models are eloquent, but when it comes to actually getting work done in business, a simple Q&A exchange is far from enough. A zero-foundation "Agent LLM" tutorial on Bilibili walks through application scenarios, technical frameworks, and training workflows, systematically explaining why Agent technology is an essential path for LLM deployment.

What Are the Typical Application Scenarios for Agents?

The tutorial opens with a few examples to quickly highlight the differences between Agents and ordinary LLM applications.

The first scenario is looking up the source of a poem. After the user asks a question, the Agent automatically calls a search engine to retrieve and summarize results, attaching relevant links to ensure knowledge accuracy.

The second scenario is more everyday: asking about both the time and the weather at once. It seems like a simple question, but the Agent will separately call a calendar tool and a weather tool to get real-time information, then organize it into a fluent natural language response.

The biggest difference from pure conversational apps like ChatGPT is that the Agent displays intermediate steps, its thinking process, and tool call results. The hallucination problem with LLMs has always made users hesitant to fully trust their output, and by presenting a verifiable reasoning chain, the Agent significantly improves answer reliability.

There's another category of scenarios that's even more complex: checking today's sales figures, asking about the cause of traffic jams, looking up Andy Lau's age. These questions are prone to failure under pure LLMs. The tutorial gives a real case: a month ago, when querying Andy Lau's age using Baidu ERNIE, the model directly cited a webpage celebrating his 61st birthday from 2022, giving a wrong answer. Now ERNIE can correctly call a date query tool, calculate the age difference, and output accurate results.

Then there are tasks like booking flights or scheduling meetings, which can't be handled with a simple Q&A. The Agent needs multi-step execution, and even interaction with the user to confirm—after checking flights, it needs to ask which time slot to choose, and after confirmation, complete the payment process.

Why Are LLM Applications Inseparable from Agent Technology?

The tutorial summarizes three core reasons.

The hallucination problem cannot be completely solved from the model itself. LLMs are fundamentally probabilistic generative models, and bias is unavoidable. In serious scenarios, external knowledge must be introduced to ensure answer correctness.

Model parameters cannot be updated in real time. Once trained, it's a static model with no real-time connection to the real world. To get the latest information, external tools and interfaces are the only option.

Complex business needs require task decomposition and multi-step execution. Real-world requirements often can't be solved in one step; the Agent needs to perform task decomposition, step-by-step execution, and interact with the user when necessary.

In one sentence: large language models can talk, but we need them to do things. Agent technology is the key bridge that helps LLMs go from "can chat" to "can work."

The essence of the hallucination problem: LLMs are autoregressive probabilistic generative models. Every token is sampled based on the context distribution. The model has no fact-checking mechanism and no "I don't know" option. When the training data lacks relevant information, or when relevant information conflicts, the model will complete the answer in a fluent but incorrect way. This isn't something that can be eradicated by scaling up parameters or using better corpora, because the probabilistic generation paradigm itself determines that bias is inevitable. The Agent approach isn't about making the model "more accurate" on its own, but rather delegating factual content to external trustworthy sources like search engines, databases, and calendars, with the model only responsible for scheduling and organizing language—thus decoupling "generation" from "facts."

What Does the Agent Technical Framework Look Like?

The tutorial references a framework diagram from an overseas website, with very clear layers.

At the bottom is the traditional Q&A mode: user prompt comes in, the LLM black box gives an answer, simple and direct.

The middle layer is RAG. When users need real-time or accurate information, the system first searches an index, knowledge base, or the entire web to get relevant context and reference materials, then hands them along with the question to the LLM. This way the model answers based on reliable knowledge sources, alleviating hallucination problems.

The top layer is the Agent, with significantly increased complexity:

• Rich toolset: Retrieval is just one category; in actual business there will be various customized tools
• Memory system: Includes short-term and long-term memory
• Planning capability: Decompose problems into subtasks, execute in multiple steps, iterate repeatedly until the task is complete

The Planning process on the right is especially critical—deep thinking, self-critique, step-by-step chain-of-thought decomposition, then generating subtasks and executing them. On the left is the toolset and Action commands, with the Memory mechanism running through the middle. This architecture makes the Agent more complete than RAG and better suited for broad application deployment.

RAG (Retrieval-Augmented Generation): The full name is Retrieval-Augmented Generation. The core idea is that before the model generates an answer, it first retrieves text fragments relevant to the question from an external knowledge base, stitches them into the prompt as context, and then hands it to the model to answer. It separates "memory" from model parameters and puts it into a vector database that can be updated at any time, making it especially suitable for scenarios like enterprise private document Q&A and policy queries where knowledge changes frequently. But RAG is still essentially "Q&A"—it just has more reliable answer sources. It won't proactively decompose tasks, won't execute in multiple steps, and doesn't retain long-term cross-turn memory. This is precisely the key dividing line between it and Agents.

Chain-of-Thought and Self-Critique in Planning: Chain-of-Thought refers to having the model explicitly write out intermediate reasoning steps before giving a final answer, decomposing complex problems into several verifiable small steps. Self-Critique (or Reflexion) means having the model review its own output after each step to check whether it's reasonable, and re-plan if errors are found. Combined, the model is no longer "one generation determines the final outcome," but rather iterates repeatedly to approach the correct solution. This is also the foundation for Agents to handle multi-step tasks like booking flights, writing code, and doing data analysis.

Memory system: Short-term memory typically refers to the context window of the current conversation, storing the most recent several turns of interaction. Long-term memory writes historical interactions, user preferences, and task results into external storage (vector databases, databases, or files), retrieving them when needed. Short-term memory is limited by context length, while long-term memory breaks through this limitation, allowing the Agent to "remember" information from days or even months ago, achieving personalization and cross-session continuity.

Why Do Agent Tuning?

Since you can implement an Agent with just Prompting (like AutoGPT), why bother with Agent Tuning?

The tutorial gives several reasons.

The model needs to be smart enough to understand the "instruction manual." Writing a Prompt is essentially writing a workflow manual for the LLM. But practice has shown that aside from GPT-4-level models, other models—including GPT-3.5—struggle to understand and complete Agent tasks from complex Prompts.

Real-world scenarios don't allow using the strongest APIs. Many businesses require private deployment, and APIs like GPT-4 can't be used. At the same time, inference costs need to be controlled.

Small models can achieve decent Agent capabilities after Agent Tuning. Models with 3B and 7B parameters, after specialized training, can achieve good Agent performance, with lower inference costs and easier deployment.

An analogy: Prompt is like general education, Agent Tuning is like on-the-job training. University students have general knowledge reserves, but when they get to a specific position, targeted training is still needed.

AutoGPT: An open-source experimental project that appeared in 2023, attempting to let GPT-4 autonomously complete complex tasks using only carefully designed Prompts—automatically decomposing goals, calling tools, and looping until completion. It validated the feasibility of "building an Agent with just Prompts" and also exposed obvious shortcomings: massive token consumption, easy to fall into infinite loops, and errors accumulating continuously in long workflows. AutoGPT's significance lies in proving the direction of the Agent paradigm, while also highlighting why Agent Tuning at the model level is necessary—internalizing capabilities into parameters is more stable and token-efficient than relying on external Prompts.

Agent Tuning: Refers to supervised fine-tuning of LLMs using large amounts of Agent task trajectory data (including thinking processes, tool calls, and execution results), teaching the model behavioral patterns like "when to call which tool, how to decompose tasks, and how to backtrack when errors occur." Unlike purely relying on Prompts, Tuning writes Agent capabilities into model weights, so even small models like 3B and 7B can achieve usable Agent performance. This satisfies the compliance requirements of private deployment while bringing inference costs down to an acceptable range.

Agent Tuning R&D Workflow and Cost Considerations

The typical workflow is: first use a strong model like GPT-4 to get the entire Agent process working, then build training data on that basis—automated generation plus manual correction—then perform Agent Tuning to train your own model, and finally replace GPT-4 to complete the loop.

There's a strong correlation between Prompts and training data, so getting the workflow working is a prerequisite for building high-quality data.

But Agent Tuning isn't necessary for every scenario. If you can use APIs and the cost is acceptable, there's no need to do your own training. The tutorial gives a cost reference: 7B model, 4.7 million tokens of training data, 5 training epochs, estimated at Alibaba Cloud A100 prices, requires 4 GPUs to train, and the overall cost is not low. If repeated adjustments are needed, the cost will increase further.

This data can serve as a reference for business decisions—do the math first, then decide whether to go down the Tuning path.

One-line Summary

Agents enable LLMs to go from chatting to working through tool calls, planning, and memory, and small models can gain deployment capabilities through Agent Tuning.

Paragraph Summary

Based on the Bilibili "Agent LLM" tutorial, this article explains why Agent technology is an essential path for LLM deployment. It first uses scenarios like looking up poems, asking about weather, checking sales figures, and booking flights to illustrate how Agents differ from pure conversational apps by calling external tools and displaying reasoning chains, thereby alleviating hallucinations and improving reliability. It then summarizes three major motivations: hallucinations cannot be eradicated from the model itself, parameters cannot be updated in real time, and complex business needs require multi-step decomposition and execution. On the technical framework, the tutorial divides applications into three layers—bottom-level Q&A, middle-layer RAG, and top-layer Agent, with the latter containing three major modules: toolset, Memory, and Planning, where Planning's chain-of-thought and self-critique are especially critical. Finally, it discusses Agent Tuning: because most models can't understand complex Prompts, private deployment scenarios can't use GPT-4, and costs need to be controlled, specialized fine-tuning of small models is very valuable. The workflow is to first get things working with GPT-4, then build data and train, and finally replace the closed-source model—but if APIs are available and costs are acceptable, there's no need for self-training.