Build an Agent Yourself: The Best Path to Understanding AI Agents

Why "Building an Agent" Is the Ultimate Way to Understand It

Many people assume a mistaken premise when using AI coding assistants like Codex: that the large model itself reads files, executes commands, and modifies code. In reality, the large model does only one thing—receives messages and outputs messages. What actually opens files, runs commands, and requests permissions for you is the Agent Harness (the runtime skeleton).

According to Bilibili creator Naiyue in the final episode of the "Codex Complete Tutorial" series, the best way to truly understand an Agent is not to keep reading concepts but to build one yourself. This Agent doesn't need to be as powerful or polished as Codex—as long as it can complete one minimal main line: receive a user message, call the model once, execute one tool call, feed the result back to the model, and finally return an answer, that's enough to grasp the core mechanics of an Agent.

DeepSig offers a concise formula: Agent = LLM + Harness. The LLM makes decisions, and the Harness executes them.

The Core of an Agent: The Division of Labor Between the LLM and the Harness

The LLM's scope of work is very clear: it can only receive language messages (including user input, system prompts, tool descriptions, and other context), then output language messages. When it needs to interact with the outside world, it declares a tool-call request in its output—but what actually executes that request is the Harness.

The Harness is an engineering problem. It is responsible for: preparing the messages sent to the model (system prompt, tool descriptions, conversation history), actually calling the model, parsing the tool-call requests the model returns, executing tools, appending tool results to the conversation, calling the model again, and looping until the model produces a final answer.

According to cross-analysis by the YouTube channel "AI Engineering Notes," the same model performs very differently across products, and the core reason is the quality of the Harness's adaptation. Plugging a DeepSeek^Ad model into Codex often performs poorly—not because of the model itself, but because Codex as a Harness isn't well adapted to that model.

Tool Calling / Function Calling is the key interface through which the LLM and the Harness collaborate. In its output, the model doesn't directly execute code; instead, it declares "I want to call a certain tool with these parameters" following an agreed JSON structure, e.g. {"name": "read_file", "arguments": {"path": "a.txt"}}. The Harness parses this structured output, validates the parameters, and then actually calls the local function or external API. This mechanism decouples the model's "intent" from "execution": the model only needs to understand when to use which tool, not how the tool is implemented. Protocol details vary slightly across vendors (e.g., OpenAI's function calling, Anthropic's tool use), but the core idea is the same—expressing action intent through structured messages. Understanding this protocol is a prerequisite for reading the Agent loop.

The Execution Path of a Minimal Agent

The complete execution path of a minimal Agent can be broken down as follows:

1. The user sends a message
2. The Harness prepares the prompt, example messages, and tool descriptions
3. The first call to the LLM
4. The model returns a tool-call request
5. The Harness checks the tool parameters and executes the tool
6. The Harness adds the tool result to the messages and calls the model again
7. The model judges whether the result is sufficient—if so, it gives a final answer; otherwise, it continues requesting tool calls

This process is completely hidden in Codex; the user only perceives "send a message, receive a reply," but in reality a single interaction may involve multiple model calls and tool executions.

The Agent Loop refers to the repeated control flow of "model outputs tool call → Harness executes → result fed back → model called again." Its termination condition is usually when the model no longer requests tools but directly gives a final user-facing answer. Each round of the loop re-sends the conversation history (including prior tool results) to the model, so the context keeps growing—which is why context management is needed. The loop count needs an upper bound to prevent the model from falling into an infinite loop of repeatedly calling the same tool. Understanding the loop's boundary conditions, how messages accumulate, and how to decide when to exit is the dividing line between "calling a model once" and "actually building an Agent."

Framework Selection: Four Agent Frameworks Worth Noticing

Naiyue recommends four frameworks suitable for "learning purposes" rather than production-grade SDKs. Which one to choose depends on your language preference and learning goals:

• LangChain / LangGraph: A veteran framework with complete abstractions, supporting Python and JS/TS. Its strength is a mature ecosystem; its weakness is a high level of abstraction—you might use it without understanding the underlying principles.
• PiAgent: Most recommended for those who want to read the source code before building. The codebase is about 10,000 lines across three layers—the bottom layer calls the model, the middle layer implements the Agent loop, and the top layer handles the Harness. The only barrier is TS.
• DeepSeek Harness: Its plugin-based design is the biggest highlight—model adapters, tools, logging, and even the loop itself are all plugins that can be freely swapped to observe changes. Also mainly TS.
• AgentScope: A domestic open-source framework, primarily Python for the main project, with an officially maintained separate Java version—suitable for developers with a Java background who want to understand enterprise-level implementations.

According to consensus between Bilibili creator Naiyue and the tutorial documentation, it's not advisable to learn four frameworks at once—pick one. In the AI era, the importance of frameworks is declining; the key is understanding the loop and the division of responsibilities of the Harness.

The Eight Stages of Your First Agent

Naiyue suggests making your first Agent a "local file-reading assistant" that exposes only three read-only tools: listing files in the working directory, reading files, and searching content. The read-only design avoids worries about accidental deletion or modification.

The eight progressive stages:

1. Complete a single model call and make it a chat-capable Agent
2. Add a list-files tool to learn how the model proposes tool-call parameters
3. Implement the Agent loop until the model stops calling tools
4. Add error handling and exception handling
5. Add session and context management (trimming, summarization, Memory, RAG)
6. Add a permission system and human approval
7. Add logging and trace observability
8. Consider MCP, RAG, and sub-agents

According to Naiyue, his personal project NetworkTestAgent uses Python + LangChain + LangGraph and gradually added model adaptation, file tools, a UI, permission-based human approval, session history, Memory, MCP, sub-agent background tasks, and more. But after deeper use, he found that LangChain has breaking version updates and permission-approval implementations that diverge from mainstream approaches, undermining the original goal of "saving code with a framework." His current recommendation is to prioritize PiAgent and DeepSeek Harness.

Context Engineering refers to deciding, within a limited context window, which information to send to the model each round. As the Agent loop progresses, conversation history and tool results balloon quickly, so you must control input size and relevance through trimming (dropping old messages), summarization (compressing history), Memory (persisting key information), RAG (retrieving external knowledge on demand), and other techniques. Context engineering directly determines whether an Agent can maintain stable performance on long tasks, and it is the most engineering-intensive part of Harness design—often more important than the choice of model itself.

The Relationship Between RAG and Agents

Naiyue specifically clarifies: RAG and Agents are two different things. RAG doesn't affect building a minimal Agent, so it's not advisable to learn RAG right away. But after finishing your Agent, you can build a separate RAG Agent to learn retrieval, since knowledge bases are a typical Agent application scenario. The complete chain he envisions is: the user gives a task to the main Agent, the main Agent calls the RAG Agent to retrieve and summarize knowledge, and then returns to the main Agent, forming multi-agent collaboration.

MCP (Model Context Protocol) is an open protocol for standardizing "how models connect to external tools and data sources." It abstracts capabilities like tools, resources, and prompt templates into a unified server interface, so an Agent can dynamically discover and invoke external capabilities without writing custom integration code for each data source. You can think of MCP as the "USB interface" of the tool layer: the Harness acts as a client connecting to various MCP servers, and the model uses them indirectly through the tool-calling protocol. This is also why the article places MCP at a later stage—it's an engineering technique for expanding the tool ecosystem after the minimal loop is working.

Editorial View

The value of this tutorial lies in breaking "Agent" down from a vague buzzword into an actionable engineering structure. The LLM handles language reasoning, the Harness handles real-world execution, and the two are connected through messages and the tool-calling protocol. For developers who want to dive deep into AI application development, implementing this loop yourself is more effective than reading ten conceptual articles. Don't agonize over framework selection—write a one- or two-hundred-line minimal loop first, then move to a framework to extend functionality. That's the more pragmatic path.