What Exactly Is an AI Agent: The Loop Outside the Model Is the Key

The term "Agent" is used extremely broadly these days—everything from a simple function call to research bots that can run for days on end gets labeled an Agent. But once you strip away the shell, what it actually refers to is quite specific: a language model placed inside a loop. It can take actions, observe results, and decide on the next step, until it produces an answer or gets halted by runtime constraints on turns, time, cost, or policy.

The Same Model, a Different "Shell"

A regular model call is a one-shot round trip: you send a prompt, and the model generates text based on its existing knowledge and whatever you've pasted in, and the interaction ends. It can't look things up, can't verify its own answers, and can't retry—unless you send another request.

When a model becomes an Agent, nothing about the model itself changes. What changes is the code around it. Logic that used to stop after a single response now becomes a harness (or orchestration layer): it feeds the model's tool requests and the tool return results back to the model, round after round.

A simple comparison: a hotel guest writes, "My air conditioner is broken." A regular call would reply with a sympathetic message and promise to contact maintenance—but no work order is actually generated, and no one is actually notified. An Agent, by contrast, would identify the guest and room information, look up the maintenance policy, create a work order, assign available staff, start an SLA timer, and notify the guest.

The difference isn't that the model is smarter—it's that the model is allowed to touch real-world processes.

The Loop: Think, Act, Observe

The most effective way to understand an Agent is to look at its core loop. Suppose an HR assistant has a single tool, search_hr_docs(query), and someone asks: Can unused parental leave for fathers be carried over?

1. Think. The answer requires the parental leave policy, so it should search for it.
2. Act. Call search_hr_docs("parental leave").
3. Observe. It returns "Fathers are entitled to 30 days of parental leave"—this answers "how many days" but not "can it be carried over."
4. Act again. Call search_hr_docs("carry forward policy").
5. Observe again. It returns "Up to 10 days of unused parental leave can be carried over to the next year."
6. Stop. Both facts are in hand, and the model writes the final answer.

If you've written retry logic, this shape isn't unfamiliar. The only difference is that the next action is proposed by the model itself. But the runtime still enforces hard constraints on maximum turns, per-call deadlines, cancellation mechanisms, and total cost budgets—no matter how much the model wants to keep going.

Engineering Insight

The model never directly touches databases, APIs, or files. It issues structured tool requests, which the orchestrator validates, executes, and returns results for. The model can only reach the outside world through the capabilities the application chooses to expose.

This boundary is the cornerstone of the entire security model. It means "what the model can do" is entirely up to you, not up to the model's imagination.

The Tool Use / Function Calling mechanism. The loop described above works because the model has the ability to output structured tool requests—a key capability of recent LLMs, commonly called function calling or tool calling. Here's how it works: the developer includes a list of tools in the request (each with a name, a description of its purpose, and parameter definitions in the form of a JSON Schema), and the model, when generating, no longer only outputs natural language but can output a chunk of structured data conforming to that Schema, for example {"name": "search_hr_docs", "arguments": {"query": "parental leave"}}. The orchestrator parses this data, validates the parameters, actually executes the corresponding function, and splices the result back into the context as a new message. The model itself doesn't execute anything—it merely "says" which tool it wants to call and what parameters to pass. This also explains why the wording of tool descriptions and parameter Schemas significantly affects Agent performance: the model can only rely on that text to judge whether a tool should be used and how to use it.

The Five Components, and Where Autonomy Should Stop

An Agent consists of five parts:

Component	Role
Model	Decides the next step based on everything observed so far
Tools	The only way to read or change anything outside the model's text
Memory and context	The current situation: this conversation and the tool results so far
Policy and guardrails	What the Agent may and may not do, e.g., approvals required beyond refund limits
Loop	Runs think, act, observe, and decides whether to stop

The most common mistake is treating "is it an Agent?" as a black-and-white judgment. In the hotel example above, what the model excels at is recognizing that "the AC is making weird noises" is a maintenance request and picking the correct workflow. The work order lifecycle, SLA calculation, permissions, approvals, and audit events should all be left to deterministic code.

In other words: let the model decide which action to take, and let your workflow engine decide how that action is executed.

This dividing line is key to the success or failure of many Agent projects. Handing too much over to the model's autonomous decisions brings uncontrollable costs and unauditable behavior; handing over too little degrades into a hard-coded rule system. The reasonable approach is to let the model handle what it's genuinely good at—understanding fuzzy natural-language intent, choosing among multiple candidate actions—while leaving state machines, permission checks, idempotency, and auditing to traditional engineering methods.

Idempotency and auditing. The article mentions that "idempotency and auditing" should be left to traditional engineering methods, and this deserves a bit of elaboration. Idempotency means that executing the same operation once and executing it multiple times has the same final effect on system state. This is especially important in Agent scenarios, because the loop may repeatedly issue the same tool call due to timeouts, retries, or model misjudgment—if "create work order" isn't idempotent, it could generate a pile of duplicate work orders. A common practice is to assign a unique identifier (idempotency key) to each intentional operation and have the execution side deduplicate based on it. Auditing means recording every thought, tool request, tool result, and policy interception to form a traceable event stream. Together they ensure that the Agent's behavior neither scrambles state due to retries nor fails to be fully reviewed afterward—this is exactly where "let deterministic code decide how actions are executed" lands in practice.

Why This Definition Matters

Defining an Agent as "model + loop + tools + policy constraints," rather than some mysterious autonomous intelligence, brings several practical benefits.

It makes the Agent's behavior testable—you can test the loop logic, tool implementations, and policy interceptions separately without rerunning the entire model every time. It makes costs predictable—maximum turns and total budget constraints are written right into the runtime, not dependent on the model's "self-discipline." It also gives troubleshooting a clear hierarchy: was it the model choosing the wrong action, the tool returning a wrong result, or the policy wrongly intercepting this call? You can tell at a glance.

The next time you hear the word "Agent," you might want to ask: how is its loop written? What tools are exposed? At which layer are the guardrails set? The answers to these questions determine the actual performance of an Agent system far more than "which model did you use?"

Testability and evaluation (Eval). Breaking an Agent into independently replaceable parts—model, tools, loop, policy—directly yields testability. You can unit-test the loop's termination judgment with fixed inputs without calling a real model; replace tools with stubs to verify how the orchestrator handles error returns; and regression-test policy interception rules using a set of recorded historical tool results. For the parts that genuinely require the model's involvement, you can use offline evaluation: prepare a batch of questions with expected tool call sequences, run them in bulk, and tally the hit rate—rather than eyeballing a few samples every time you make a change. This layered approach to testing and evaluation is key to moving an Agent from "the demo works" to "we can keep iterating and dare to ship it," and it's one of the greatest engineering values of this pragmatic definition.

One-line Summary

The essence of an AI Agent is a language model placed inside a loop—the model doesn't change; what changes is the surrounding orchestration code.

Paragraph Summary

The article points out that although the term Agent is overused, its core definition is quite specific: a language model placed inside a loop, able to take actions, observe results, and decide the next step, until it produces an answer or is halted by runtime constraints. Nothing about the model itself changes—what changes is the surrounding orchestration layer, which feeds the model's tool requests and return results back to it round after round. Using two examples—a hotel AC repair request and an HR policy query—the article shows that the difference between an Agent and a regular call isn't a smarter model but its ability to touch real processes. An Agent consists of five parts: model, tools, memory and context, policy guardrails, and loop. The most critical dividing line is: let the model decide which action to take, and let deterministic code decide how the action is executed. The runtime must enforce hard constraints on maximum turns, deadlines, cancellation, and total cost budgets.