What Makes Up an Agentic AI System? Five Core Components

Title

What Makes Up an Agentic AI System? Five Core Components

Meta Description

An agentic AI system consists of five core parts: loop, tools, connection, memory, and coordination. This article breaks down the most confusing points in agent interviews and certification exams, including the MCP protocol, tool-calling mechanisms, memory management strategies, and multi-agent patterns, helping you accurately understand the architectural boundaries of agentic AI systems.

Article Content

The interviewer leaned back in their chair and asked, "So, what is an Agent?" Most candidates answer "an LLM that can use tools," and the interviewer nods but writes nothing down. The same happens in foundational AI certification exams—a question presents four seemingly plausible options, and the wrong ones are derived precisely from that half-right answer.

An article from confidentprep.com offers a stronger answer: an agentic AI system consists of five parts, in a fixed order—loop, tools, connection, memory, and coordination. The article also specifically addresses "when not to add more agents." Each part has a confusion point that interviewers and exam writers deliberately probe, and most of this confusion stems from failing to ask one question clearly: what are the two ends of the connection?

The Loop Comes Before Any Technology: The Defining Feature of an Agent

A strong candidate opens like this:

"An agent is a system that uses a foundation model to decide which actions to take toward a goal. It executes those actions, reads the results returned, and then decides again. It doesn't produce an answer in a single inference. The defining property is that loop. If nothing can come back and change its decision, then it's just a single model call, no matter how good the output."

A foundation model is a large model trained on broad data, on top of which many applications are built. The loop is the heart of the answer—you could even replace every technology in the system, and as long as the loop remains, it's still an agent.

Think of it as the difference between "a researcher with a library card" and "someone answering from memory." The researcher looks things up, and what they find changes their next question; the person answering from memory never gets steered by a surprise.

The agent decides, acts, observes results, then decides again or gives an answer

That branch after results return is the essence of an agent: either act again or give an answer. The model itself doesn't gain new capabilities—the new capability lies in results being sent back as context.

Foundation Model is the prerequisite concept for this discussion. It refers to a large model pretrained in a self-supervised manner on massive, diverse data (such as the GPT, Claude, and Gemini series), possessing broad language understanding and generation capabilities that can be used directly or fine-tuned for downstream tasks. Unlike traditional specialized models trained for a single task, the generality of foundation models makes them an ideal base for building agents—they can carry diverse responsibilities like decision-making, tool calling, and result interpretation without retraining for each scenario. Understanding this is important: an agent's new capabilities don't come from the model itself being replaced or upgraded—the model remains the same foundation model throughout the loop. The real increment is in the system architecture—connecting the model's output to the runtime, and sending the runtime's results back to the model. Therefore, in an interview, if a candidate attributes an agent's capabilities to "a stronger model," it often reveals a lack of understanding at the architectural level.

Follow-up Question 1: The Model Requests, the Runtime Executes

"When an agent uses a tool, what does the model actually do?"

The standard answer: tool use means giving the model access to external functions, APIs, or data sources. The model decides that it needs a certain tool and decides what arguments to pass. The runtime is the software around the model that executes the actual call and returns the result to the context. The model then interprets that result and decides what to do next.

A common mistake among weak candidates is saying "the model queries the database" or "the model booked the appointment." Any answer implying the model directly performed an action gets the mechanism backwards. To an interviewer, this is equivalent to saying: you've never actually watched an agent run.

It's like a doctor ordering a blood test—the doctor doesn't run the lab work themselves; they order it, the lab executes, results come back, and the diagnosis may change as a result.

Runtime is the core concept here and worth expanding on. The runtime is the software layer wrapping the model, responsible for parsing the model's output intent, actually initiating function calls or API requests, capturing returned results, and formatting and re-injecting them into the conversation context. It typically also handles argument validation, timeout handling, error retries, and permission control. The model itself is a pure reasoning component—it has no network access, no file system, and cannot perform any side-effect operations. Distinguishing "model decision" from "runtime execution" is key to understanding agent security boundaries: the model can suggest a database write, but what actually determines whether to execute it and with what credentials is the runtime. This also explains why security policies for agent systems are typically implemented at the runtime layer rather than in the model's prompt—because that's where code capable of causing external effects resides.

Follow-up Question 2: MCP Connects Agents to External Systems

"What is the Model Context Protocol (MCP), and what problem does it eliminate?"

The standard answer: MCP is an open protocol. It standardizes how AI applications connect models to external tools and data sources, replacing custom integrations for each system with a universal interface. The problem it eliminates is "multiplicative": three applications each connecting to four systems requires 12 separate integrations; with a shared protocol, each side implements once, reducing it to 7. Adding a fifth system changes the cost from three implementations to one.

It's like USB-C: previously every device needed its own cable, but with a unified interface standard, any device can connect to any peripheral without either side needing to know the other in advance.

The most common mistakes weak candidates make, in order of frequency:

• "MCP lets agents talk to each other." No. MCP connects agents to external systems. Agent-to-agent communication is a separate concept called communication patterns.
• "MCP is a product or hosted service." It's an open protocol. Cloud services can support it, but that's a different matter. It should never be listed under "optional services."
• "You must use MCP to use tools." Tool use existed before MCP and works without it. When there are only two or three connections, custom integration is acceptable; the value of a shared protocol grows with the number of connection pairs.

One question clears up most of this confusion: what are the two ends of the connection? If one end is an agent and the other is an external system, that's MCP; if both ends are agents, that's a communication pattern.

Model Context Protocol (MCP) was proposed and open-sourced by Anthropic in late 2024, using a client-server architecture. MCP servers expose tools, resources, and prompt templates, and MCP clients (typically AI applications) discover and invoke them in a unified format. The "multiplicative" problem mentioned in the article is mathematically the difference between N×M and N+M: N applications each custom-integrating with M systems requires N×M units of work, while with a shared protocol each side implements once, totaling N+M. This pattern has recurred throughout software engineering history—it follows the same logic as interface standardization for USB, HTTP, and ODBC. It's worth emphasizing that MCP itself doesn't implement any tools—it only specifies "how to describe and invoke tools," while the actual capabilities are still provided by each system's server.

Follow-up Question 3: Memory Is a Decision About What to Keep

"How does an agent remember things?"

The standard answer: the model is stateless between calls. Previous conversation turns must be resent each time, so memory is never automatic and never free. Memory management means deciding what the agent carries forward, for how long, and what to discard. Working memory holds the current task, such as recent turns and the last tool result, and is discarded when the task ends. Persistent memory holds durable facts, such as customer preferences or past results, stored outside the model and selectively retrieved.

A typical mistake weak candidates make: "Just give it the entire conversation history." But everything carried forward is re-provided and recomputed on every call. Cost and latency grow, and the important stuff gets diluted. Retention also carries risk: a one-time verification code placed in persistent storage is a liability, not a feature.

It's like a hospital shift handoff—you don't pass along everything that happened during the entire shift. What needs to be conveyed is, and the handoff is a summary, not a verbatim transcript.

So the useful question isn't "how do we remember more?" but "what must be retained, and what must not be?"

Stateless is the technical foundation for understanding the memory problem. Large model APIs themselves don't store conversation history—each call is independent, and the model can only see what's explicitly passed in the current request. So-called "conversation memory" is actually the application layer re-concatenating historical turns into the context on each call. This has two direct consequences: first, context length is cost, with token counts growing linearly or even faster with each turn; second, the model's "memory" of history is always passive—it cannot actively recall content that wasn't passed in. Therefore, the division between working memory and persistent memory isn't just a design style but an architectural necessity: working memory exists and is destroyed with the task lifecycle, while persistent memory is stored outside the model in vector databases or key-value stores, retrieved and injected only when relevant.

Follow-up Question 4: Arrangement and Message Flow Are Two Independent Questions

"List some multi-agent patterns." Weak candidates just give a list; strong candidates first ask: which kind of pattern do you mean?

The standard answer: multi-agent systems use multiple specialized agents to handle situations where a single general-purpose agent would be stretched too thin. There are two independent questions here.

System patterns describe how agents are organized:

• Supervisor: a lead agent decomposes the task, delegates to specialists, and assembles the results
• Sequential: each agent's output is the next agent's input
• Parallel: independent agents work simultaneously, and results are merged
• Hierarchical: a supervisor of supervisors

Communication patterns describe how agents exchange information:

• Direct (point-to-point): one agent calls another and waits
• Broadcast: one agent notifies multiple at once
• Shared state: agents read and write to common storage
• Message queue: messages are delivered asynchronously

Weak candidates' mistake is answering "Supervisor" when asked how agents exchange information. That answers a different question.

Think of it as an organization: system patterns are the org chart, communication patterns are how memos get passed. A supervisor system can use point-to-point messaging or shared state, and choosing one doesn't determine the other.

Follow-up Question 5: Coordination Owns the Plan, Including When to Stop

"What is workflow orchestration, and who should own it?"

The standard answer: workflow orchestration is coordinating the steps of an agentic application. It covers sequencing, routing, failure handling and retries, and deciding when the task is complete. That last responsibility is easily overlooked. The plan can be owned in two ways: in deterministic orchestration, the developer defines the workflow, which is predictable, testable, and auditable, but cannot adapt to unexpected situations; in model-driven orchestration, the model chooses each step, which can adapt to new situations, but the execution path varies across runs.

Weak candidates' mistake is treating model-driven orchestration as the "advanced default option." In regulated or auditable processes, "the path varies across runs" is a defect.

It's like an air traffic controller—each pilot flies their own aircraft, while the controller decides who moves, in what order, and when the sequence is complete.

Follow-up Question 6: A Single Agent with Good Tools Is the Default

"When would you use multiple agents instead of one?"

The standard answer: use multiple agents only when the task spans genuinely distinct areas of expertise, and be able to state why. A task that's long but stays within the same area of expertise is still one agent doing more steps. Each additional agent amplifies four things: model calls (each with cost), latency (steps waiting on each other), failure modes (any agent can get stuck or pass along bad results), and debugging cost ("which agent made that decision" becomes a real problem).

Weak candidates' mistake: reaching for multi-agent because the word "complex" appeared in the scenario.

The article provides an illustrative case. A logistics assistant answering "Where is my package?" can do so by reading the tracking database. It needs to remember each customer's preferred notification channel. When there's a logistics disruption, its advice needs expertise in pricing, customs, and capacity. The considered design has two paths: ordinary queries go through a single agent with tools; only disruption scenarios go through a supervisor coordinating three specialists. Sending simple queries down the multi-agent path only adds cost and latency without buying anything.

Key Takeaways

• An agent is defined by the loop. If no result can come back and change the decision, it's just a single model call.
• The model requests, the runtime executes. Reject any answer implying the model itself queried, called, or booked something.
• Ask what the two ends of the connection are. Agent-to-external-system is MCP; agent-to-agent is a communication pattern. MCP is a protocol, not a product, and tools work without it.
• Memory is a decision about what to keep. Distinguish working memory from persistent memory; "keep everything" is a cost, not a safe default.
• Structure and message flow are two independent questions, and orchestration owns the plan, including knowing when to stop.
• A single agent with good tools is the default. Escalate only when there are genuinely distinct areas of expertise.

The article's boundaries are also clear: it discusses the "components" of agentic systems, and doesn't answer the business question of "what agents are actually used for," nor does it cover how to choose among specific hosting services for implementing these components, or their costs at scale. Those are separate questions, and exams typically test them under different objectives.

One-line Summary

An agent is defined by the loop; the model only requests and the runtime executes; MCP is a protocol, not a product; memory is about trade-offs; a single agent is the default.

Paragraph Summary

The article centers on the high-frequency interview question "what is an Agent," pointing out that equating an agent simply with "an LLM that can use tools" is a half-right answer easily exploited by wrong options in certification exams. The author proposes that an agentic system consists of five parts—loop, tools, connection, memory, and coordination—in a fixed order. The core argument is that what defines an agent isn't tools or the model, but the loop of "decide—act—observe results—decide again"; without results flowing back to change decisions, it's just a single model call. The article then breaks down six common follow-up questions: the model only requests tools and the runtime executes; MCP is an open protocol standardizing connections between agents and external systems, not a product or agent-to-agent communication mechanism; the core of memory is deciding what to retain and discard; system patterns and communication patterns are two independent questions; orchestration determines the plan, including when to stop; a single agent with good tools is the default, and escalation to multi-agent is only worthwhile when spanning genuinely distinct areas of expertise.