The Real Tech Stack for Production-Grade AI Agents: Frameworks Are Just Scaffolding

The term "AI Agent" is being overused. A function that calls a tool is called an Agent. A chatbot with memory is called an Agent. A script with a loop is also called an Agent. This semantic dilution isn't just a naming problem—it's creating real engineering errors: simple pipelines get over-engineered, while genuinely complex systems end up under-designed.

This article distills the hands-on judgment of an engineer who has long built and observed AI systems in the field, focusing on a question that demo videos tend to obscure—what production Agents actually look like.

Where the Boundary of an Agent Lies

The author offers a restrained definition: an Agent is a system that has a goal, not just instructions. It decides on its own what to do next, handles failures, and knows when it's done. Everything else is just a function call in fancy packaging.

This definition works as a yardstick:

• If a system needs a human to tell it step by step what to do, it's a chat interface, not an Agent.
• If a system can retry a failed tool call a different way, it's starting to approach being an Agent.
• If a system can decompose a goal into subtasks and dispatch them, that's a true Agent.

A common misjudgment seen in the comments is teams spending weeks adding "agentic orchestration" to a workflow that a single structured prompt could have solved. Measuring against this yardstick before starting often saves a lot of rework.

The Real Face of Production: Narrow and Deep

Looking at teams that actually ship, the vast majority of Agent deployments are narrow in scope. Customer service ticket triage, document extraction, code review for a specific codebase—these aren't general reasoning engines, but purposeful pipelines with an intelligent decision layer.

The dividing line between teams that get good results and those that get bad ones isn't model selection.

Teams that get good results obsess over three things:

1. Tool design: What the Agent can actually call, and whether the interfaces are clean. Vague tool signatures are the number-one source of failure.
2. Failure handling: How the system degrades, retries, or escalates to a human when a tool returns empty or irrelevant results.
3. Observability: Whether you can trace why the Agent made a particular decision, not just see what it did.

Teams that get bad results share a common pattern: swap GPT-4 for the latest frontier model, change nothing else, and expect behavior to change. Model upgrades don't fix architectural flaws.

Observability in the Agent context isn't just logging. Traditional service observability relies on the trio of metrics, logs, and distributed tracing, but the hard part with Agents is that the decision process itself isn't directly observable: the same input can take different paths, and failures often manifest as "the tool call succeeded but the result is useless." Agent-oriented practice typically requires recording complete traces—every model call, tool input and output, retries and degradations, and intermediate reasoning produced along the way—and stringing them together by session or task so they can be replayed. With traces, teams can answer "why did it choose this" rather than just "what did it do," and can turn failure samples into improvements to tool signatures or prompts. Without this layer, tuning is basically guesswork, and model upgrades can't be verified as genuine improvements.

The Framework Wars Are a Distraction

LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel—a new framework appears every month, and every month someone writes an article declaring the old ones dead. The author's view is blunt: frameworks matter far less than patterns.

A few patterns repeatedly proven effective across frameworks:

• Plan first, then execute: Use one reasoning step to produce a plan and a separate independent step to execute it—don't mix them together.
• Separate retrieval from reasoning: Fetching context and using context are two different things; systems that mash them together tend to get messy.
• Explicit handoffs: When Agents pass work between each other, the handoff should be structured and recorded, not a string stuffed into a prompt.

The author says they've rebuilt the same architecture in three different frameworks, with broadly similar results. Frameworks are scaffolding; architecture is the building itself.

Plan-and-Execute differs from the common ReAct-style think-as-you-go approach: the system first produces an explicit plan in one reasoning pass, then an independent execution step works through it step by step and revises the plan based on results. The value of the separation lies in two things: the plan is an inspectable, cacheable, reusable intermediate artifact that supports human intervention and auditing; and the execution phase can retry, parallelize, or swap tools for individual steps without rerunning the entire reasoning chain. The cost is the rigidity of the plan—when the environment changes quickly, static plans go stale, so in practice it's often combined with replanning. Along with "separate retrieval from reasoning" and "explicit handoffs," it belongs to the same school of thought: break work of different natures into clearly bounded stages instead of cramming everything into a single prompt and letting the model improvise.

The Old Problem with Retrieval: Chunk Boundaries

RAG is standard now, but tutorials rarely explain one thing thoroughly: chunk boundaries are often wrong.

When you split documents into chunks and embed them, you're making assumptions about "which context should stay together," and those assumptions frequently don't hold. A paragraph that can't be understood without its preceding text gets retrieved on its own, and the model fills in the missing context itself—hallucination is born from this.

Improvement directions include overlapping windows, semantic chunking, and parent document retrieval. But the real fix is rethinking what object you store—sometimes what should be stored isn't raw text but a structured representation of the information.

A practical troubleshooting mantra: if RAG returns results that are technically correct but contextually useless, the problem is almost certainly in chunking or metadata, not the embedding model. Swapping embedding models is usually a waste of budget.

Parent Document Retrieval is a retrieval pattern designed for the chunk-boundary problem: at index time, documents are split into small chunks for embedding and similarity matching, but when a chunk is hit, what's returned isn't the chunk itself but the larger unit it belongs to (the parent document or parent paragraph). This keeps vector retrieval granular enough for accurate recall while giving the generation stage complete context, reducing the model's tendency to fill in gaps when preceding text is missing. There are two common implementations: one maintains a mapping from small chunk to parent chunk ID and fetches by ID on a hit; the other uses sentence-window retrieval, expanding several sentences before and after a matched sentence. It can't automatically fix an original structure with broken semantics, but it's clearly effective for problems like "a paragraph can't be understood without its preceding text," at the cost of longer context and an extra layer of indexing in storage.

Industry Developments Worth Noting but Requiring Independent Judgment

While discussing the tech stack, the original article also cites a few industry news items. These are single-source observations, and readers should verify them independently: Anthropic announced it would commit 5% of its shares to a "public interest fund"; Disney agreed to take on most of Character.AI's staff (a "stalking horse" bid in a competitive process, with other companies still possibly bidding); Meta removed wording from a clause that had raised concerns about crawling and AI training, with its spokesperson insisting the crawling approach hadn't changed.

These developments have no direct causal relationship with Agent engineering, but they reflect growing tension between model vendors and the content ecosystem—which will indirectly affect data acquisition and compliance costs.

What the Next Battle Is Really About

Models will keep getting stronger, context windows will keep expanding, and per-token costs will keep falling. None of this changes the core engineering challenge: building a system that behaves correctly when no one is watching.

The things truly worth investing in are governance, observability, and a reliable tooling system—not leaderboard chasing. The author's judgment is that in two years, the truly scarce engineers will be those who can build AI systems that other engineers can maintain and trust—closer to system design ability than to fine-tuning or prompt engineering.

One-line Summary

Production Agents are narrow, deep, purpose-built systems; the game is won on tools, failure handling, and observability.

Paragraph Summary

The article addresses the overuse and semantic dilution of the term "AI Agent," offering a restrained definition: an Agent is a system with a goal rather than merely instructions—one that decides its own next steps, handles failures, and judges completion—while everything else is just a function call in packaging. Drawing on frontline delivery experience, the author notes that production Agents are overwhelmingly narrow, deep, purpose-built pipelines such as ticket triage, document extraction, and code review, and that the difference between teams lies not in model selection but in three things: tool design, failure handling, and observability; swapping in a stronger model without changing the architecture won't fix flaws. The framework wars are a distraction—what matters are patterns that hold across frameworks: plan-then-execute, separating retrieval from reasoning, and structured explicit handoffs. RAG's hidden problem lies in chunk boundaries and metadata; when results are technically correct but contextually useless, check chunking first rather than swapping embedding models. Ultimately, what's truly scarce are system-design-minded engineers who can build maintainable, trustworthy AI systems.