AI Not Working? Learn to Diagnose Which Layer Is the Problem — Loop, Harness, or Context

Three "Engineerings" — Three Layers of the Same Idea

Recent discussions around AI agents have surfaced three concepts that all share the word "engineering": Loop Engineering, Harness Engineering, and Context Engineering. Many people's first reaction is — are these three separate things, or just three ways of saying the same thing?

The answer: they are three layers expanding on the same core idea. That idea is — "the model isn't the product; the system you build around the model is."

Stack these three layers together and you get a map that narrows from the outside in. The real value of understanding this map isn't memorizing the terminology — it's that when your AI "isn't working," you can quickly pinpoint: which layer is broken, and where to go to fix it.

The Loop Layer: Managing Rhythm, Letting AI Run Itself

The core idea behind Loop Engineering comes from Andy Osmedie: stop prompting AI yourself one message at a time — design a system that prompts it for you. A loop is a recursive goal: define the objective, let the AI relay it forward, and keep going until it's achieved.

Boris Cherney, the lead on Claude Code, put it even more bluntly: "I don't prompt Claude myself anymore. My job is to write loops." Building a loop requires five components: automated workflows, skills, connectors, sub-agents, and memory that persists to disk.

Loop Engineering is rooted in the core design pattern of AI agent architecture — the ReAct (Reasoning + Acting) loop. Proposed by Princeton University in 2022, this architecture has the model reason at each step, execute an action, observe the result, and continue reasoning — forming a self-driving iterative cycle. Modern agent frameworks like LangGraph, AutoGen, and CrewAI build on this foundation to enable multi-agent collaboration and subtask delegation. "Memory persisted to disk" corresponds to the Long-term Memory layer in agent memory architecture, as distinct from "Working Memory" that only exists within a single conversation. A code repository, as external persistent storage, naturally supports version control and change tracking — compensating for the fundamental stateless nature of large language models. This is the technical foundation that keeps the loop layer running continuously.

Why does memory need to live on disk, in a code repository? Because AI forgets, but code repositories don't. This is the key to why the loop layer can keep running — it needs an external anchor that doesn't depend on the model's short-term memory.

Using a factory analogy, the loop is the assembly line: it manages rhythm — when work starts, how tasks are scheduled, and who verifies completion.

The Harness Layer: Managing the Environment — Configuration Often Matters More Than the Model

Harness Engineering also comes from Andy Osmedie's definition: an agent = model + everything you build around it (prompts, tools, hooks, sandboxes).

The most important conclusion at this layer: a mediocre model with a great harness can beat a top-tier model with a poor harness. This isn't just talk — it's backed by data. On Terminal Bench, one model climbed from outside the top thirty into the top five with zero changes to the model itself, just by improving the harness.

Terminal Bench is a benchmark specifically designed to evaluate AI agents' ability to complete engineering tasks in terminal environments, covering real-world scenarios like file operations, code debugging, and system configuration. Its results reflect the actual engineering capability of a "model + harness" combination, rather than isolated model reasoning ability. The core components of the harness layer include: Prompt Templates, Tool Use / Function Calling, Hooks (e.g., pre/post-execution validation logic), Sandbox environments (e.g., containerized isolated execution), and the recently prominent MCP (Model Context Protocol, an open tool-interoperability standard led by Anthropic). Together, these components form the "adapter layer" between the model and the real world — determining how much of the model's capability actually gets released. This is the fundamental reason why tools built on the same underlying model can perform so differently.

So the key conclusion at the harness layer is: more often than not, it's not a model problem — it's a configuration problem. In factory terms, the harness is the workstation: it manages the environment — how tools are arranged, where the guardrails are, and how to recover when things go wrong. A poorly set-up workstation makes even the smartest assembly line useless.

The Context Layer: Managing Attention — Not Filling the Window, but Curating It

Context Engineering comes from Anthropic's own framing: it is the highest-leverage single point within the harness layer. It governs — at each reasoning step, which information actually enters the model's limited attention window.

The theoretical foundation of Context Engineering lies in the Self-Attention mechanism of the Transformer architecture. Although the context windows of modern large language models have expanded from an initial 4K tokens to 200K or even 1 million tokens, research consistently shows that the "Lost in the Middle" phenomenon persists — models process information at the beginning and end of the context more accurately, while tending to overlook or confuse information in the middle. This means window size and usable information don't scale linearly: simply packing in more content doesn't improve reasoning quality — it dilutes the signal-to-noise ratio. Curation corresponds in engineering terms to RAG (Retrieval-Augmented Generation), dynamic context compression, memory summarization, and similar techniques — all aimed at maximizing information density within a limited Attention Budget. Tokens aren't just a measure of length; they're the fundamental unit of both reasoning cost and quality.

The key insight here: no matter how large the window, what you can stuff in and what you should stuff in are two different things. One-line summary — not filling, but curating.

In factory terms, context is the blueprint in the worker's hands right now: it manages what they're actually looking at in this moment. Even the best workstation can't save you if the blueprint is full of noise.

The Three-Layer Map: Match Your Situation, Pinpoint the Problem

Map the tools you use every day to this framework and the picture becomes clearer:

• Context layer: The message you type to the AI, the document you paste in. Everyone operates at this layer — most people just don't realize they're doing curation.
• Harness layer: The tools you've installed — Claude Code, Cursor — along with their rule files, MCPs, and hooks. The underlying model is sometimes the same; the difference in experience comes entirely from this layer.
• Loop layer: The system that wakes up on its own, runs tasks, schedules work, and writes its own logs. Most people haven't reached this layer yet — and that's completely fine.

When AI Isn't Working, Each Layer Has a Distinct Symptom

The most practical use of this map is helping you identify which layer is broken before you start fixing things:

• The same mistake keeps happening — you keep stepping in the same hole → Harness layer problem. What's missing is a mechanism. Turn every failure into a permanent rule instead of just hitting "retry."
• More context makes it more confused — longer conversations get dumber → Context layer problem, also known as Context Rot. This describes how, as conversation turns accumulate, irrelevant information piles up and low-value content consumes the attention budget, causing reasoning quality to continuously degrade. The fix is subtraction, not addition.
• Everything needs you to watch over it — the moment you step away, it stops → The loop layer was never built.
• It runs happily on its own but can't tell you whether the output is correct → Also a loop layer problem. Remember this: "Done" is a declaration, not a proof.

After Models Get Stronger, Which Layer Disappears?

A worthwhile question: as models keep improving, which of these three layers goes away? The answer — none of them — but in very different ways.

• The harness is fluid: each component encodes an assumption about "something the model can't do yet." As models improve, old components get retired — but new requirements spawn new ones. It's always swapping parts, but it never disappears.
• The context layer is the hardest to eliminate: the limitations of the attention window are an architectural issue inherent to Transformers — bigger windows still lead to "more means messier." Curation is a permanent craft. Even if a breakthrough memory architecture emerges in the future, the engineering question of "what information to pass to the model and how to organize it" doesn't disappear — it just migrates to a new form.
• The loop layer's most enduring element is verification: verification always stays with you. The more automated the tooling, the more valuable your judgment becomes.

Getting Started: Build the Foundation from the Inside Out

If you want to systematically get into AI agents, the recommendation is to work in reverse — from the inner layer outward:

1. Practice context first: Even with just a web chat interface, learning to "pack less and pack right" produces immediate results.
2. Then build the harness: Create a rule file and accumulate every mistake you've made into it.
3. Finally, build the loop: Get the whole system running on its own.

Don't chase full automation from day one — the foundation is built outward from yourself. After reading all three layers, the biggest takeaway is still the same sentence: the model isn't the product; the system you build around it is. And ultimately, that system comes back to you.

Key Takeaways