Harness Engineering in Practice: Building Scalable Infrastructure for Production-Grade Agents

At the AI Engineer conference, Mike Chambers, Senior AI Specialist Developer Advocate at AWS, delivered a talk on "Harness Engineering." Rather than staying at the conceptual level, he walked through a live coding demo showing how to take a prototype agent running on a laptop and progressively upgrade it into a production-ready system deployable at scale in the cloud.

Two Fundamentally Different Types of Agents

Mike opened with a distinction worth thinking carefully about: the "agents" people talk about actually fall into two categories with completely different design philosophies.

The first type is agents we use — coding assistants like Claude Code, Cursor, and Kiro, along with various productivity tools. With these, it doesn't matter how aggressively you consume tokens or how you push them, because you're the one using them.

The second type is agents we build. This is the core of the talk. When you're deploying an agent to thousands of users, your entire mindset has to shift — you need to design carefully for your end audience, ensuring stable, secure, and scalable operation.

Interestingly, these two types are often chained together: the agent you build may be the very agent someone else uses.

What Is a Harness

Mike used a blunt analogy to define harness. In the dictionary, a harness is "a set of straps and fittings used to control an animal" — swap "animal" for "model" and the definition fits surprisingly well.

In more engineering terms: take the model out of an agent, and everything left is the harness. LangChain, martinfowler.com (including Brigitta's piece Harness Engineering for Coding Agents), and others have published on this topic, suggesting the concept is gaining traction across the industry.

For agents we use, the harness is relatively simple: a coding assistant runs on your local machine, has file access, and is configured with memory management, skills, tools, and MCP servers. Mature engineering teams have had coding standards for decades — those now evolve into "harness specifications" deployed uniformly across everyone's coding assistants.

Mike also recommended AWS's Agent Toolkit (open source and free on GitHub), whose core purpose is to eliminate what he calls "slop ops." Just as professional cloud development has long pushed back against "click ops" (deploying by clicking around in the console), you shouldn't let an agent directly spin up an S3 bucket or launch an EC2 instance. Instead, have it generate Infrastructure as Code (IaC) so you retain real control over cloud deployments.

MCP (Model Context Protocol) is an open standard released by Anthropic in late 2024, designed to unify how AI models interact with external tools and data sources. Think of it as the "USB port" for AI — developers package tools according to the MCP spec, and any agent supporting the protocol can plug in and use those tools without custom integration code for each agent. MCP servers can expose capabilities like filesystem access, database queries, and API calls, dramatically lowering the cost of onboarding tools into the harness layer. This also explains why "a system prompt plus a few MCP tools" can cover a large share of agent use cases — the flourishing MCP ecosystem puts ready-made tools within easy reach.

From Local Prototype to Cloud-Scale

Mike's live demo progressed through several stages.

The Simplest Strands Agent

The first example was a minimal agent built with the Strands Agent SDK: import the agent and tool decorators, pass in a system prompt, attach two tools — a calculator and a time fetcher. The loop is managed automatically by the framework.

Is this a harness? Barely. It's extremely thin. It runs on a laptop, has no scale, and lacks the critical properties a production-grade agent needs.

Adding Session and Memory Management

The second example, generated by Kiro, focused on introducing a session manager. It maintains session state across multiple calls — a form of short-to-medium-term memory — while also persisting long-term memory to a local file.

The agent defines a remember tool that can actively decide to retain information about the user. In the demo, when Mike asked "who will win the World Cup," the agent drew on previous conversation memory — knowing he lives in Australia and hopes Australia wins — which is exactly session memory in action. Of course, the agent honestly admitted it has no ability to predict the outcome, since the answer ultimately comes from an LLM.

Cloud-Scale Deployment with Agent Core

The real challenge: how do you deploy an agent like this at cloud scale, with memory, the agentic loop, and other components each scaling independently? Cramming everything into one container and scaling it horizontally won't support tens of thousands of users.

This is where harness engineering gets serious. Mike enumerated the components that need independent consideration in a production agent: loop management, scaling, payments, memory, identity, skills, runtime, context management — and what he said "should have been first but saved for last" — observability and evaluations.

Mike used AWS Bedrock Agent Core to address these concerns. Through the agent core CLI, developers can configure and deploy an agent step by step:

• Language: Python or TypeScript
• Interface: HTTP, MCP service, or AG-UI (for building interactive chat agents)
• Framework: any framework — Mike used the Strands Agents SDK
• Model: any model, not limited to Amazon Bedrock; the demo defaulted to Sonnet 4.5
• Memory: opt in to deploy long-term and short-term memory; the system creates dedicated cloud infrastructure, asynchronously managing that memory and keeping it connected to the agent

The resulting production agent code adds little on top of the local version — just the integration with the Bedrock Agent Core App — but that single addition enables multi-tenant isolation and runtime scaling. In other words, you write agent logic for a single user, and horizontal scaling works without hand-rolling multi-tenant code, which dramatically simplifies security and identity management.

Local Debugging and One-Command Deployment

Running agent core dev spins up a web UI connected to the locally running agent, letting you see code changes in real time and switch between the local and cloud-deployed versions. agent core deploy uses IaC to push the agent — along with its memory, runtime, and other components — to the cloud. Afterward, you can inspect trace logs, memory stores, and other data through the UI for debugging.

Multi-tenant isolation is one of the core security requirements of any SaaS system. It means that when serving multiple users on shared infrastructure, each user's data, sessions, and compute resources are completely isolated — invisible to and unaffected by one another. For agents, this means User A's conversation memory cannot be read by User B, and a tool call triggered by one user cannot affect another user's state. The traditional approach requires developers to manually maintain tenant IDs in code and design isolated data storage schemes — significant engineering effort that's also prone to security gaps. Bedrock Agent Core bakes this mechanism into the platform layer, so developers only need to write agent logic for a single user; the platform handles tenant isolation automatically. This is a major engineering advantage over self-built solutions.

IaC (Infrastructure as Code) is a core practice in modern cloud engineering — using code (such as Terraform, AWS CDK, or CloudFormation) to declaratively describe cloud resource configurations rather than creating them through the console by hand. Its key benefits are repeatability, auditability, and version control: every infrastructure change lives in code, can be governed through code review and CI/CD pipelines, and can be rolled back quickly when something goes wrong. Mike's repeated emphasis on "fighting slop ops" reflects exactly this philosophy — letting an agent directly call APIs to create cloud resources bypasses all governance processes and produces "ghost resources" that are hard to track. Agent Core's deploy command expresses the entire deployment of an agent and its dependencies as IaC, keeping the lifecycle management of cloud agents consistent with traditional software engineering practices.

80% of Agents May Need No Code at All

Mike made a striking claim: since deploying a functional agent requires little more than a system prompt and a few MCP tools, perhaps 80% of agent use cases and 80% of the development work have already been solved.

To that end, Agent Core has harness capabilities built in. Developers simply provide a small JSON configuration — specifying which model to use and what system prompt to apply — and can deploy directly, without writing any agent code at all.

Even more powerful is Agent Core's composability: all capabilities (memory, runtime, identity, etc.) are independent modules that can be mixed, matched, or used in isolation. If you already have a stable production agent and just want serverless long-term memory management, you can integrate only that piece.

Closing Thoughts

The value of this talk lies in making "agent engineering" concrete and actionable. From distinguishing agents we use from agents we build, to using the harness concept to unify everything in an agent that isn't the model, to achieving independent component-level scaling through Agent Core — Mike laid out a clear path from prototype to production. For developers thinking seriously about deploying AI agents into scaled production environments, these ideas and tools are well worth a deeper look.

If you're interested, check out the open-source Strands Agents framework (model-first, fast, and powerful) and Amazon Bedrock Agent Core.