How Postman Runs Agent Mode on Amazon Bedrock for 40 Million Developers

From Demo to Production: The Real Challenges of Scaling AI Agents

Building an AI agent that works in a demo and running one reliably for 40 million developers are two entirely different engineering problems. Postman and AWS jointly shared the architectural design behind Agent Mode, with core lessons centered on three key themes: controlling tool sprawl, schema-based read-only access, and treating context as the true performance bottleneck.

What makes this writeup valuable is that it steps away from the typical "AI agent showcase" narrative and directly confronts the toughest engineering trade-offs in large-scale production environments. For any team building enterprise-grade agent applications, these patterns offer immediately applicable guidance.

Controlling Tool Sprawl: More Tools Doesn't Mean a Better Agent

A common intuition is that giving an agent access to more tools makes it more powerful. In real large-scale deployments, however, unchecked tool proliferation is one of the primary sources of agent reliability degradation.

When an agent faces dozens or even hundreds of callable tools, the model must make the right selection from an enormous candidate set — significantly increasing the probability of incorrect calls, hallucinated invocations, and decision latency. Postman's approach is to actively control "tool sprawl" by carefully defining the boundaries and granularity of tools, ensuring the agent only faces a small, curated toolset in any given context.

This reflects an important engineering philosophy: an agent's reliability often depends not on how much it can do, but on how controllable its decision space is at each decision point.

The technical root causes of tool sprawl are worth understanding in depth. When an LLM calls tools, it requires the names, descriptions, and parameter schemas of all available tools to be placed in the context, from which the model selects the most appropriate one during inference. As the number of tools grows from 10 to 100, the tool descriptions alone consume a significant portion of the token budget. Meanwhile, the "tool selection" task transforms from a simple multiple-choice question into a complex retrieval problem, dramatically increasing the likelihood of confusion between semantically similar tools. Mitigation strategies used by some teams include: tool grouping and dynamic loading (injecting only a relevant subset based on current task intent), tool aggregation (combining multiple fine-grained operations into a single higher-level semantic tool), and vector retrieval-based tool recall. Postman's approach represents a more fundamental design philosophy: preventing tool proliferation through boundary definition at the architecture stage, rather than attempting to govern it after the sprawl has already occurred.

Schema-Based Read Access: Trading Structural Constraints for Predictability

Another pattern Postman emphasizes is "exposing read-only access based on schemas." Rather than letting an agent freely explore and assemble requests, explicit schema definitions constrain read operations within predictable structures.

This approach carries several benefits. The schema itself provides the model with clear semantic cues, enabling more accurate understanding of what can be read and in what format it will be returned. Read-only operations naturally reduce the risk surface of agent behavior, preventing destructive side effects during exploration. Structured inputs and outputs also make the entire pipeline easier to monitor, debug, and replay.

For a platform serving tens of millions of developers, predictability is nearly synonymous with operability. Schema-based design transforms an agent from a "black-box explorer" into a component operating under a well-defined contract.

The schema here typically refers to an OpenAPI Schema or JSON Schema, which describes in a machine-readable format what inputs an API endpoint accepts and what structure it returns. In an agent architecture, injecting the schema directly into the context allows the model to accurately know field names, data types, and valid value ranges — without needing to "guess" interface behavior through trial and error. The read-only constraint corresponds to HTTP GET semantics: compared to write operations like POST/PUT/DELETE, it is idempotent with respect to side effects — repeated model calls will not produce data modifications or state changes. This design is especially valuable in scenarios involving multi-round retries and debug replay. Restricting an agent's data access layer to schema-defined read-only interfaces essentially introduces a "contract" between the agent and the underlying data system, converting the model's free exploration into auditable, versionable API calls.

Context Is the Real Bottleneck

The most insightful point in the entire presentation is treating context — not model capability — as the true bottleneck.

In real agent systems, the model's reasoning ability is often not the primary constraint. What actually degrades performance is the challenge of providing the model with precisely the right information within a limited context window — neither missing critical evidence nor being overwhelmed by volumes of irrelevant content. Context management directly determines an agent's accuracy, cost, and response speed.

This forms a closed loop with the previous two points: controlling tool sprawl reduces the tool descriptions that need to be packed into the context; schema-based read access makes the information entering the context more compact and structured. All three fundamentally serve the "context budget." When teams treat context as a scarce resource to be carefully managed, the engineering quality of an agent truly becomes controllable.

The context window is the maximum number of tokens a large language model can process in a single inference call. Although recent frontier models have extended this limit from 4K to 128K or even million-token scales, context management remains an engineering bottleneck for multiple reasons. On cost: token count is directly linearly correlated with inference fees, and redundant context can create significant additional overhead in high-concurrency scenarios. On latency: longer contexts translate to higher time-to-first-token (TTFT). On accuracy: research shows that models exhibit attention degradation toward information in the "middle of long contexts" (the "Lost in the Middle" problem), where large volumes of irrelevant content actually dilute the weight of critical evidence. As a result, context management has gradually evolved into its own engineering discipline, encompassing techniques such as summarization compression, sliding windows, and memory externalization (storing historical state in vector databases for on-demand retrieval).

Amazon Bedrock as the Scalability Foundation

To support the call volume of 40 million developers, the underlying model hosting and inference infrastructure is equally critical. Postman chose to run Agent Mode on Amazon Bedrock, leveraging its managed large model service capabilities to address the challenges of large-scale deployment.

As AWS's managed generative AI service, Bedrock provides multi-model access, elastic scaling, and enterprise-grade security and compliance. For a platform like Postman that needs to remain stable under high concurrency, using managed infrastructure allows the team to focus on agent architecture design and business logic, without bearing the operational burden of running large-model inference in-house.

Takeaways for Agent Developers

Several universal lessons can be distilled from Postman's experience. The engineering focus for agents is shifting from "making it work" to "making it stable and controllable at scale." Tool design, schema constraints, and context management may seem like separate concerns, but they all point toward the same objective: reducing uncertainty at every decision point in an agent's execution.

For teams planning to take agents into production, rather than chasing feature completeness, it pays to first establish rigorous governance over the toolset, data access, and context. This may not be as eye-catching as a flashy demo, but these straightforward engineering principles are precisely what determines whether an agent can serve real-world users at scale.