Adjudicated Query Pattern: Batch Lease Compliance Verification with Amazon Quick

Why Compliance Q&A Demands "Provably Complete" Answers

In highly regulated domains like finance, real estate, and legal services, AI Q&A systems face a fundamental challenge: answers must not only be "probably correct" — they must be "provably complete and defensible." Traditional large language models generate answers directly, and even when accuracy is high, it's difficult to produce conclusive evidence during audits, litigation, or regulatory review.

When you need to scan thousands of lease agreements for compliance, this problem becomes impossible to ignore. Missing a single non-compliant clause can translate into real legal and financial risk. The Adjudicated Query pattern recently proposed by AWS is designed precisely to address this pain point.

The Core Architecture of the Adjudicated Query Pattern

The key to this pattern lies in the coordination of three components: the Amazon Quick chat agent, a bounded MCP server, and the underlying deterministic rules engine.

Three-Layer Separation of Concerns

As described in the official documentation, this architecture completely decouples "understanding intent" from "making judgments":

• The Amazon Quick chat agent handles conversational interaction — understanding compliance questions posed in natural language and presenting results in a readable format.
• The bounded MCP server acts as an intermediary layer, constraining the capabilities the agent can invoke and preventing the model from "freelancing" outside defined rules.
• The deterministic rules engine is where actual judgments are made. It evaluates data against predefined rules line by line, producing results that are reproducible and fully traceable.

The immediate benefit of this separation is that the "judgment logic" behind an answer is not determined by a probabilistic model — it's handled by deterministic rules. The model is only responsible for translating questions into queries the rules engine can process, and translating results back into plain language.

MCP (Model Context Protocol) is an open protocol proposed by Anthropic in late 2024, designed to standardize how large language models interact with external tools and data sources. Its core idea is to define a structured "capability boundary" for models — specifying which functions they can call and which resources they can access — replacing the fragmented, vendor-specific tool-calling implementations that existed before. In this architecture, the "bounded MCP server" refers to a deliberately trimmed MCP implementation that only exposes interfaces related to the rules engine, blocking any path through which the model might bypass rules via free-form tool calls. This "bounded" design is the engineering foundation of the entire defensibility guarantee: even if a model wanted to "freelance," there are simply no corresponding tools available at the protocol level.

Why This Design Is "Defensible"

The problem with traditional RAG or pure LLM approaches is that it's hard to explain to an auditor why the system deemed a contract compliant. The Adjudicated Query pattern, by routing through a deterministic rules engine, makes every conclusion traceable to a specific rule entry and data input.

Two key terms emphasized in the original text are worth examining closely — provably complete and defensible:

• Provably complete means the system can traverse all relevant clauses, rather than stopping after "retrieving the most similar few" like vector search does. For compliance scanning, omission equals risk — completeness is non-negotiable.
• Defensible means every answer is backed by a deterministic reasoning chain that can be presented as evidence in regulatory or legal contexts.

This is precisely the point of wrapping the rules engine in a "bounded MCP": it demotes the LLM from decision-maker to translator, preserving the convenience of natural language interaction while handing accuracy and completeness to a controllable engineering system.

Traditional RAG (Retrieval-Augmented Generation) relies on vector similarity to retrieve relevant document fragments, then has the LLM synthesize an answer. This pipeline has two inherent weaknesses. First, vector retrieval ranks by similarity and then truncates — it's fundamentally a best-effort approximation that cannot provide a definitive answer to completeness questions like "did we cover all applicable clauses?" Second, the LLM's final reasoning is probabilistic sampling; the same input can produce outputs that differ slightly in wording or even logic across runs, making them difficult to use as reproducible evidence in audit contexts. The Adjudicated Query pattern sidesteps both problems by externalizing judgment logic to a deterministic rules engine: the engine always produces the same output for the same input, and its execution path can be fully logged and reviewed.

Real-World Application: Lease Compliance

The original article uses lease compliance as its running example. Imagine an enterprise holding thousands of lease agreements that need to be periodically checked against a new regulation or internal policy — manual review is costly and error-prone.

With this pattern, compliance staff can ask questions directly in natural language — for example, "which leases fail to satisfy this new regulatory requirement?" — and the system uses the rules engine to batch-scan all contracts and return a complete, traceable list of violations. The key is the ability to "sweep thousands" of agreements, giving this approach real value at scale.

A Deployable Reference Implementation

Notably, AWS doesn't stop at the conceptual level — it provides deployable sample code based on AWS CDK (Cloud Development Kit). This means developers can get a reference architecture directly and quickly stand up a prototype in their own environment without designing the entire system from scratch.

Delivered as infrastructure as code, CDK lowers the barrier to replication and makes the pattern easier to evaluate and customize. For teams looking to validate a "deterministic + LLM" hybrid architecture, this is a relatively low-cost starting point.

AWS CDK (Cloud Development Kit) is AWS's infrastructure-as-code framework that lets you define cloud resources using mainstream programming languages like TypeScript, Python, and Java. The CDK toolchain synthesizes these definitions into CloudFormation templates and handles deployment. Compared to writing CloudFormation YAML directly, CDK's advantage is that you can use real programming languages to express conditions, loops, and abstractions, making complex architectures far more intuitive to reproduce and customize. For teams evaluating the Adjudicated Query pattern, the CDK sample means you can spin up a complete reference environment in your own AWS account with just a few commands — including Lambda functions, API Gateway, the rules engine service, and all supporting components — without manually configuring each piece.

The Boundaries of This Pattern

The Adjudicated Query pattern is not a universal solution. It works best in scenarios where rules are well-defined, completeness must be guaranteed, and results need to be defensible — compliance reviews, policy checks, contract screening, and similar use cases. For questions where rules are ambiguous, subjective judgment is required, or the goal is exploratory, a deterministic rules engine becomes a constraint rather than an asset.

From another angle, this pattern reflects an important trend in enterprise AI adoption: in high-stakes scenarios, organizations are no longer trying to have LLMs independently complete end-to-end judgments. Instead, they're embedding LLMs within engineering systems with clear boundaries and audit trails. The MCP protocol plays the role of a "safety valve" here, constraining the model's operational scope so that AI remains both useful and controllable.

For teams exploring AI deployment in regulation-intensive industries, this combination of "chat agent + bounded MCP + deterministic rules engine" offers a compelling engineering paradigm worth adopting.