Beyond Prompt Engineering: Building Intent-Aware AI Agents

Most deployed AI agents fail silently. A user makes a request, the agent retrieves irrelevant context, and then returns an answer that sounds reasonable but is actually wrong. The difference between an agent that truly delivers value and a shell that merely wraps an LLM lies in intent recognition and context management—not prompt quality. Anthropic, OpenAI, and dozens of startups are specifically hiring "AI agent engineers" precisely because basic prompt engineering doesn't scale. You need a systematic approach to understand what users really want, retrieve the right context, and act decisively.

Intent Classification Must Happen Before the LLM Call

Intent recognition shouldn't be left to the LLM on every call. Every LLM call consumes tokens, adds latency, and incurs cost. A more sensible approach is to build a lightweight classifier that sits upstream.

Take a customer service agent as an example. A user writes: "My order hasn't arrived, and I need it by Friday." The surface request is a logistics inquiry, but the real intent is actually urgency resolution. A naive agent would retrieve order history and stop there. A context-aware agent, however, recognizes that the intent is deadline-driven, and then surfaces expedited shipping options or refund eligibility.

from enum import Enum

class SupportIntent(Enum):
    ORDER_STATUS = "order_status"
    URGENT_DELIVERY = "urgent_delivery"
    REFUND_REQUEST = "refund_request"
    PRODUCT_INQUIRY = "product_inquiry"

def classify_intent(user_message: str) -> tuple[SupportIntent, float]:
    # Use a small fine-tuned model or rule-based classifier
    # Runs locally and fast, instead of calling GPT-4
    keywords = {
        "urgent|asap|deadline|friday|today": SupportIntent.URGENT_DELIVERY,
        "where|tracking|arrived|status": SupportIntent.ORDER_STATUS,
        "refund|return|money back": SupportIntent.REFUND_REQUEST,
    }
    for pattern, intent in keywords.items():
        if re.search(pattern, user_message.lower()):
            return intent, 0.85
    return SupportIntent.PRODUCT_INQUIRY, 0.6

This classifier can use regex patterns, a lightweight sklearn model, or even a smaller quantized LLM. The key point: run it before the expensive LLM call, then route the user message and detected intent to the context retrieval module.

Actionable takeaway: Add the intent classification step as middleware. Log intent predictions against actual user outcomes to measure accuracy. Retrain when precision falls below 85%.

Quantized Model refers to the technique of compressing a deep learning model's original FP32/FP16 floating-point weights into lower-bit representations such as INT8 or even INT4, thereby shrinking the model size several times over and reducing inference latency by a similar factor. The "50MB quantized model" mentioned in the article refers precisely to this class of small locally-runnable models, with typical examples being the INT4 versions of Qwen, Phi, and Gemma. Quantization typically introduces a 1%–3% accuracy loss, but for a discriminative task like intent classification with a limited label space, it's almost imperceptible. Compared to calling a cloud-based large model like GPT-4, a quantized small model can run entirely offline, complete a single inference in milliseconds, and requires no per-token payment—making it ideal for serving as an upstream "gatekeeper." Note that quantization is divided into post-training quantization (PTQ) and quantization-aware training (QAT); production environments generally prioritize PTQ for fast deployment, and only fall back to QAT when accuracy falls short.

Intent-Based Hierarchical Context Retrieval

Once the intent is clear, context can be retrieved in a targeted way. Different intents require different data sources, different time windows, and different access patterns.

A financial advisor agent facing "What should I do with this $50,000?" needs portfolio history, market conditions, and risk profile. Whereas another query, "Has my deposit arrived?", needs the transaction log from the past 48 hours. The two access different databases with different query patterns and latency requirements.

Structure context retrieval into intent-specific pipelines:

Intent: urgent_delivery
├─ Primary: Order status (DynamoDB, <100ms)
├─ Secondary: Shipping partner API (network call, <500ms)
├─ Tertiary: Customer tier/VIP status (cache, <50ms)
└─ Fallback: Previous support tickets (search index)

Intent: refund_request
├─ Primary: Transaction record (database)
├─ Secondary: Refund policy version (config store)
├─ Tertiary: Similar refund decisions (vector search on previous cases)
└─ Fallback: Manager escalation template

Each pipeline has timeouts and degradation behavior set up. If DynamoDB responds slowly, it falls through to the secondary data source. If all data sources fail, the agent should state so explicitly rather than hallucinate. Stripe's API error handling follows this pattern: primary endpoint, backup region, timeout cutoff, graceful degradation. Apply the same rigor to your agent's context retrieval.

Actionable takeaway: Map your top 5 intents to their respective required context sources. Set SLA targets for each pipeline (p99 latency, cache hit rate). Monitor and measure weekly.

Vector Search is the technique of encoding unstructured data such as text and images into high-dimensional vectors (embeddings), then using approximate nearest neighbor (ANN) algorithms to retrieve semantically similar items. Common implementations include FAISS, Milvus, Pinecone, and pgvector. In the scenario described in the article—"Similar refund decisions (vector search on previous cases)"—raw tickets are converted into vectors by an embedding model (such as text-embedding-3 or bge-m3) and written into the index. When a new refund request arrives, it is encoded with the same model, and the Top-K most similar historical cases along with their approval conclusions are taken as reference. Compared to keyword inverted indexes, vector search can capture expressions that are literally different but semantically identical, such as "my mouse is broken" and "the cursor won't move." The trade-off is that it requires an additional embedding service and index maintenance costs, which is why the article places it at the Tertiary layer—only falling back to it when the primary path data sources are insufficient, in order to control latency and cost.

State Management and Memory Windows

Context-aware agents need memory. Not hallucinated memory, but actual structured state persisted across turns. When a user says "Create an invoice for that," "that" refers to something mentioned three turns earlier.

Most prompt-based agents handle this by dumping the full conversation history into context. That's wasteful and introduces noise. A better approach is to maintain explicit state:

class AgentState:
    user_id: str
    current_intent: SupportIntent
    conversation_id: str
    extracted_entities: dict  # {"amount": 50000, "product": "laptop"}
    context_retrieved: dict   # what was fetched and when
    decision_history: list    # previous actions and outcomes
    timestamp: float

def update_state(state: AgentState, llm_response: str, user_input: str):
    # Extract entities from the response (structured output, not free text)
    entities = extract_entities(llm_response)
    state.extracted_entities.update(entities)
    
    # Record decisions for auditing
    state.decision_history.append({
        "action": llm_response,
        "timestamp": time.time(),
        "confidence": get_confidence_score(llm_response)
    })

When the next turn needs context, query this state object rather than the full conversation. Pass only relevant prior decisions, not chat noise. GitHub Copilot's code generation works this way: it doesn't analyze every line of the codebase, but rather the specific files open in the editor and the git change history of those files.

Actionable takeaway: Store agent state in a timestamped, queryable format (JSON in a key-value store is fine). State expires after 24 hours unless explicitly refreshed. Version your state schema.

Entity Extraction is the task of identifying structured fields (person names, amounts, order numbers, product names, etc.) from unstructured text, and is a core subproblem of information extraction. The reason extract_entities in the article emphasizes "structured output, not free text" is that directly parsing an LLM's natural language response to update state is extremely fragile—a slight change in wording or an extra comma from the model can cause downstream parsing to fail. In engineering practice, three approaches are typically used to obtain stable structure: first, have the LLM return in JSON Schema or function call (tool call) format; second, use a dedicated extraction model (such as spaCy NER or GLiNER); third, use regex as a fallback for high-frequency fields. Once extraction results are written into state.extracted_entities, subsequent turns can directly read fields like amount and order_id without stuffing the entire conversation back into context, thereby significantly reducing token consumption and improving auditability.

Error Recovery and Explicit Degradation

Context-aware agents either fail gracefully or fail loudly. They don't fail silently.

When an agent cannot retrieve the necessary context, it should state so explicitly. When confidence is below threshold, it should escalate. When ambiguity exists, it should ask clarifying questions rather than guess.

def should_proceed_with_action(state: AgentState, confidence: float) -> bool:
    MIN_CONFIDENCE = 0.75
    REQUIRED_CONTEXT = ["customer_id", "order_id"]
    
    if confidence < MIN_CONFIDENCE:
        return False  # Request clarification
    missing = [key for key in REQUIRED_CONTEXT if key not in state.extracted_entities]
    if missing:
        return False  # Fetch missing context or ask the user
    return True

# In the agent loop:
if should_proceed_with_action(state, confidence):
    execute_action(state)
else:
    agent_response = generate_clarification_prompt(state, missing_context)

This prevents misjudgments. It's slower than always executing, but a hundred times better for trust.

Actionable takeaway: Define minimum confidence thresholds for each intent type. Log every degradation decision. Set a target: the proportion of user interactions requiring clarification should be below 5%.

Start This Week by Acting on Intent Classification

You don't need to rebuild your entire agent architecture. Start by adding one layer: intent classification before the LLM call. Pick your top 3 use cases, define 5–7 intents for each, and then build a classifier using regex or a 50MB quantized model. Measure its accuracy. Once it's stable, route to different context sources by intent.

This one change—routing context by intent instead of treating all queries the same—is the watershed between agents users barely tolerate and agents that actually work.

One-line Summary

Placing intent classification before the LLM call, retrieving context hierarchically by intent, and managing state explicitly is the key to agent success.

Paragraph Summary

The article points out that most deployed AI agents fail silently, and the root cause lies not in prompt quality but in intent recognition and context management. The author advocates placing intent classification as lightweight middleware before expensive LLM calls, using regex or quantized small models to reduce cost and latency; then routing to different hierarchical context retrieval pipelines based on detected intent, with timeouts and degradation strategies set for primary, secondary, and fallback data sources. At the same time, replace passing the full conversation history with a structured state object that retains only entities, retrieval records, and decision history. Finally, emphasize confidence thresholds and explicit degradation mechanisms—prefer clarification over guessing. The author recommends starting with a single intent classification layer and progressively building a truly context-aware architecture.