AML Agent in Practice: Optimizing Memory Is More Effective Than Swapping LLMs

In finance, anti-money laundering (AML) has long been a labor-intensive challenge with complex rules. When the LLM boom arrived, many teams' first instinct was to "switch to a bigger model." But in a recent tech talk, AI developer advocate Nacho Martinez offered a different answer: an agent's performance bottleneck is often not the model itself, but the design of the memory layer and context engineering. Based on that talk and its accompanying workshop, this article breaks down how to build a self-running AML analysis system using agent architecture.

Why Financial Scenarios Need an "Agent System" Rather Than a Chatbot

Nacho divides AI applications into four forms: LLM chatbots, RAG applications, LLM-driven workflows, and AI agents. What truly creates value in production is the combination of the latter two—automation plus autonomy.

He uses a mid-sized bank as an example: when a $1 million deposit arrives and $50,000 is withdrawn in batches via ATMs within three days, such suspicious transactions should be flagged automatically rather than requiring analysts to check each one manually. A chatbot can't do this because it lacks the ability to run continuously, call tools, and maintain memory. An agent system, by contrast, can operate 24/7 in the background, automatically generate suspicious activity reports (SARs), and hand them to humans for review.

The key insight here is that the reasoning core is provided by frontier models, over which developers have limited control; memory, tool use, and perception (the scaffolding) are the parts that can be deeply customized. Nacho emphasizes that with the same frontier model, simply changing the scaffolding configuration can make performance on a specific task significantly better or worse.

The Seven Pieces of Agent Architecture: The Memory Layer Is the Main Battleground

A complete agent framework consists of seven to eight components: model, storage, retrieval components, memory layer, implicit context, agent loop, and context engineering. Nacho focuses on the data layer because it is what he calls "the part we can actually operate on."

The Ceiling of Short-Term Memory: The Larger the Context, the Lower the Accuracy

Short-term memory is the context window. Multiple studies show that when the context window expands from zero tokens to 256K tokens, the model's problem-solving accuracy actually declines. Even if vendors advertise a 1-million-token limit, continuing to fill it only degrades performance. This explains why stuffing all information into the context is the wrong strategy.

There are two ways to address this: context summarization (have the model summarize and start a new conversation) and compression (store conversation fragments in a database with assigned IDs and retrieve them by ID when needed). Both are core techniques of context engineering.

Context Window refers to the maximum number of tokens a model can process in a single inference, including the input prompt and generated content. It is essentially the model's "working memory," with a fixed capacity and a cost that grows linearly with length. The industry often assumes bigger is better, but research shows that when irrelevant information fills the window, the model's attention to key information is diluted, leading to the "Lost in the Middle" phenomenon—information in the middle of the context is most easily overlooked. Therefore, effective context management is not about pursuing the largest window but about ensuring a high signal-to-noise ratio. Summarization and compression are designed to retain the most relevant signals within a limited window, rather than simply expanding capacity.

The Three Categories of Long-Term Memory: Episodic, Procedural, Semantic

Nacho divides long-term memory into three categories: episodic memory (historical conversations, tool logs, interaction summaries), procedural memory (skills and workflow sequences), and semantic memory (user preferences, identity settings). This classification is not for academic elegance but to give the agent loop a clear retrieval path when assembling context.

His advice to engineers is direct: keep it simple. Start with just two structures—short-term memory plus long-term memory. When something in short-term memory is worth keeping, promote it to long-term memory as raw text, then vectorize it so the agent can recall it on demand.

Toolbox and Skillbox: Don't Let Tools Eat Your Context

An easily overlooked pitfall is the context space consumed by tools and skills. Nacho reveals that he has installed over 100 skills, each taking up about 3,000 to 5,000 tokens; stuffing them all into the context would instantly blow past the limit. His solution is the toolbox and skillbox pattern: vector-embed tools and skills, have the agent loop perform a vector similarity search based on the task, extract only the three most relevant (K=3), optionally add a reranker to select one, and unload them after use.

When calling the chat completion API, tools and skills often account for 80% of the tokens in the entire request (roughly 35,000 to 100,000 tokens). The reason costs don't quintuple is that the KV cache reuses repeated prefixes. This also shows from an economic perspective that trimming the scaffolding is not just a performance issue but also a cost issue.

KV Cache (Key-Value Cache) is an optimization mechanism in Transformer inference: when generating each token, the model caches the key-value pairs of previous tokens to avoid redundant computation. When multiple API requests share the same prefix (such as a fixed system prompt or tool definitions), the cache can reuse that computation, reducing latency and cost. However, the cache only works for identical prefixes; once the tool list or skill descriptions change, the cache is invalidated. This explains why a lean and stable scaffolding not only improves performance but also significantly reduces token overhead by increasing cache hit rates.

Files or Databases: An Old Debate Being Reinvented

A sharp point from the talk: when multiple agents write to the same file simultaneously, the current mainstream workaround is to add a work queue—yet this is precisely the ACID consistency problem that databases solved thirty years ago. Nacho's position is that you don't have to choose: files are good for storing large amounts of unstructured data, databases are good for transactional writes, and agent architecture should mix them according to the scenario.

For SQL generation, he recommends using pre-validated query templates via the Model Context Protocol (MCP) rather than having the agent dynamically fabricate SQL each time—the latter is the main source of failures when agents handle databases.

ACID refers to the four core properties of database transactions: Atomicity (a transaction either completes entirely or rolls back entirely), Consistency (data satisfies constraints before and after the transaction), Isolation (concurrent transactions do not interfere with each other), and Durability (data is permanently saved after commit). When multiple agents write to the same file simultaneously, these guarantees are absent, making dirty writes and lost updates likely. Traditional databases natively solve concurrency consistency through mechanisms like locks, logs, and MVCC, so in agent architectures involving transactional writes, reusing database capabilities is more reliable than building a queue at the file layer.

Hands-On Workshop: An AML Agent Running on Codespace

After registering with a GitHub account, workshop participants automatically get Codespace access. The environment includes a Llama 3 model deployed on Oracle Cloud, database connections, and several Jupyter notebooks. The agent loop has a maximum of 8 iterations and a 60-second inference time per iteration, and is configurable.

Participants see a self-running AML agent: it generates simulated transactions, identifies suspicious patterns, calls tools from the toolbox, retrieves evidence from a vector knowledge base, and finally automatically produces a suspicious activity report. The interface provides a visualization of the agent's context window to help understand what is assembled in each round.

Nacho specifically highlights the value of hybrid retrieval (RRF): the combination of semantic search and keyword search, at a slight increase in computational cost, yields noticeably better retrieval quality over time. This is especially critical for financial scenarios that require precise matching of transaction characteristics.

RRF (Reciprocal Rank Fusion) is an unsupervised hybrid retrieval fusion algorithm. It does not rely on the raw scores returned by different retrievers (which have different scales and are not directly comparable); instead, it computes a fused score based on each document's rank in each retrieval result: score = Σ 1/(k + rank), where k is a smoothing constant (typically 60). The higher the rank, the greater the contribution. RRF effectively combines the strengths of semantic search (vector similarity) and keyword search (such as BM25). In financial scenarios, it captures both conceptual similarity and precise literal features such as transaction codes and amounts, and can be deployed without training.

Takeaways for Developers

The core argument of the talk can be condensed into one sentence: Don't obsess over switching frontier models; optimize the 90% you can control—memory structure, context assembly, and tool scheduling. For builders in finance, this means tailoring the framework to the specific problem rather than copying a generic agent solution. Nacho advises consultants to build a custom framework from scratch and strip out the 90% of components they don't need, because "everything is built around the chat completion API"—the leaner, the better.

There are three paths for continuous learning: parametric memory (fine-tuning weights), representational memory (embedding models), and non-parametric memory (the architectural layer discussed in this article). Nacho's workshop focuses on the third because it is the most controllable and fastest-iterating direction with frozen models.