Hand-Building an AI Customer Service Agent in Kotlin: Engineering Practices for RAG, MCP, and Multi-Agent Systems

While frameworks promising to "build autonomous agents in two lines of code" dominate the tech headlines, one reality keeps getting proven out: drop a demo-grade Agent onto a real customer service front line, and it will hallucinate, ignore business rules, and leak user data. Developer Diego Ferreira Caetano offers a different answer with a Kotlin Multiplatform project called DSupport — without using any Agent framework, hand-building RAG, MCP, and multi-agent orchestration from scratch, and turning each step into a runnable git tag (etapa-1 through etapa-12).

The core claim of this article isn't "anti-framework" but a more nuanced division of labor: which decisions should be handed to a probabilistic model, and which must be locked down in deterministic code.

Core Thesis: The Model Handles Understanding, Code Handles Guarantees

DSupport's architecture can be summarized as a responsibility table. The model (non-deterministic) handles four things: understanding natural language, deciding which tool to call, drafting responses, and routing between experts. The code (deterministic) handles four others: validating types and rules, hiding sensitive data, blocking operations that can't be allowed to proceed, and running offline tests.

The original article offers an intuitive comparison: pure if/else routing, when it encounters an expression like "I want a refund" that doesn't contain the word "refund," silently falls into the OTHER branch; while a pure model approach produces unpredictable answers, infinite loops, and runaway costs. DSupport chooses a hybrid path — 5 roles use the model, 14 components are pure code (guardrails, validators, TF-IDF retrieval, tools, MCP transport, tracing, evaluation scoring, etc.).

This ratio itself is an engineering stance: the vast majority of safeguard logic should not be left to the "suggestions" of a prompt.

Architectural Layers: Dependencies Point Inward Only, Models Are Replaceable

The project adopts Clean Architecture with Ports & Adapters, with clear module boundaries:

• contract: shared type definitions, zero external dependencies
• core: agent loop, orchestration, guardrails, review, and port interfaces; unaware of HTTP, Anthropic, or JSON
• adapters: implements the ports, including the Anthropic client, RAG engine, JSON files, MCP client
• mcp-orders / mcp-tickets: independent MCP protocol servers
• cli / server / app: terminal, Ktor SSE interface, Compose desktop and iPhone app
• eval: evaluation suite with a golden dataset

core's entire knowledge of the model is distilled into a single interface: LlmClient, with two methods — next (returns text or a tool call request) and structured (forces the model to fill in a JSON form). Swapping Claude just requires writing a new adapter, with zero changes to the core code. The direct benefit of this design: over 380 tests all run offline, and not a single test consumes API credits.

The Agent Loop: A Bounded for Loop

The original article describes the Agent loop in extremely plain terms: the model requests a tool, the code executes it, the result is appended back into the history, and the loop continues until an answer is produced. The repeat(maxTurns) in the code is the first non-negotiable rule — without a cap, the model will keep calling forever.

A few details worth remembering:

• The full history and available tool list are sent every turn
• Tool calls execute in parallel via async + awaitAll; two independent queries take as long as the slower one
• Results are appended back in request order, because the model API requires correspondence
• When the turn limit is reached, the code returns an escalation-to-human message rather than letting the model decide on its own

The author's summary hits the nail on the head: the model never executes anything, it only requests. Execution, validation, deciding what to return — all of that is the code's job.

On the "Tool Calling" (Tool Calling / Function Calling) mechanism: When generating text, modern large models can output, in addition to natural language, a structured "tool call request," typically containing a tool name and JSON parameters. The model itself does not execute this function; it merely returns "which tool I want to call and what parameters to pass" as a special output to the host program. The host program parses it, actually executes the function, then appends the execution result as a new message to the conversation history, so the model can "see" that result on the next turn and continue reasoning. This round trip of "request—execute—append—reason again" is the physical foundation of the Agent loop. Understanding this makes clear why the author emphasizes "the model never executes anything, it only requests" — tool calling is essentially schema-constrained structured output, and the authority to execute and validate always stays on the deterministic-code side. This is also the precondition that makes engineering measures like parallel execution, timeout control, and ordered result appending viable.

The Key to RAG Isn't Retrieval, It's the Relevance Threshold

DSupport's RAG implementation is deliberately kept simple: articles are split by paragraph, TF-IDF is used for word-weight vectorization, and VoyageEmbedder can optionally be plugged in for semantic search. What really matters is the minScore = 0.25 cutoff in SearchKbTool.

Without a relevance threshold, retrieval will always return "something," which is precisely the number one cause of "giving a confident answer based on irrelevant paragraphs." When the filtered results are empty, the tool returns nothing_relevant_found, and the knowledge expert is required to answer NO_ANSWER rather than fabricate. Each result carries the article title so the final answer can cite sources, and the validator checks whether the cited sources actually came from this retrieval.

You may not have noticed that retrieval itself is deterministic code — it works without an API key.

MCP: Move Tools into Independent Processes, Integrate with One Line of Config

The MCP portion is built on the official Kotlin SDK, runs over stdio, and the client launches the server as a child process. The session has only three messages: initialize, tools/list, tools/call.

The key to the server implementation is reuse: any Tool list can become an MCP server — the same tool class can be made remote without rewriting, and the readOnly and destructive annotations pass along with the protocol so the client can decide whether to require confirmation. On the client side there is strict fault tolerance: every remote call has a timeout, and both timeouts and exceptions are converted into model-readable error results, while genuine coroutine cancellations are rethrown as-is.

Adding a tickets server just requires one line in config/mcp-servers.json, with the Agent code completely unchanged — it doesn't know whether a tool is local or remote.

There's an easily overlooked trap here: in stdio mode, stdout is the protocol itself, and a casual println will corrupt the entire session, so the server redirects System.out to System.err.

On stdio Transport and JSON-RPC Sessions: MCP (Model Context Protocol) is an open protocol that decouples model applications from external tools/data sources. Its communication layer is based on JSON-RPC 2.0, and messages are just lines of JSON. Besides HTTP-based transport, the most common local mode is stdio: the client launches the server as a child process, and both sides write JSON messages to each other through that process's standard input (stdin) and standard output (stdout). This brings a natural constraint — stdout is the protocol channel itself, and any non-protocol content mixed in will cause the peer's parsing to fail, so the server must redirect logs and debug prints to stderr (which is why the original article redirects System.out to System.err). Understanding this makes clear why "adding a server with one line of config" works: the protocol standardizes tool discovery (tools/list) and tool invocation (tools/call), so the client doesn't need to write special code for each server, and the process boundary brings both isolation and reuse.

Multi-Agent: The Orchestrator Is Not Allowed to Have Data Tools

DSupport's orchestration design has a hard constraint: the orchestrator is given the role of a senior customer service coordinator, but holds no data tools. Its job is to determine who handles which part and always delegate to the experts.

How experts are exposed to the orchestrator is elegant — each expert is wrapped as a SubAgentTool, which to the orchestrator is just an ordinary tool. One detail in the code is worth learning from: SubAgentTool receives an agent factory rather than an agent instance, because each agent holds its own history and parallel calls cannot share it.

The original article explicitly warns: if you also give the orchestrator data tools, it will skip delegation and the whole design collapses. So DSupportAssembly fails to assemble outright when the orchestrator holds tools — enforcing architectural constraints in code rather than relying on documentation conventions.

On "Programming to Ports" and Sub-Agents as Tools: Ports & Adapters is the core technique of Clean Architecture — the core module only declares the interfaces it needs (ports), which are implemented by outer adapters, and the dependency direction always points inward. DSupport wrapping each expert agent as a SubAgentTool is exactly this idea reused at the Agent level: to the orchestrator, an expert is no different from an ordinary tool — both are just a callable request — and the orchestration logic need not be aware of how the expert works internally, which model it uses, or how long its history is. The reason a "factory" rather than an "instance" is passed in is that agents are stateful — they hold their own conversation history; if multiple parallel calls shared the same instance, the histories would contaminate each other. Using a factory to create an independent instance per call is the key to parallel safety. This design also makes experts replaceable and remote-able, in line with the MCP approach.

Guardrails: Rules the Model Can Neither See Nor Bypass

Guardrails are deterministic code running at three fixed points in the loop, exposed through the Hook interface with three aspects — beforeTool, afterTool, beforeFinal — each of which can allow, deny, or modify.

Two examples best illustrate the point. IdentityGate only allows queries carrying a valid order number and email to pass; on rejection it returns an instruction written for the model to read ("ask the customer for the order number"), and the model, reading the reason, can act correctly instead of blindly retrying. TicketDedup uses a Mutex to handle parallel scenarios: two simultaneously issued create_ticket calls reserve the customer identity through an in-flight set, preventing duplicate tickets.

The original article's conclusion is worth writing on the wall: writing "please do not create duplicate tickets" in a prompt is a wish; a Mutex is a guarantee.

Structured Output and the Producer-Reviewer Loop

The final answer is not a piece of prose but the SupportAnswer JSON form: containing answer, intent, sources, confidence, escalated, and ticketId. But the schema only guarantees the shape; the code validates the truth. The validator checks whether intent is within the closed set, whether confidence is between 0 and 1, whether the ticket number actually appears in the tool results (invention is rejected), and whether the cited sources actually came from retrieval.

The reviewer is itself a model, so the code first runs the cheap and precise validation gate, and only after passing does it ask the review model to judge. The review loop has a hard cap (2 rounds of revision); when exceeded, the code opens a human ticket, and the model has no authority to decide escalation.

On TF-IDF and Relevance Thresholds: TF-IDF is a classic word-weight vectorization method that uses "term frequency (TF) × inverse document frequency (IDF)" to measure how representative a word is for a piece of text — words that appear often in this document and rarely in the whole corpus get high weight, while high-frequency function words like "the" and "is" are pushed almost to zero. Its advantages are that it needs no model or API, is fully deterministic, and can run offline, which is why DSupport's retrieval consumes no keys; its drawback is that it only looks at surface word overlap and cannot understand paraphrasing, so the original article keeps an optional path for plugging in an embedding model. The threshold minScore treats the similarity score as a gate: results below the threshold are simply discarded, preferring to return "nothing relevant" over feeding weakly relevant paragraphs to the model. The significance of the two working together is this — the retrieval algorithm can be plain, but the decision of "when to judge that nothing was found" must be backed by a deterministic threshold; otherwise the model will take irrelevant paragraphs and fabricate a confident answer.

Observability and Evaluation: The Black Box and the Golden Dataset

Each event carries a runId and parentRunId; the parent id is passed through the coroutine context without any parameter threading, so the event list alone can reconstruct the call tree. The server flushes each event as a frame via SSE, flush lets the app show a real-time trace, and the API key always stays on the server side.

The evaluation module's data is even more telling. The golden dataset contains 15 cases, each run 3 times — because the model is non-deterministic, and the original article mentions that 3 cases produced different results between two single runs. Metrics improved from before to after: expected-fact hit rate 93% to 100%, cited sources 86% to 100%, escalation precision 80% to 100%, and the degradation rate after review failure 9% to 0%.

More noteworthy is that the evaluation uncovered problems that code review could never find: two of its own rules contradicting each other, the review model fabricating objections, a deterministic routing error, and a test case based on the author's incorrect assumption. Total cost was about 30 cents, taking an afternoon.

A Cost Reminder: Multi-Agent Isn't Free

The original article gives a concrete runtime figure: a single "order delay" question consumes 12 model calls, about 15k input tokens and 2k output tokens, costs about $0.0025, and takes about 10 seconds. The author's advice: use a single agent with tools for hour-long tasks, and only split out experts when the prompt and toolset bloat and the roles are clearly distinct.

Conclusion

The most valuable thing about this project is not the code itself, but the reusable way of asking questions it demonstrates: in your project, which rule could be code instead of a prompt? Software engineering hasn't changed because of AI. An Agent lacking strong typing, clear architecture, code-level guardrails, and automated evaluation is still just an unpredictable toy.

One-line Summary

A Kotlin project that backstops with deterministic code, hand-building RAG, MCP, and multi-agent customer service.

Paragraph Summary

This article introduces an open-source Kotlin Multiplatform project called DSupport. Without using any Agent framework, the author hand-builds RAG, MCP protocol integration, and multi-agent orchestration from scratch, presenting the evolution through 12 runnable git tags. The core thesis is a division of responsibilities: the model handles understanding language, selecting tools, drafting replies, and routing, while code handles validating type rules, hiding sensitive data, blocking dangerous operations, and offline testing — with 5 roles using the model and 14 components being pure code. The article breaks down, in turn, the bounded agent loop, RAG with the relevance threshold as the key, MCP moving tools into independent processes, the hard constraint that the orchestrator holds no data tools, guardrails running at three cut points, structured output and the producer-reviewer loop, as well as observability with parent-child runIds and golden-dataset evaluation, and finally reminds us that multi-agent isn't free.