Stop Making LLMs Do Math: A Sandboxed Agent Coordinator for Reconciling Indian Village Ledgers

Why We Stopped Letting LLMs Do Math

While building an AI agent system for reconciling Indian village ledgers, we ran into a fundamental problem: LLMs are bad at math, and even worse at running business logic directly. Initially, we had the LLM handle amount calculations and account reconciliation, and the errors piled up quickly. Indian village ledgers are often handwritten, chaotically formatted, and multilingual, and reconciliation involves a large number of additions, subtractions, and rule-based judgments. The LLM's "approximately correct" outputs simply don't work here—a one-cent error can cause the entire reconciliation to fail.

So we decided: the LLM should only understand and extract information, and all math and process control should be handed off to external tools. That sounds simple, but implementing it requires a sandboxed agent coordinator.

The reason LLMs are unreliable at arithmetic lies in their autoregressive generation mechanism: they "predict" the next most likely character token by token, rather than performing deterministic symbolic operations like a calculator. The model has no addressable registers or carry chains internally; operations like multi-digit addition and subtraction or cross-row aggregation that require precise intermediate state can only rely on pattern approximations learned from training data. Once the number of digits grows or carries and borrows appear, the error probability rises rapidly. Temperature sampling, context window truncation, and prompt wording changes also introduce additional uncertainty, causing the same problem to get different answers across calls. This fundamentally conflicts with the reproducibility of traditional programs, where "same input yields same output." In financial scenarios, this "approximately correct" behavior cannot pass any test case, because the criterion for reconciliation is zero error, not high-probability correctness. As a result, the industry is gradually reaching a consensus: positioning the LLM as a semantic parser and intent router, and pushing deterministic computation down to the code layer, is the more reliable division of labor under current engineering conditions.

Design Philosophy of the Sandboxed Agent Coordinator

The core idea is to limit the LLM's capability to the "understanding" layer, while handing "execution" off to deterministic code. We built a coordinator that parses the LLM's output and dispatches tool functions in a sandboxed environment. These tool functions are written in Python and focus on mathematical operations, data validation, and ledger comparison. Each LLM call returns structured instructions; the coordinator validates the instructions and then executes the corresponding tool in the sandbox.

The benefit is safety: even if the LLM hallucinates or produces malicious instructions, the sandbox prevents it from accessing system resources or performing dangerous operations. At the same time, the math becomes fully reproducible and auditable—every calculation has a code record instead of being a black-box LLM output.

Why Not In-Process Execution?

We initially tried letting the LLM call tools within the same process, but quickly discovered that this introduces serious risks. The LLM might generate infinite loops, memory leaks, or unexpected file operations. Sandboxing avoids these problems by isolating processes and limiting resource usage. Each tool call runs in a separate container, can only access the necessary input data, and its output undergoes strict validation.

In-process execution means directly calling LLM-generated code or tool functions within the same Python interpreter or the same operating system process. In this mode, the LLM's output shares memory space, file descriptors, and environment variables with the host program. Once the model produces hallucinated code—for example, mistakenly writing a recursion condition that causes a stack overflow, constructing an enormous list that exhausts memory, or assembling a path that deletes a directory—the failure directly affects the coordinator itself and even the entire service process. Sandboxing, by contrast, uses namespace isolation, cgroup resource limits, read-only filesystems, and network disabling to lock each tool call inside a disposable container: even if the code crashes or behaves abnormally, the blast radius is confined to that container and can be forcibly reclaimed after a timeout. This is not just a security consideration but also an availability consideration—a process-level failure interrupts the entire reconciliation task, whereas a container-level failure only requires retrying a single call. For financial processes that require auditing, the sandbox can also fully record the input and output of each call, forming a traceable execution log.

How the Coordinator Works

The coordinator is essentially a state machine. It receives user input, calls the LLM to parse the intent, generates a sequence of tool calls, and then executes them in order within the sandbox. The execution results are returned to the LLM for the next decision, until the task is complete. Throughout this process, the LLM only decides "what to do next" and never directly manipulates data.

This architecture performs excellently in the Indian village ledger reconciliation scenario. For handwritten digits, multilingual mixtures (English, Hindi, and local scripts), and ambiguous entries in the ledgers, the LLM can extract and classify them reasonably well. But once the amounts and categories are extracted, all subsequent aggregation, verification, and discrepancy calculation are done by code in the sandbox, and accuracy improves dramatically.

A state machine is an engineering paradigm that models system behavior as a finite set of states and transitions between them: at any moment the system is in a definite state, and upon receiving an input event it transitions to the next state according to predefined rules, possibly producing side effects. Here, the coordinator breaks the reconciliation task into a loop of "parse intent—generate tool call—wait for execution result—evaluate next step," and the LLM's output is constrained to structured instructions that conform to a schema (such as a tool name plus parameters) rather than free text. Before invoking a tool, the coordinator validates parameter types, value ranges, and preconditions; if validation fails, it rolls back and re-queries the LLM. This design confines uncertainty to the discrete decision of "which tool to choose," while each tool's semantics are deterministic and unit-testable. Compared with having the LLM generate an entire business process as code in one shot, the combination of a state machine and a toolset is easier to debug: when something goes wrong, you can pinpoint whether it was an intent recognition error, a parameter extraction error, or a tool implementation error, rather than facing an inexplicable black-box output.

Real-World Results and Reflections

Through this separation, we not only improved accuracy but also enhanced the system's maintainability. When reconciliation rules change, we only need to modify the tool functions in the sandbox, without retraining or re-prompting the LLM. The LLM's prompts can also be more concise, focusing on the information extraction task.

Of course, this architecture adds engineering complexity. It requires designing a robust coordinator, sandbox environment, and tool interfaces. But for high-stakes financial reconciliation scenarios, it's worth it. Our experience is: don't trust LLMs to do math, and don't let them run business logic directly. Let the LLM do what it's good at—understanding messy input—and then let deterministic code do the precise work.

This Reddit share, though brief, hits a key pain point in LLM applications. In multi-source analysis, this view has been echoed by other developers: LLMs are better suited as "translators" rather than "calculators."