From 50% to 92%: A Practical Methodology for Teaching AI Agents to Master Spreadsheets

At the AI Engineer conference, engineer Nunu shared his team's four-month technical journey: teaching a coding agent to handle spreadsheets as fluently as it handles Python or JavaScript. They raised accuracy on a financial analysis benchmark from 50% to 92%. This methodology offers valuable insights for anyone building AI agents in specialized domains.

Why Spreadsheets Are Surprisingly Hard for AI

Spreadsheets look simple, but for large language models (LLMs), they represent a seriously underestimated challenge. When a human opens Excel, they instinctively recognize the structure — here's a revenue sheet, there are the assumptions, that's the chart section. It's a highly visual, intuitive understanding.

LLMs see none of that. When you ask one "what's the revenue?", it first has to figure out: do you mean net or gross revenue? Which quarter? Which year? Is this a hardcoded number or a formula result? These are judgments humans take for granted, but they require explicit reasoning from a model.

This difficulty has deep technical roots: LLMs perceive the world as sequences of tokens, while spreadsheets are fundamentally two-dimensional — even multi-dimensional — relational structures, where cells reference each other through coordinates and a single value might simultaneously serve as an input variable, an intermediate result, and a final output. Humans use parallel visual processing to instantly recognize region boundaries and data flows in a table; LLMs must "flatten" this 2D structure into a linear sequence, a process that inherently loses spatial relationship information. On top of that, the Excel formula language (VLOOKUP, INDIRECT, array formulas, etc.) is its own domain-specific language whose semantics differ significantly from the natural language and general-purpose programming languages in LLM training data — making understanding and generation even harder.

This "implicit structure" problem is exactly what caused the team to hit wall after wall in their early attempts.

The Dead Ends

The team's first attempt split the work across three agents, with an "editor agent" at the core following a five-step flow: define end state → make a plan → execute → verify. This architecture did change the nature of the errors: instead of mistakes happening while building the financial model, they now occurred during planning — which is easier to correct. But the architecture turned out to be too rigid. The discovery phase only ran once at the start with no way to backtrack, and context couldn't flow between agents.

This reflects a common challenge in multi-agent systems. The theoretical basis for multi-agent architectures comes from distributed systems and microservices thinking: break complex tasks into specialized agents and integrate results through a coordination layer. But each sub-agent runs in its own context window, information transfer between agents requires explicit serialization, and the large amount of implicit "working memory" in human cognition gets lost in that transfer. The "non-backtrackable discovery" problem is essentially an inherent flaw of DAG-style workflows when facing dynamic, exploratory tasks — real-world spreadsheet analysis is often non-linear and requires frequent hypothesis revision as new information surfaces. This prompted the team to reconsider the tradeoff between "one powerful agent + rich tools" vs. "multiple specialized agents."

The team then went on a long exploration of how to represent spreadsheets to an LLM, trying nearly every format imaginable:

• SQL: Plenty of training data, agents should be good at structured data — but it didn't work in practice.
• XML: This is actually Excel's native storage format, so it seemed natural — still failed.
• CSV/TSV views: Poor as the sole interface, but frequently useful as a supporting component in a broader solution.
• HTML: Introduced layout and formatting concepts — the right direction, and ultimately inspired the team to build a rendering engine that let the agent "see" a rendered image of the table.

These apparent failures weren't without value — each had theoretical merit, and two of these representations ended up becoming effective components in the final REPL-based solution.

The Core Breakthrough: A Single REPL Tool Architecture

The real turning point was replacing the roughly 15 accumulated tools with a single Node.js REPL tool. All those 15 tools became JavaScript functions that the agent could freely combine within a single REPL call.

Why This Design Works

REPL (Read-Eval-Print Loop) is an interactive programming environment dating back to Lisp interpreters in the 1960s. Its defining feature is persistent state: results from each execution are retained in memory, and subsequent calls can directly reference prior results, forming a continuous reasoning chain — a fundamental difference from stateless function calls like REST APIs. In the AI agent space, REPL architecture is LLM-friendly because modern large models have been trained on massive amounts of code and can fluently express complex operational intent through scripting. Node.js's V8 engine also provides natural sandbox isolation for safely executing model-generated code, while JavaScript's async capabilities support concurrent operations. This paradigm of "expressing tool calls as code" essentially shifts the tool interface design from "nouns (a list of tools)" to "verbs (composable functions)."

Why JavaScript? The team needed a scripting language that's easy to run in a sandbox and that LLMs know very well. Python would have worked too — they just chose JavaScript. The actual spreadsheet processing logic underneath is implemented in a completely different language: C#. This is the elegance of the architecture: use a scripting language for agent interaction, use the right language for actual file processing.

The before-and-after contrast is stark. Before the change, an agent exploring a spreadsheet often needed 10 to 15 tool calls and frequently timed out — because calls were sequential, and parallel calls didn't help since results couldn't be combined. After the change, the agent can combine multiple operations in a single call and get all the results at once.

The Key Difference Between REPL and Code Mode

The concept of "Code Mode" isn't new — Anthropic's API and Cloudflare have both discussed it, with the core idea being to merge multiple tool calls into one. But REPL goes further. The critical difference is persistent state: the agent makes one REPL call, defines some variables, checks the results, does some reasoning, and on the next call those variables are still there — it can build on what it already did.

The team observed an interesting pattern: in pure Code Mode, agents often wrote long ~50-line scripts; with REPL semantics, they tended to write shorter scripts and interleave more reasoning between each step. This "interleaved" working style often led to faster, better answers.

Extending capabilities also became trivially simple: to add a new method (say, exploring formula dependencies), you just expose it as a method in the JavaScript REPL, declare it in the TypeScript type definition file, and put it in the prompt — no need to pile more tools onto the tool schema.

The Feedback-Validation Loop: The Foundation of Agent Quality

When writing code, allowing agents to run the compiler, linter, and test cases — then iterate based on results — significantly improves performance compared to running without feedback. Spreadsheets are highly analogous to programming: without the ability to check and verify, high-quality output simply isn't possible.

To that end, the team built two core engines:

• Formula engine: Accurately computes formulas.
• Rendering engine: Renders a specified region as an image with full formatting and layout, serving as the "source of truth."

This validation loop lets the agent confirm whether an operation was correct, and fix formulas or formatting when it wasn't. But there's a critical prerequisite: the engines must be high-fidelity.

There's deep engineering logic behind this requirement. Excel's formula engine has hundreds of built-in functions with numerous edge-case behaviors involving floating-point precision, date systems (the 1900/1904 date systems), and implicit intersection operators. If a custom engine only covers common functions, the "correct behaviors" the agent learns during testing will fail in production due to differences in function behavior — causing distribution shift. This also explains why an engine with 50% coverage actually performs worse than no engine at all: incomplete validation is more dangerous than no validation, because it gives the agent false confidence signals that drive it to over-confidently continue down incorrect paths. The quality of the validation loop is entirely determined by the completeness of the engines behind it.

What Changes, What Doesn't

REPL is fundamentally an interface — a way of presenting tools to the agent. The reason REPL is the best choice right now is that today's top models are strongest at programming. But this may not always be true: the major labs are investing heavily in "computer use" capabilities, and future models may be as fluent with a mouse and keyboard as they are with code. At that point, REPL may no longer be the optimal interface.

What won't change is the need for a validation loop — that's the more durable underlying principle. The team went through four or five model iterations during the project and consistently found: the more capable the model, the more it benefited from the validation loop.

In the leap from 50% to 92%, REPL contributed the most (jumping directly from 50% to 74%), with subsequent gains stacking from domain knowledge injection, fuzzy search, formula tracing, system prompt optimization, and bug fixes. Notably, this approach nearly eliminated timeouts — tasks that previously triggered five-minute timeout limits frequently now have zero timeouts due to the dramatic efficiency improvements.

Practical Lessons in Domain Knowledge Injection and Evaluation

Domain Knowledge Is About "Reminding," Not Teaching

Injecting domain knowledge into prompts was a consistently effective optimization across every iteration. This isn't because LLMs don't know what revenue or ARR means — quite the opposite. They know too much, and need to be "scoped" to what matters for the current task. These domain knowledge prompts are highly portable and carry over almost unchanged across different tool architectures.

Building a Trustworthy Evaluation System

The team initially relied exclusively on LLM-as-Judge evaluation. Popularized by research from Stanford and others, this approach excels at evaluating open-ended outputs and handling subjective quality dimensions. But it introduced "evaluator drift": when the underlying evaluation model gets updated, the prompt is adjusted, or the model applies different scoring standards in different contexts, longitudinal comparability of scores degrades severely — when a score changes, you can't tell whether the agent changed or the evaluator did.

To address this, they shifted to using deterministic comparison wherever possible — borrowing from software engineering's unit testing philosophy. Given a fixed input, verify whether the output exactly matches the expected value; the result is a boolean 0/1 with no ambiguity. For example, a "golden table" with predefined inputs and outputs serves as a black box: feed the same numbers into the model-generated table and verify whether the output matches. This hybrid evaluation strategy — deterministic verification for quantifiable outputs, LLM judgment only for outputs that can't be structured — is increasingly becoming the best-practice consensus in AI product engineering.

Another important lesson: an agent's "reasoning failures" are often actually infrastructure bugs. There might be a bug in the code itself, an example in the prompt might be wrong (and the model faithfully reproduced the error), or after a tool error the model gets stuck in a retry loop. Digging into execution traces often reveals fixable root causes.

A Transferable Methodology for Other Domains

The value of these lessons extends well beyond spreadsheets. The following principles apply to building AI agents in any specialized domain:

1. If your agent is making a large number of sequential or parallel tool calls, you've essentially invented a bad scripting language — give it a real one (Code Mode or REPL) instead.
2. Feedback-validation loops are essential. If your domain doesn't have off-the-shelf validation tooling, it's worth investing specifically in building rendering engines, computation engines, and similar infrastructure.
3. The interface matters enormously and needs to be continuously re-evaluated as model capabilities evolve.
4. Don't underestimate the value of planning and "thinking before acting" — simple optimizations here can yield significant gains.
5. Domain knowledge is about reminding, not teaching — remind the model what to focus on, rather than trying to instill knowledge from scratch.
6. Push for deterministic evaluation, and always inspect execution traces and the underlying pipeline, because an agent's confusion is sometimes just an infrastructure bug.

Distilled from hands-on spreadsheet work, this methodology provides a practical, actionable roadmap for building AI agents in any vertical domain.

Key Takeaways