Omni CTO in Practice: How Claude Code Is Reshaping Data Analysis Agents

At the Code with Claude conference, the CTO of Omni — an AI-powered data analytics platform — shared a surprisingly relatable story. As the technical lead of a growth-stage company with hundreds of customers and a 25-person engineering team, he had assumed he'd eventually stop writing code altogether. Then Claude Code came along and changed everything. "That was a really wonderful side effect of the rollout," he said. "It's let me still write code here and there."

This talk was more than praise for a tool. It was a thorough retrospective on how Omni built their data analysis agent "Blobby" from scratch using Claude — including 18 months of hard-won engineering lessons and the pitfalls along the way.

From Tentative Steps to Explosive Growth: The Inflection Point in Commit History

Omni's CTO showed a chart of commit volume to the main branch over time. The slope of the curve steepened sharply at a certain point — and that wasn't a coincidence.

Early in 2025, he told his team something simple but pivotal: "I don't know when or how, but I know our work is changing — so let's just start trying." The team ran a lot of experiments with mixed results, until Claude Code paired with a significantly stronger model was released. That's when senior engineers finally became convinced: "This is real. This can reliably help us."

From there, it was a runaway train. Engineers returned from the January holiday already up and running, and the steeper commit curve directly reflected the productivity leap. This velocity aligned perfectly with two of Omni's core values: one called "SHIP IT" and another around transparency. Every Friday, their all-hands meeting devotes over 50 minutes to live demos, which the CEO edits and publishes publicly on YouTube.

What Is Blobby: How a Semantic Layer Lets AI Understand Your Business

Omni's core product capability is letting users "talk to their data." A user asks a question, Claude translates it into a Semantic Query, which is then converted to SQL, executed against the data warehouse, and returned as a result.

The key here is the Semantic Layer — a "translation layer" that sits on top of the data warehouse. The CTO emphasized a challenge that's easy to overlook: Claude is very capable of answering general business questions, but to answer questions about your specific business, you have to encode your business definitions.

He gave a sharp example: even something as simple as "last quarter" means different things to different departments. At Omni, the product and engineering teams calculate by calendar quarter, while sales runs on fiscal quarters. These subtle distinctions must be encoded into entity context, cognitive layers, and data definitions before you can get the right answer.

The Semantic Layer does three main things: it organizes thousands of datasets (a real enterprise might have hundreds of revenue tables and hundreds of opportunity tables), provides localized context, and handles permissions. The CTO highlighted one key insight that also echoes the Claude Code philosophy: "Context is good, but context that's localized — placed right next to the definition it describes — works better." It's the same reason people rely on config files in Claude Code: the closer the context is to the code it applies to, the better the results.

The Semantic Layer is an abstraction layer in the modern data stack that sits between the raw data warehouse (e.g., Snowflake, BigQuery) and end users. Its core job is to translate physical table column names, relationships, calculation logic, and business rules into human-understandable business concepts. For example, a field called arr_delta in the database might be defined in the semantic layer as "Annual Recurring Revenue Delta," complete with its calculation formula, owning department, and applicable time granularity. For AI systems, the semantic layer is especially valuable: large language models have no prior knowledge of a specific company's business logic, so the semantic layer essentially provides a "domain dictionary" — helping the model understand what terms like "active user" or "net revenue" actually mean at that particular company before generating any queries. Common semantic layer tools in the industry include dbt Semantic Layer, LookML (Looker), and Cube.js. Omni itself is a player in this space.

Three Key Evolutions: From Single Q&A to a True Agent

Blobby wasn't always this mature. The CTO walked through its 18-month growth arc.

Phase 1: Fill in the metadata. The initial version was single-turn question and answer. The team quickly realized they needed more metadata and introduced three key types: AI context designed specifically for LLMs (telling the model how to use a given field), sample queries (example queries for typical use cases), and values (sample field values — for instance, region abbreviations so the model could infer that "US" means United States).

Phase 2: Add the Agentic Loop. This was a major engineering investment — the team built their own agentic harness from scratch. The most important lesson the CTO shared: agentic loops are exceptionally good at recovering from errors. So they did two things: they told Blobby how to recover from errors and gave it a budget to try, and they invested serious effort in crafting high-quality error messages (clearly describing what went wrong and how it might be fixed). That single change dramatically improved performance on harder evals. As conversation complexity grew, they also switched from Haiku to Sonnet, which increased token consumption — but that was by design.

Agentic Loop refers to an execution framework that allows a large language model to autonomously plan, call tools, observe results, and iteratively correct itself within a single interaction — as opposed to a one-shot input-output exchange. The basic structure is: model receives task → decides which tool to call → executes the tool and gets an observation → feeds results back to the model → model decides whether to continue or return a final answer, and so on. Agentic Harness is the engineering scaffold built around this loop, responsible for managing tool registration, context windows, error capture, retry logic, token budget control, and other infrastructure. Omni chose to build their own harness rather than use an off-the-shelf framework, giving them fine-grained control over every layer — including the ability to optimize error message formats for Blobby's specific domain, something that's often difficult to achieve with highly abstracted frameworks.

Blobotomies: Brain Surgery via Trace Analysis

When the CEO — described as the team's "loudest and most demanding user" — kept pushing to fix inconsistent answers, the team spent serious effort reading and analyzing traces (execution logs). The major refactors this triggered became affectionately known as "blobotomies."

A classic example was the "split brain" problem. Blobby was originally designed with an outer agent that generated task lists and an inner subagent dedicated to generating queries. It seemed like a clean design — but traces revealed that the outer agent had no idea which questions could be answered with a single query. So it would hand a compound question requiring multiple queries to the subagent, causing confusion.

The fix was what engineers called "unifying the brain" — lifting the tools directly into the outer agent harness to eliminate the cognitive disconnect between subsystem and outer layer. This change significantly improved performance on complex evals. The CTO's core takeaway: be extremely careful about how information and knowledge are distributed across different agent layers.

Betting on SQL: Letting Claude Do What It Does Best

Another major pivot came from an observation: generating SQL directly with Claude often answered questions Blobby couldn't handle at the time.

Omni had actually built a full SQL parsing engine years earlier, but it had been shelved for years because it couldn't reliably handle the wide variety of SQL inputs in the wild. The team made a bet: if Claude could generate expressive SQL, and the parser only needed to handle queries that roughly conformed to a standard shape, there was an opportunity. They also bet that Anthropic would keep making Claude better at SQL.

Engineer Steven dug up the old parsing code and switched the query generation interface from a strict private JSON format to SQL parsing mode. The results were immediate: questions that previously required three or four attempts — or awkward chaining — could now be answered in a single query. The CTO also noticed an interesting detail: Claude has a strong preference for writing SQL using CTEs (Common Table Expressions), and their parser happened to be especially good at handling CTEs — an unexpected perfect fit.

CTE (Common Table Expressions) are SQL query structures that begin with the WITH keyword, allowing one or more named temporary result sets to be defined before the main query executes. Compared to cramming all logic into a single deeply nested SQL statement, CTEs make query structure much closer to natural-language step-by-step reasoning — first define "active users," then define "paying active users," then compute the ratio. This structure is LLM-friendly for an intrinsic reason: a large proportion of human-written "readable SQL" in training data naturally tends to use CTEs to decompose complex logic. So Claude reproduces that style when generating SQL. Omni's optimized parser support for CTEs is essentially an engineering investment in handling structured, layered query patterns — and the two design preferences happened to align perfectly, validating the strategic choice of "betting on the output style the model does best."

Being Your Own Tool's User: Claude Code Feeds Back into Product Design

Today, Blobby has a complete agentic system: an outer loop handling checkpointing and failure recovery, and an inner loop with a rapidly growing toolkit — dashboard generation, visualization, data modeling, validation tools, and more.

The CTO was especially enthusiastic about the eval system — but for an unusual reason. What he values most is the raw observability that evals provide through trace data, letting him ask "why didn't this work?" and then dig into the data to find the answer.

The deeper insight: as heavy users of Claude Code, Omni's engineers came to deeply understand what a well-designed agent harness should look like. The CTO gave an example: when the team wanted to design new ways to explore the Semantic Model, they would first look at how Claude Code does it — after all, "a Semantic Model is really not that different from a codebase." This ability to be a genuine user of your own tools lets engineers deeply inhabit the problem and see what the state of the art actually looks like for solving it.

Closing Thoughts

Omni's talk laid out a clear evolutionary path for agent engineering: localize and enrich context → build an agentic loop with robust error recovery → use trace analysis to drive structural refactors → bet on the model's strongest capabilities (SQL) → use evals to ensure predictability. For any team building AI agents, this set of lessons distilled from a real production environment is far more actionable than abstract methodology. And that sharply steepening commit curve might just be the most visceral proof point for what AI-assisted development can actually deliver.