Anthropic Engineer on AI Agent Trustworthiness and Observability

When Claude answers a complex question in seconds, users typically only see the fluent output — not whether the underlying data is accurate, well-structured, or privacy-safe. At Anthropic, one engineer's daily job is to find answers to exactly these questions.

According to IEEE Spectrum, Gurmeet Saran is a member of the technical staff at Anthropic and a senior IEEE member. He is building infrastructure to understand the mechanisms behind chatbot behavior and to ensure its underlying architecture produces reliable outputs. In his words: "My job is to make sure data is accurate, structured, and trustworthy — and that private data stays private. Privacy can't be a layer bolted on after the fact; it has to be part of the architecture from the start."

From Yahoo to Meta: Two Decades of Data Quality Problems

Saran's interest in computing systems began in his childhood in Kolkata. After graduating from high school in 1999, he completed his education in India: a bachelor's degree in Electronics and Communication Engineering from Kalyani University in 2004, and a master's degree in Software Engineering from Birla Institute of Technology in 2010.

In 2007, while still in graduate school, Saran joined Yahoo's Bangalore data team as a technical lead. He encountered a problem that would shape the trajectory of his career: different teams were computing the same website metric — say, daily active users — using slightly different logic, producing contradictory results. When leadership made resource allocation and product decisions, not knowing which number was correct became a serious problem.

Saran's team built a central data warehouse that defined a unified standard for metrics — essentially a "single source of truth." The models he created clearly defined what "daily active site users" meant: which events counted, how edge cases were handled. After that, data teams across the company could rely on a standard methodology to produce comparable results. On top of this, they also developed Yahoo! Digits, an internal web analytics tool used across all of the company's products.

In 2010, Saran left Yahoo to join Intuit as a senior engineer, helping build analytics infrastructure that unified data across products like TurboTax and QuickBooks. During this time, Intuit developed the ability to detect fraudulent tax filings in real time, which required the team to protect taxpayer data while complying with strict IRS policies. "That taught me that privacy and governance can't be layers bolted onto software after the fact — they have to be part of the architecture from the start."

In 2016, Saran joined Meta (then still called Facebook) as a principal software engineer. At the time, the company was mired in controversy over its video metrics: Meta publicly admitted that an error had caused it to overstate the number of views for published videos. Saran believed: "This wasn't a coding bug — it was an architecture problem." He built a system similar to what he had done at Intuit, defining a single source of truth for video viewing metrics across the entire site. That system was Metric360, which eventually became part of the site's architecture. By the time he left Meta in 2021, Metric360 was being used company-wide to compute video metrics. He then joined Wealthfront as a principal software engineer, helping build the company's data infrastructure.

What Is a Single Source of Truth (SSOT)

A single source of truth (SSOT) means that within an organization, only one authoritative definition and one authoritative copy of a given piece of critical data is maintained, and all downstream systems read from it rather than each maintaining and computing their own. The core problem it solves is the decision-making chaos caused by "one metric, multiple definitions" — exactly what Saran encountered at Yahoo with contradictory daily active user numbers. SSOT typically consists of three layers: the semantic layer (clearly defining what a metric means, which events are included, and how edge cases are handled), the computation layer (a unified ETL or metrics pipeline), and the access layer (a unified query interface). Meta's Metric360 was the SSOT for video metrics. In the AI era, the significance of SSOT extends further: RAG knowledge bases, evaluation benchmarks, and training data annotations all require unified definitions — otherwise the facts the model learns are themselves contradictory.

Building Trustworthy AI Systems at Anthropic

Saran joined Anthropic in 2025 as a member of the technical staff. He uses a medical analogy to explain the significance of his work: "Imagine a doctor monitoring a patient's vital signs and making treatment decisions based on them. What if one or several of those medical devices were slightly miscalibrated? Even if most readings are fine, that's a serious problem, because difficult decisions happen precisely at the margins. The same applies to AI systems. My job is essentially to make sure the instruments give accurate readings — that the data means what it's actually supposed to mean."

AI agents are especially hard to observe. Saran points out: "When a system performs a series of actions, you need a way to capture what it did, what information it was working from, and whether the results match what the end user expected. And you have to do all of that without exposing anyone's private information — the system is designed to identify sensitive data and keep it protected as it flows through. This is more demanding than any observability problem I've dealt with before."

Solving system reliability problems while AI adoption is still in its early stages is both Saran's passion and the problem that keeps him up at night. "I think making AI systems genuinely auditable and transparent is one of the most important engineering problems of the next decade," he says. "But it gets far less discussion than model capabilities do."

What Is AI Observability

Observability originated in cybernetics and refers to the ability to infer a system's internal state solely from its external outputs. In traditional software, it typically consists of three pillars: logging, metrics, and tracing. But observability for AI systems is far more complex: large language models are probabilistic — the same input can produce different outputs — their billions of internal parameters can't be audited one by one, and the reasoning process is often a "black box." As a result, AI observability must capture not just inputs and outputs but also intermediate steps like tool calls, retrieval sources, chain-of-thought, token consumption, and latency, in order to answer "why did the model give this answer?" The "action sequences and decision basis" that Saran emphasizes are precisely what elevates observability from traditional operations to the level of trustworthiness and compliance — only what can be observed can be audited; only what can be audited can be trusted.

IEEE's Role in Shaping AI Development

Saran joined IEEE in March 2025. "After more than 20 years building data infrastructure at organizations like Yahoo, Intuit, Meta, and Wealthfront," he says, "I wanted to give back to the technical community through peer review, publishing papers, and participating in the broader engineering research ecosystem."

He believes IEEE can play a key role in shaping the future of AI through standards work. "From building codes to software testing, we have standards for everything. AI infrastructure needs something similar, and IEEE has the credibility and influence to make that happen."

Another problem IEEE can help address is access: "IEEE operates in many countries where engineers don't easily have access to a global professional community. That connection matters, especially as AI development expands beyond a handful of organizations in a handful of cities."

For those focused on putting AI agents into practice, Saran's work reveals an easily overlooked fact: model capability is only half the story — data trustworthiness and system observability are the other half of whether agents can truly be trusted. As the industry races toward more powerful models, making AI systems auditable, explainable, and privacy-built-in may be the engineering direction more worth investing in.

IEEE's Practical Role in AI Standardization

IEEE (the Institute of Electrical and Electronics Engineers) is the world's largest technical professional organization, and its standards association, IEEE SA, has published more than 1,200 active standards. In the AI field, it has advanced a series of concrete efforts — for example, the IEEE 7000 series focuses on "ethically aligned system design," IEEE 7001 addresses transparency of autonomous systems, and IEEE 7010 concerns assessment of AI's impact on well-being. While these standards don't carry the mandatory certification force of ISO standards, they are often used by companies as compliance references and best-practice guidance. For AI infrastructure, the value of standardization lies in translating "trustworthy" from a slogan into testable engineering metrics — such as privacy protection levels, auditable log formats, and model card disclosure specifications. What Saran means by "AI infrastructure needs standardization similar to building codes" is precisely this process of turning abstract principles into verifiable requirements.

One-line Summary

Anthropic engineer Gurmeet Saran believes AI agent trustworthiness depends on data quality, auditability, and privacy built into the architecture.

Paragraph Summary

The article profiles Gurmeet Saran, a member of Anthropic's technical staff and a senior IEEE member. He is building infrastructure to understand the mechanisms behind chatbot behavior and ensure the underlying architecture produces reliable outputs. Saran's career has been defined by a throughline of data governance: building a central data warehouse with unified metric definitions at Yahoo, constructing analytics infrastructure at Intuit with an emphasis on privacy not being bolted on after the fact, and creating the single source of truth for video metrics, Metric360, at Meta. After joining Anthropic, he brought this philosophy to AI, using a medical device calibration analogy to illustrate the importance of ensuring data accuracy, structure, and privacy protection. He points out that AI agent observability is more demanding than traditional systems, requiring the capture of action sequences and decision basis while protecting privacy. Saran believes making AI systems genuinely auditable and transparent is one of the most important engineering problems of the next decade, and he hopes to give back to the technical community through IEEE standards work.