Enterprise AI Agents in Practice: Using Agents for Incident Root Cause Enrichment

When enterprises start deploying AI Agents, the real dividing line isn't "what agents can do" — it's "what you dare trust agents to do." This observation from YouTube creator Niran, made in his vlog series, cuts to the heart of the core tension in enterprise AI Agent adoption: the tradeoff between capability and trust.

This article is based on a specific use case he shared — Incident Enrichment — and breaks down how to make agents genuinely useful in a low-risk, high-value scenario, rather than staying at the level of toy demos like "ask the VCF system how many VMs you have."

Don't Treat Agents Like Automation — And Don't Let Them Fight Fires Alone

Niran offers a fitting analogy: using AI is like strapping on an exoskeleton. If your boss asks you to write a document, you don't just have the AI write it and send it over directly — you draft it yourself, then let the AI polish it. Agents in the enterprise serve a similar role: augmenting human capability, not replacing human judgment.

He draws two boundaries that are easy to blur:

First, agents are not the same as automation. Automation excels at highly repeatable processes. The real value of agents lies in scenarios with many variables, hard-to-standardize logic, and a need for reasoning. If a process can be scripted, it shouldn't be handed to an agent.

Second, don't blindly trust agents to fix systems. Niran is candid: he would not put an agent on a network to directly modify systems. What agents should actually do is help humans locate problems faster and assist with root cause analysis (RCA), thereby shortening the time to identify and resolve incidents — that's where the real business value lies.

Why "Latency" Events Are the Right Starting Point

The use case centers on a business-critical application (like a checkout system), since these systems directly tie to business outcomes and their availability is enormously valuable.

Niran deliberately chose "latency" events rather than "component outages." The reason is straightforward: outages are easy to locate — you can see what's down. Latency is amorphous. It could originate from compute, networking, storage, the runtime, or any combination thereof, making it infuriatingly difficult to diagnose. That high degree of uncertainty is precisely what makes it ideal for showcasing an agent's reasoning value.

For latency issues, he identified three systems as potential root cause sources:

• VCF: Private cloud infrastructure covering compute, network, storage, and virtualization
• Kubernetes: The runtime for some application components
• Load Balancer (LB): The entry point through which users and systems access the application

Core Architecture: One Response Agent + Multiple Diagnostic Agents

The centerpiece of the design is an Incident Response Agent. When an incident fires, it's fed structured context and topology information so it accurately understands the composition of the monitored system. It then asks itself: which systems should I investigate?

This is where the first key architectural decision emerges — should one large agent have access to all systems?

One Large Agent vs. Multiple Smaller Agents

Niran gives two reasons to argue against a single large agent:

1. Blast radius is too large: A single agent holding access to all systems amplifies the impact of any failure.
2. Cross-system hallucinations lead to false positives: When one agent processes information from multiple systems simultaneously, the probability of misinterpretation, hallucination, or reasoning errors increases, generating excessive false positives.

His solution is to split into three diagnostic agents: an LB Diagnostic Agent, a Kubernetes Diagnostic Agent, and a VCF Diagnostic Agent. Each agent has a smaller context and a more singular responsibility — reducing both false positive rates and the surface area of trust.

Design Principles for MCP Servers

Each diagnostic agent accesses its target system through its own MCP server. Niran flags an important engineering practice: don't build MCP servers as mirrors of your API schema, or you'll expose all of the API's "ugliness" directly. The right approach is to design tools around "what you're trying to do," for example:

• LB MCP: get VIP, inspect pool, etc.
• VCF MCP: inspect VM, inspect host, get storage, etc.
• Kubernetes MCP: get pods, list events, etc.

Notably, all these tools are read-only/query-type operations (collecting data, latency, CPU info) — not modifications or system changes. This is the source of the "low risk" property.

---

What is MCP?

MCP (Model Context Protocol) is an open protocol introduced by Anthropic in late 2024, designed to standardize the way AI models connect to external tools and data sources. Think of it as the "USB-C port" of the AI world — regardless of whether the underlying system is a database, an API, or a local file, wrapping it as an MCP server lets agents call it through a unified interface. The core unit an MCP server exposes is a "Tool" — each tool maps to a specific operation, and agents use tool names and parameter descriptions to understand "what's available" and decide "when to call it." This is exactly why Niran emphasizes not mirroring API schemas: raw APIs often have dozens or hundreds of endpoints laden with internal naming conventions and historical baggage. Exposing them directly causes agents to get lost in a sea of irrelevant tools. A small set of carefully designed, read-only tools aligned to business intent significantly improves agent reasoning accuracy while naturally limiting the possibility of unintended actions.

---

What is Incident Enrichment?

Incident Enrichment is a well-established concept in IT operations (ITOps/AIOps). It refers to the process of automatically augmenting a raw alert or incident ticket with relevant contextual information — such as affected system topology, historical change records, correlated metrics, and potential root causes — transforming it into a "higher-information-density" ticket that shortens the mean time to diagnose (MTTD) for on-call engineers. Traditional enrichment relies on rule engines or fixed scripts and can only handle known patterns. Agent-based enrichment, by contrast, can perform cross-domain reasoning across evidence from multiple systems — particularly effective for symptoms like latency, where the presentation is vague and root causes are varied. The core business value metrics are typically the reduction in MTTD (Mean Time to Detect) and MTTR (Mean Time to Resolve), since every minute of downtime on a business-critical system translates directly to revenue loss.

---

Using an Identity Gateway to Abstract "Trust"

How do you control what these agents can access? Niran introduces an Identity Gateway (he mentions tools like AgentMinder for this purpose).

The gateway's role is to abstract "trust" away from the agents themselves: tokens precisely control which MCPs each agent can access and which intents it can execute, with grants managing permissions. Even if an agent behaves unexpectedly, it's strictly confined to its permitted scope.

He also stresses the necessity of layered security — for example, preventing the VCF Agent from bypassing AgentMinder and directly accessing the system. A single identity gateway layer isn't sufficient on its own.

---

Understanding Blast Radius

The concept of "Blast Radius" is borrowed from security engineering, where it originally referred to the maximum scope of impact from a security incident or system failure. In the context of AI Agents, it describes: when an agent behaves abnormally (hallucination, misclassification, or malicious prompt injection), what is the worst-case damage boundary? A single large agent holding read/write access to all systems has a blast radius equal to the sum of all connected systems. Splitting into multiple single-responsibility agents paired with fine-grained authorization from an identity gateway compresses the blast radius of any given incident to a single system or even a single operation type. This is philosophically aligned with the "fault isolation" design principle in microservices architecture — rather than assuming components always behave correctly, the architecture limits the propagation of single-point failures. For enterprise AI governance, this is also a critical piece of evidence for demonstrating "controllability" to audit and compliance teams.

---

What an Enriched Incident Looks Like

Here's how the full flow runs: the Incident Response Agent wakes the relevant diagnostic agents → each agent queries its system and returns diagnostic evidence → the response agent aggregates and reasons over the evidence → it generates an enriched incident update.

Sample diagnostic evidence:

• Kubernetes Agent: Mild degradation detected, 87% confidence; found 3 affected pods all running on worker 7; assessed no control plane issues or resource contention; suspected infrastructure dependency issue.
• LB Agent: Load balancer healthy, connection pool normal; unlikely to be the source of the problem.
• VCF Agent: I/O elevated, datastore latency increased; assessed storage degradation most likely the infrastructure root cause.

The response agent delivers a synthesized conclusion: "Most likely cause is storage degradation affecting VKS worker node 07, confidence 91%", with correlated metrics and reasoning.

The critical point: the agent itself executes no remediation actions. It only calls other agents, collects evidence via MCP, performs reasoning, and writes the results back to the incident system.

Imagine the difference: instead of a ticket that just says "critical application checkout latency degraded," you get a fully enriched record with affected nodes, confidence scores, and root cause hypotheses. When a human engineer steps in, their time-to-diagnosis drops dramatically.

Architectural Tradeoffs Without a Universal Answer

Niran is candid at the close: this is one use case, one proposed architecture — not a universal solution. He leaves two open questions for enterprises to work through themselves:

• Use a large agent to do many things, or use small agents each handling a single domain?
• Should MCP servers integrate a large number of tools, or maintain a lean toolset?

The answer depends on your requirements for "trust" and "deterministic outcomes." This is fundamentally an architectural tradeoff, not a question of technical superiority.

For enterprises, the lesson from the incident enrichment use case is clear: start by finding existing processes that are low-risk, high-value, and highly variable. Let agents handle evidence collection and reasoning enrichment, and keep final decision-making authority with humans. That may be the most pragmatic approach to enterprise AI Agent deployment right now.