AI Agent Adoption Is Soaring, but No One Can Prove ROI: What's the Real Problem?

AI agents are becoming standard equipment in enterprise tech stacks. From customer service automation to code generation, from data analysis to process orchestration, nearly every company of a certain size is piloting or deploying some form of agent. But there's an awkward reality: almost no one can produce convincing evidence of ROI. Amber Roberts, a technical marketing manager at Databricks, pointed out the core of this contradiction in a recent analysis, and her observations reflect what many in the industry are feeling.

The ROI Dilemma: Investment Is Rising, Where's the Return?

The AI agent adoption curve is climbing steeply. Databricks data shows that more and more enterprises are putting agents into production environments rather than stopping at proof-of-concept. Yet when finance departments or management ask for quantified ROI, most teams give vague answers.

The problem isn't that agents aren't producing value. Many teams have indeed observed positive signals like efficiency gains, shorter response times, and reduced manual intervention. But these signals are hard to systematically attribute and measure. How much labor cost did a customer service agent save the company? Of the tickets it handled, how many were truly completed independently? These seemingly simple questions often become extremely complex in actual measurement.

Why Is ROI So Hard to Prove?

Missing Measurement Frameworks

Traditional software investment returns have a mature measurement system: license costs, deployment time, user adoption rates, efficiency gains. But AI agents work in a probabilistic, non-deterministic way. The same agent can perform vastly differently across scenarios, making standardized ROI calculations difficult.

The Value Attribution Chain Is Too Long

An agent may have impact across multiple stages simultaneously: reducing repetitive work for frontline staff, accelerating approval processes, improving customer satisfaction. These impacts are scattered across different departments and metrics, making it hard to aggregate them into a single ROI report. What's worse, some value is "avoiding loss" rather than "creating revenue"—like preventing a compliance risk—and this kind of value is nearly impossible to quantify.

Cost Structures Are Underestimated

When calculating agent costs, enterprises often only see API call fees or cloud resource bills, while overlooking hidden investments in data preparation, prompt engineering, human review, and continuous tuning. Databricks' analysis points out that these hidden costs are often several times the visible expenses. When all costs are included in the calculation, the ROI figure may be far less optimistic than expected.

The fundamental difference between AI agents and traditional software tools lies in their operating logic being probabilistic rather than deterministic. Traditional software follows fixed code paths: given input A, it always outputs B, so its value can be precisely calculated—for example, automating a process that originally took 10 minutes, the time saved multiplied by labor cost is a clear benefit. But agents based on large language models (LLMs) may produce different results each time they execute: the same customer service ticket, the agent might complete independently today, and might request human intervention midway tomorrow. This non-determinism means the agent's output is a probability distribution rather than a fixed value. To calculate ROI, you must know the mean and variance of this distribution—that is, what's the average success rate and how large is the fluctuation. Most enterprises currently aren't collecting this kind of data at all, so naturally they can't arrive at credible return figures.

Hidden costs are the systematically underestimated part of AI agent projects. Beyond visible expenses like API call fees and cloud resource bills, enterprises bear multiple easily overlooked investments throughout the agent lifecycle: data cleaning and labeling (agents need high-quality structured data to work reliably), prompt engineering and iterative tuning (a production-grade agent's prompt often requires dozens or even hundreds of revisions), human review and fallback (when agents make mistakes, humans need to take over—this labor doesn't disappear, it just shifts), and continuous monitoring and maintenance (model updates and scenario drift can cause agent performance to degrade). In industry practice, these hidden costs are typically 3-5 times the visible API expenses. Ignoring them leads to severe optimistic bias in ROI calculations.

Don't Rush to Prove ROI—Build Measurement Capabilities First

Roberts' perspective suggests a more pragmatic path: rather than forcing together an ROI number, first build reliable measurement infrastructure. This includes tracking the agent's success rate in real tasks, human takeover rate, cost per task, and value distribution across different business scenarios.

Only after these foundational data are in place does the ROI discussion become meaningful. Otherwise, all debates about investment returns are just different assumptions trying to convince each other.

For teams currently pushing AI agent deployment, the priority right now may not be proving something to management, but ensuring they have the ability to prove something in the future.

Human Takeover Rate is a key metric for measuring agent autonomy, referring to the proportion of tasks where the agent requires human intervention during processing. This metric matters because it directly determines how much labor the agent actually saves. If an agent claims to have handled 1,000 tickets, but 700 of them ultimately required human takeover to complete, then the work it truly completed independently is only 300—far from the ticket volume it appears to handle. Paired with this is Cost per Task, which is the total resources consumed by the agent to complete one task, including token fees, computing resources, and allocated tuning and review labor. Only by simultaneously tracking success rate, takeover rate, and cost per task can enterprises form a complete picture of an agent's true value.

One-line Summary

AI agent enterprise adoption is soaring, but most teams can't prove ROI due to missing measurement and hidden costs.

Paragraph Summary

AI agents are becoming standard in enterprise tech stacks, ubiquitous from customer service to code generation, but almost no team can produce convincing ROI evidence. Databricks' Amber Roberts points out the core of this contradiction: agents are indeed generating positive signals like efficiency gains, but this value is hard to systematically attribute and measure. There are three reasons ROI is hard to prove—traditional software ROI frameworks don't apply to probabilistic, non-deterministic agents; value is scattered across multiple departmental metrics and includes unquantifiable "loss avoidance" benefits; and enterprises systematically underestimate hidden costs like data preparation, prompt engineering, human review, and continuous tuning, which are often several times visible expenses. Roberts suggests that rather than forcing together an ROI number, teams should first build measurement infrastructure tracking success rate, human takeover rate, and cost per task—otherwise ROI discussions are just different assumptions trying to convince each other.