How AgenticOS Tracks Agent Run Costs by Task

Title

How AgenticOS Tracks Agent Run Costs by Task

Meta Description

AgenticOS brings a run's model, token usage, reported cost, and tool events together in the Activity view, tying cost figures to specific tasks. Based on a demo record, this article explains how to read these fields, how to do task-bound cost comparisons, how to handle usage from failed runs, and where the accounting boundaries of budget limits lie.

Article Content

Most teams only find out how much an agent cost them when the vendor invoice arrives at the end of the month, and even then they can't map that figure to the task that produced it. AgenticOS brings a run's model, token usage, reported cost, and tool events together in the Activity view, connecting cost figures to specific tasks. Based on an existing demo record, this article explains how to read these fields and how to use them to test agents and set budgets.

One caveat: the amounts in this article come from existing demo screenshots. They are not vendor invoices, not current pricing recommendations, and not a benchmark for any typical workload.

Opening a Run Record

The demo record shows Support Copilot answering "How do I rotate vendor keys?" via Slack. It's tagged with OpenRouter and anthropic/claude-opus-4.5, 5,397 input tokens, 170 output tokens, a reported cost of $0.0312, and a completed search_documents call in the timeline.

These fields need to be read together. An amount without a task tells you nothing about whether the run was useful; token counts without the resolved model aren't enough to explain pricing; and a status of success only means the run completed — the quality of the source passages and the answer still needs human review.

What this interface does is tell you where to look. It can't tell you how much a different document question would cost, nor how much switching models would save. For that kind of comparison, you need to run it yourself with a fixed task and input.

OpenRouter is an API gateway that aggregates multiple model vendors, exposing models from Anthropic, OpenAI, Google, and others through a unified interface, and forwarding and billing per call. The anthropic/claude-opus-4.5 in this article is OpenRouter's model identifier format: the vendor identifier comes before the slash, and the specific model name comes after. This naming convention means the model field in a record may be resolved by a proxy layer and isn't necessarily the same as what you originally wrote in your configuration.

A token is the smallest unit of text a model processes, roughly corresponding to a word or part of a word. Input tokens refer to the prompt sent into the model (the combined total of instructions, tool definitions, retrieved material, conversation history, and so on), and output tokens refer to the reply the model generates. Vendors price the two categories separately, usually with a higher unit price for output than input. That's why a figure like "5,397 input, 170 output" doesn't determine an amount on its own — you need the resolved model to know which price list applies.

Binding Cost Comparisons to Specific Tasks

Suppose you want to shorten the prompt for a customer service agent. The approach is to pick a small sample of questions, keep the expected source material, and run both the current configuration and the modified version against the same set of cases.

Record the model, answers, retrieval sources, status, and total usage for each configuration. If fallback is enabled, the line in the documented record may still show the primary profile even if a different model actually answered. For a controlled model comparison, use a test profile with no fallback, or pull vendor-side evidence separately. Also check whether the shorter prompt still retains the instructions the task depends on — if the answer drops an important exception, lower usage doesn't mean an equivalent result.

When evaluating the complete task, include retries and failed attempts as well. Cherry-picking the cheapest successful run hides the cost of getting to that answer. If you changed the model or vendor settings alongside the prompt, record that change rather than attributing the whole difference to the prompt.

Fallback refers to the mechanism where, if the primary model call fails or times out, the system automatically switches to a backup model to continue answering. It's valuable for availability but creates headaches for cost attribution: the documented record may still label the primary profile while the backup model is what actually completed the request and incurred the cost, so the amount and the model field don't match up.

For a controlled model comparison, use a test profile with fallback disabled, or pull vendor-side evidence directly to verify the real calls. Only when you can guarantee "one run corresponds to one known model" are cost and quality differences between models interpretable; otherwise you may be comparing two entirely different call paths.

Work Before a Failure Costs Money Too

A run can fail after the billable model request has already been sent. AgenticOS retains failed runs and their recorded usage in history, which makes failure records meaningful when auditing workflow costs, even if the user never received a final answer.

When reviewing a failed run, first pinpoint where it stopped. A tool error after model work is a different matter from a request rejected before execution. The record helps you map usage to the progress completed before the failure.

Don't assume every failure costs the same, and don't assume a failure is necessarily more expensive than a success. Look at the actual events and usage recorded for the task at hand. If retries occurred, evaluate the resources needed to complete by looking at the earlier attempts together with the successful one.

Understanding the Coverage of the Spend View

The existing Spend screenshot shows a warning about unpriced runs. A missing price is not the same as a price of zero: the displayed total may be a lower bound on usage within that time window.

Before comparing the dashboard to your vendor account, confirm which model identifiers have pricing and whether the vendor or proxy added fees outside that query. Also compare the same reporting period — calendar-month figures and rolling-window figures can describe different sets of runs even when they appear on the same page.

Model usage is only part of the cost of an agent workflow. External tool services, sandboxed compute, and storage may each have their own bills. When estimating the operating cost of a whole task, keep all those components visible instead of treating the model cost field as the entire invoice.

"Unpriced run" means the system can't find a price list entry corresponding to a model identifier in the record, and therefore can't compute an amount for that usage. This is distinct from "price is 0": the former means the amount is unknown, the latter means it's known and free of charge. When unpriced runs exist within a time window, the total shown in the view is a lower bound on that window's usage rather than the full picture, and the real amount may be higher.

Common pricing gaps include: newly launched or privately deployed models not yet entered into the price list; proxy layers adding service fees or markups during forwarding; and vendors settling outside the reporting period. When reconciling, also make sure you're comparing the same reporting period — a calendar month and a rolling window can cover different sets of runs even when displayed on the same page.

Setting Budgets on the Target Configuration

AgenticOS checks accounted spend before sending a model request. If the applicable limit is already exhausted, the next request may be rejected. To determine which limit applies to a given run, look at the published configuration — modifying draft settings and publishing them are two different things.

A useful experiment is to use a one-off configuration, deliberately set a very small limit, and observe the rejection path. Check the run status and recorded usage to confirm which setting caused the stop. What this establishes is the behavior of that experiment, not a universal ceiling for all possible workflows.

A budget is not a full reservation against future bills. Costs can still accrue after a request passes the check, and concurrent or already-submitted work may not yet be reflected in the accounted total. Pricing gaps are another reason the displayed amount may not describe all usage.

When deciding how much traffic to admit, look at these conditions alongside the limit. Spend limits and request rate limits answer different questions, and external service costs need their own controls.

Looking at the Inputs Behind the Numbers

Input usage may include instructions, tool definitions, retrieved material, and conversation history. The actual request also depends on runtime behavior, such as selection, caching, and history handling. Counting the words visible in a chat window won't reproduce that request.

If input grows over a long task, check what's being retained. Compaction can summarize history, but the summary may miss details needed for the next step, and generating the summary itself takes work. Before deciding whether the change helps, compare both task results and total usage.

Switching to a smaller model follows the same method: fix the source input and success criteria, check the answer, and record usage for the whole task. A cheaper single request only matters if it still does the work you need, or reduces whole-workflow cost at acceptable quality.

Compaction is a technique for summarizing earlier conversation history into shorter text during long tasks to control input token growth. It reduces the input cost of a single request but introduces two costs: first, the summary is generated by the model, which itself consumes tokens; second, the summary is lossy and may drop details that the next step actually needs, causing the task to fail or require extra follow-up questions.

So evaluating whether compaction helps can't rely on token counts alone. Fix the source input and success criteria, and compare both the final task result and total usage for the whole task (including summary generation). This mirrors the logic of switching to a smaller model: a cheaper single request only matters if it still completes the task, or reduces overall workflow cost at acceptable quality.

Keeping a Useful Run Record

On your first experiment, keep the task input, published configuration, resolved model, output, and usage together. Add failed attempts and flag external costs the platform didn't account for. That way, when you later change a prompt or swap models, you have concrete objects to compare against.

The governance guide explains budgets and their accounting limitations. The demo interface above shows how to read a single run; to estimate a workload, you need observations from the tasks and traffic you actually intend to support.

Which agent's cost do you most want to see first, and what would you compare it against?

One-line Summary

AgenticOS binds cost figures to specific runs and explains how to test agents and set budgets accordingly.

Paragraph Summary

Using an AgenticOS demo record as an example, this article explains how to read the model, token usage, reported cost, and tool events of a single run together in the Activity view, so that cost figures map to the tasks that produced them. The author emphasizes that the amounts are demo data only, and that cost comparisons must be bound to fixed tasks and inputs, with the model, answers, retrieval sources, status, and usage fully recorded. With fallback enabled, records may still label the primary profile, so controlled comparisons require a test profile without fallback. Failed runs can also incur billable requests, so failures and retries should be included when evaluating workflow costs. Unpriced runs in the Spend view mean the displayed total is only a lower bound, and model cost is just one part of overall operating cost. Budget limits are checked before requests but are not a full reservation against future bills.