AI Agent Cost Accounting: Price by Runs, Not by Tokens

Key insight: Tokens are the visible meter on the dashboard, but seconds are the meter that the deadline actually reads. Capping tokens while ignoring time delays delivery while your charts look healthy.

---

Why free capacity distorts cost judgment

Free tokens make retries cheap, so people stop counting attempts. But attempts are not free in time.

This failure mode shows up whenever free capacity is involved: retries are free in tokens but expensive on the wall clock. Two retries on a shared free tier might mean 90 seconds of generation plus 4 minutes waiting in queue. For an overnight batch job, that's fine; for anything where a user is staring at a progress bar, it's fatal.

Every retry re-buys the context

A retry doesn't resume from the last good token. In most agent loops, you append the failure message or validator output and send the entire context again.

• First attempt: base context
• Second attempt: context + the first error
• Third attempt: context + two errors, and so on

Total token cost per run grows faster than the attempt count itself, and wall-clock cost also includes queuing you don't control.

Wall-clock time is a measure of real-world elapsed time which, unlike CPU time or token consumption, includes all waiting: queuing, network round-trips, rate-limit backoff. In agent workflows, token usage is usually visible in the API response, but wall-clock time is often ignored. Under free capacity, each retry can trigger rate limits, leading to exponential backoff that makes wall-clock cost far exceed token cost. For example, two retries might consume only a few tokens, but if you queue 4 minutes on a shared free tier, the actual delivery delay destroys the user experience. So wall-clock time is the real measure of the deadline, and it needs to be explicitly recorded on every attempt.

Record both meters on every attempt

The cheapest fix is bookkeeping. Write one line per attempt, before deciding anything about the outcome.

# runledger.py — write one JSONL line per model attempt
import json, time, pathlib, hashlib

LEDGER = pathlib.Path("runs/ledger.jsonl")
LEDGER.parent.mkdir(exist_ok=True)

def attempt(task_id, prompt, client, accept, **gen_kw):
    rec = {
        "task": task_id,
        "prompt_sha": hashlib.sha256(prompt.encode()).hexdigest()[:12],
        "prompt_chars": len(prompt),
    }
    started = time.monotonic()
    resp = client.generate(prompt, **gen_kw)
    gen_s = time.monotonic() - started
    ok = accept(resp)
    rec.update(
        gen_s=round(gen_s, 2),
        wall_s=round(time.monotonic() - started, 2),
        prompt_tokens=getattr(resp.usage, "prompt_tokens", None),
        completion_tokens=getattr(resp.usage, "completion_tokens", None),
        accepted=bool(ok),
    )
    with LEDGER.open("a") as fh:
        fh.write(json.dumps(rec) + "\n")
    return resp, ok

Note what's not in the record: the model output. You're recording cost, not content. Keep outputs in your own storage so the ledger stays small enough to read.

Let the retry cap fall out of arithmetic

Most teams pick a retry cap by feel—three, five, or whatever number the last owner wrote down. You can actually derive it.

If each failure appends about e tokens to the initial context c, the cumulative token cost of n attempts is n*c + e*n*(n-1)/2. Invert that against your ceiling.

def attempt_cap(context_tokens, error_tokens, ceiling_tokens):
    """How many attempts (including resends) fit under a token ceiling."""
    spent = 0
    n = 0
    while True:
        spent += context_tokens + n * error_tokens
        if spent > ceiling_tokens:
            return n
        n += 1

print(attempt_cap(context_tokens=8000, error_tokens=400, ceiling_tokens=120000))

Run this with your real context size. The answer is usually smaller than the number you're currently using. A cap is a budget decision, not a personality trait.

Aggregate by accepted task, not by call

Per-call dashboards flatter themselves: every call succeeded, therefore the system is efficient. Group by task instead, and charge the cost of failed attempts to the task, because that's where the money went.

import json, collections

runs = collections.defaultdict(lambda: {"tok": 0, "s": 0.0, "n": 0, "ok": False})
for line in open("runs/ledger.jsonl"):
    r = json.loads(line)
    g = runs[r["task"]]
    g["tok"] += (r["prompt_tokens"] or 0) + (r["completion_tokens"] or 0)
    g["s"] += r["wall_s"]
    g["n"] += 1
    g["ok"] = g["ok"] or r["accepted"]

done = [g for g in runs.values() if g["ok"]]
print("accepted:", len(done), "of", len(runs))
print("tokens/accepted:", sum(g["tok"] for g in done) // max(len(done), 1))
print("seconds/accepted:", round(sum(g["s"] for g in done) / max(len(done), 1), 1))
print("attempts/accepted:", round(sum(g["n"] for g in done) / max(len(done), 1), 2))

The third number tells you whether to tighten the acceptance test or lower the cap. Once attempts per accepted task drifts above 2, your prompt is guessing rather than working, and no amount of free capacity fixes a guessing loop.

An acceptance test is a function that decides whether a model output is usable—for example, format validation, passing unit tests, or human review. In an agent loop, the acceptance test is the gate that decides whether to retry, and its strictness directly affects attempt count and total cost. If it's too loose, bad results get accepted and downstream fixes cost more; if it's too strict, retries climb and both token and time costs spike. The accepted field in the article is exactly the result of the acceptance test, and aggregation only charges cost to tasks that were eventually accepted. Optimizing the acceptance test is therefore the key lever for balancing cost and quality, and the core basis for judging whether "your prompt is guessing rather than working."

When free capacity is the wrong choice

Free capacity is a fair trade when it buys money with queue time. It becomes a bad trade the moment that queue time lands on someone else's critical path.

Task shape	Free capacity	Committed capacity
Someone watching a progress bar	Rarely worth it	Yes
Overnight batch, delivered in the morning	Yes, with a derived cap	Only if the deadline is a contractual obligation
Long-context deep retries	Only if the cap is enforced	Yes, because each retry resends everything
One-off task bursts	Yes, accept jitter	Yes, if jitter breaks downstream SLAs

Read the second column carefully. This is not a downgrade. Trading seconds for tokens is a legitimate engineering choice—as long as you know which currency you spent and it's on the record.

Limitations of this approach

The ledger only measures what the transport layer exposes. If token counts are missing, your totals are estimates and should be labeled as such. Queue waits vary by time of day, so a single afternoon's sample is misleading; collect at least a week before setting caps in production. The numbers are yours alone and not transferable—another team's tokens-per-task tells you nothing about your prompts.

Don't use this approach if you have hard external deadlines, sub-second interactive surfaces, or compliance rules requiring prompts to stay on your own network. And if your acceptance test is "looks fine," skip the ledger entirely; without a real test, accepted is a constant and the third metric becomes noise.

What the free tier is actually good for

One project worth knowing about: MonkeyCode is an open-source coding agent project, and its operators say new accounts get 10 million tokens on their free model plus a free server option. Treat that as operator-supplied information and check the current project page before you plan a budget around it, because quotas change.

Disclosure: this article was written as part of a MonkeyCode product promotion.

The honest way to use it is as a measurement environment, not a production answer. Point the ledger at a free model, run your own task set for a week, and you get real tokens-per-accepted-task and seconds-per-accepted-task without paying to observe. The free server option means the collector and aggregator cron jobs can live off your laptop.

Then read your own table. If seconds per accepted task is acceptable for your workload, keep spending the cheap currency. If it isn't, you now have a number to argue with instead of a feeling. It was never about the provider. It's about pricing the run.

One-line Summary

Tokens are just the meter; seconds are what the deadline actually reads, and capping only tokens delays delivery.

Paragraph Summary

The article argues that for AI agents, token billing and wall-clock time are two separate cost meters. Free capacity makes retries free in tokens while ignoring queue-time cost, which can leave users waiting too long. Each retry resends the entire context, making cumulative token cost grow quadratically, and wall-clock time is further affected by queuing. The author proposes logging both token and time metrics per attempt in a JSONL ledger and aggregating by accepted task to get tokens-per-task, seconds-per-task, and attempts-per-task. Retry caps should be derived from context size and token ceilings rather than guessed. When attempts per task exceed 2, the prompt is likely guessing rather than working. Free capacity suits scenarios like overnight batch jobs that can tolerate jitter, but not interactive surfaces with people waiting or tasks with hard deadlines.