Agent Sends Duplicate Emails/Bookings, Yet All Traces Look Normal? Here's How to Fix It

Agent retries and dual-active scenarios cause duplicate side effects, while traces lack a business deduplication perspective—idempotency keys and reconciliation are needed as a safety net.
Starting from a Reddit discussion, the article describes scenarios where an Agent worker crashes mid-tool-call and retries, or freezes and later resumes causing dual-active writes, resulting in duplicate irreversible operations like duplicate emails and duplicate bookings—yet both execution traces appear normal. The author points out that conventional observability is execution-unit-centric and lacks a cross-trace perspective on business side-effect deduplication; there's a missing translation layer between system-level "at-least-once" semantics and business-level "exactly-once" expectations. Viable defenses from the comments include: pushing idempotency keys down to the tool layer, using leases and fencing tokens to prevent old workers from writing, and periodically reconciling with external systems. The article emphasizes putting "how to detect duplicates" into design reviews.
An Agent Failure Scenario That Keeps Developers Up at Night
Recently I came across a Reddit discussion where a developer tested what happens when an Agent worker crashes mid-tool-call, and discovered a hidden vulnerability: duplicate operations occurred, but the observability system showed no reaction whatsoever.
The scenario isn't complicated, but it's very typical. The Agent calls a tool to send an email to a customer. The email provider has already accepted it, but the worker crashes before saving any record. The retry mechanism kicks in, and the email is sent again. In the trace, this appears as one failed run plus one successful run—two independent execution traces, with nothing linking them together.
Worse still is when a worker freezes mid-send (say, a GC pause or the machine gets suspended), another worker takes over the task and sends, and then the first worker wakes up and sends again. Both traces look completely normal.

The poster asked a critical question: if your Agent performs irreversible operations (sending emails, booking cargo, placing orders), would you even notice the duplicates? Or would you only find out when a customer replies, "You sent it twice"?
What makes this problem thorny is that it simultaneously exploits the soft spots of both distributed systems and Agent architecture.
Why Conventional Observability Can't Catch Duplicates
The traces and logs most teams rely on are fundamentally execution-unit-centric. Every run, every span, every tool call has its own ID and status—success is success, failure is failure. This model works well for troubleshooting performance issues and error rates, but it inherently lacks a cross-trace "side-effect deduplication" perspective.
When two runs each successfully execute the same business action, the trace system sees two normal events, not one duplicate. It has no business semantics and doesn't know that the action "send email to Zhang San" should globally occur only once.
The two failure modes the poster described—crash-before-write and freeze-then-dual-active—are nothing new in distributed systems; idempotency keys and two-phase commit are classic solutions. But Agent scheduling patterns amplify the problem: task retries are default behavior, workers have no strong leader election, and a frozen process may resume execution without checking task ownership.
In other words, the system-level "at-least-once" semantics meet the business-level "exactly-once" expectation, and there's a missing translation layer in between.
At-least-once semantics is a common delivery guarantee in distributed messaging and task queues: the system ensures that a message or task will eventually be processed, but under network jitter, worker crashes, timeout retries, and similar conditions, the same task may be delivered multiple times. In contrast, exactly-once semantics means each task is processed once and only once, which is generally difficult to truly achieve end-to-end and typically requires idempotent consumption, transactional deduplication, or two-phase commit. Agent architecture adopts retry mechanisms by default, essentially inheriting "at-least-once" semantics; yet business actions like sending emails and placing orders expect "exactly-once." This gap is precisely the root cause of duplicate side effects. Understanding this makes it clear why relying solely on "run success/failure" judgments in traces is insufficient—the system level permits multiple executions, while the business level accepts only one.
Viable Lines of Defense: From Idempotency to Business Reconciliation
The poster asked, "Have you caught this before, and what did you change afterward?" The comment section's approaches roughly fall into several layers.
The first layer is pushing idempotency keys down to the tool layer. Generate a deterministic ID for each business action (for example, based on order number plus action type), and the tool checks whether this ID has already been processed before executing. The email-sending API may not support idempotency, but you can write an "intent record" before calling and mark it complete after success. On retry after crash recovery, read this record first and skip if it already exists.
The second layer is leases and fencing tokens. For the frozen-worker-resumes scenario, increment a fencing token when a task is taken over, and all downstream writes validate whether the token is the latest. Writes from the old worker get rejected. This requires cooperation from the tool side, but it solves the dual-active problem.
The third layer is business-side reconciliation. This is the most reliable but also the most easily overlooked. Periodically compare the set of "system believes sent" against the set of "provider actually sent," and alert on discrepancies. In the poster's scenario, if a business ID were recorded before each send, reconciliation would detect two send records for the same customer with the same content.
One comment noted that many teams only do tracing and not reconciliation early on, and only start filling the gap after customer complaints. The cost of duplicate irreversible operations far exceeds that of writing an extra reconciliation table.
Fencing token is a classic mechanism for solving the "old node resumes and continues writing" problem in distributed systems. The approach: whenever a task is taken over by a new worker, the coordinator (such as a lock service or task scheduler) increments a monotonically increasing token number and issues it to the new worker. All downstream storage or tools validate whether the token carried in a write request is the maximum value they've seen; if the request carries a stale token, it's rejected outright. This way, even if a frozen old worker wakes up and tries to continue executing, its writes are discarded due to stale tokens, preventing dual-active writes. The key prerequisite for fencing tokens is that downstream must participate in validation, so corresponding changes are needed at the tool or storage layer—the agent framework alone cannot guarantee it.
Observability Needs Business Semantics
The most valuable part of this discussion is that it exposes a gap in current Agent observability: we monitor "whether execution succeeded," not "whether a business action occurred only once."
In traditional microservices, idempotency and deduplication are often guaranteed by the business code itself, and the framework doesn't care. But Agents abstract tool calls into generic retryable actions, the retry granularity becomes coarser, and side-effect visibility actually weakens.
If your team is building Agents that send emails, create tickets, or deduct inventory, the poster's question deserves a direct place in design reviews: when a worker dies between the last side effect and the next retry, what does your system rely on to detect the duplicate? If the answer is just "the traces look fine," that may mean the defense line hasn't been established yet.
A pragmatic starting point: all irreversible tool calls carry business idempotency keys, write intent logs before and after each execution, and periodically reconcile with external systems. Traces continue to be used for performance troubleshooting, while reconciliation is responsible for answering "are there duplicates?"
The poster's closing line—"or wait for the customer to reply that you sent it twice"—is, to some extent, the real state of many Agent systems today. The signal that something went wrong comes from outside, not from internal monitoring.
Business reconciliation differs from traditional technical monitoring: technical monitoring focuses on "whether calls succeeded, how much latency, what error rate," while reconciliation focuses on "whether business states are consistent." A typical approach is to maintain a system-side record of business actions (e.g., "email sent to customer A, business key is order 123"), then periodically pull the actual execution records from external providers (e.g., the email provider's send logs), compare sets by business key, and alert on discrepancies such as "system believes sent once, actually sent twice" or "system believes sent, actually not sent." Reconciliation doesn't depend on trace execution paths, so it can catch duplicate side effects across runs and across workers. Its cost is additional storage and scheduled jobs, but relative to the customer complaints and refund costs caused by duplicate charges or duplicate emails, it's usually the more cost-effective defense.
One-line Summary
Agent retries and dual-active scenarios cause duplicate side effects, while traces lack a business deduplication perspective—idempotency keys and reconciliation are needed as a safety net.
Paragraph Summary
Starting from a Reddit discussion, the article describes scenarios where an Agent worker crashes mid-tool-call and retries, or freezes and later resumes causing dual-active writes, resulting in duplicate irreversible operations like duplicate emails and duplicate bookings—yet both execution traces in the trace system appear normal. The author points out that conventional observability is execution-unit-centric and lacks a cross-trace perspective on business side-effect deduplication; there's a missing translation layer between system-level "at-least-once" semantics and business-level "exactly-once" expectations. Viable defenses from the comments include: pushing idempotency keys down to the tool layer, using leases and fencing tokens to prevent old workers from writing, and periodically reconciling with external systems. The article emphasizes putting "how to detect duplicates" into design reviews—traces handle performance troubleshooting, reconciliation handles answering whether duplicates occurred.
Related articles

Gemini 4 Neon Leak: A Potential Strong Contender Against Opus 5.5
A Hacker News thread suggests Gemini 4 Neon looks like an Opus 5.5 competitor. This article unpacks the product positioning behind this community signal, shifts in the top-model competitive landscape, and what it means for developers' model selection strategy.

Claude Fable 5.2's Self-Authored Short Film 'Fable' Sparks Discussion
A short film 'Fable,' credited to Claude Fable 5.2 for script, visuals, and directing, appears on Hacker News. This article analyzes the significance of full-pipeline AI creation, community reactions, and what it means for content creators.

Anthropic IPO Pricing: A Signal of AI Giants Going Public
Anthropic's IPO pricing draws attention. This article analyzes its potential impact on AI industry valuation, commercialization paths, and the competitive landscape, and highlights public-market expectations and challenges for AI companies.