LangGraph Node Crashes Causing Duplicate Payments? Solve It with Durable Side Effects and Idempotency Keys

When AI agents perform irreversible operations such as payments and refunds, a subtle failure mode can cause real financial losses: a LangGraph node crashes after calling the payment API, and when the thread is resumed, the entire node re-executes, sending the payment again. Developer u/Sevenbox shared the reproduction process and solution to this problem on Reddit—using durable side effects and idempotency keys to intercept duplicate calls upon recovery.

Problem Reproduction: Why Crash Recovery Triggers a Second Payment

When users use LangGraph's checkpointer mechanism, the execution state of nodes is saved. If a node throws an exception after calling the payment API but before returning, the last saved checkpoint is still the state before that node executed. When the thread is resumed, LangGraph re-runs the node from that checkpoint, and the payment call is sent again.

According to their tests, using InMemorySaver in the installed langgraph package and throwing an exception immediately after the payment call, 2 payments were indeed sent upon recovery. This is not a defect in LangGraph, but a classic challenge when combining "at-least-once" execution semantics with side-effect operations in distributed systems. For scenarios like refunds and transfers, duplicate execution means direct financial loss.

Checkpointer is the core component of LangGraph's durable execution. At the end of each step (super-step) in graph execution, it serializes and saves the entire graph state—including node inputs, outputs, channel values, and pending tasks—to external storage (such as memory, SQLite, or Postgres). When a thread is resumed, LangGraph reloads the state from the most recent checkpoint and continues executing nodes that have not yet completed. The key point is that the checkpoint saves a snapshot of the state before node execution, not the intermediate state of each side-effect call within the node. Therefore, if a node crashes after completing an external API call but before writing the result back to state, it will be considered incomplete and re-executed upon recovery, causing duplicated side effects. This is a direct manifestation of "at-least-once" semantics at the state machine level.

Solution: Durable Side Effects and Idempotency Keys

Sevenbox v0.2.0 introduced a mechanism: before a side-effect call, the attempt is first written to SQLite, and a current_operation_id() is generated that remains unchanged across retries and restarts, serving as the provider's idempotency key. The Reddit post above shows the key code structure:

• Use the @tool(durable=True, identity=[...]) decorator to mark irreversible operations (such as refunds) and specify uniqueness fields (such as refund_request_id, order_id, amount).
• Pass idempotency_key=current_operation_id() when calling the payment API to ensure the same logical operation uses the same key.
• Write a reconciler function that declares the idempotency window (such as 24 hours) and queries whether the key has already been successfully executed upon recovery.

When the thread is resumed and issue_refund is called again, the system refuses to execute until session.recover() confirms through the reconciler whether the operation has completed. If completed, the recorded result is returned without sending a new request; if not completed, it retries with the same idempotency key; if the provider cannot determine the status, it suspends and waits for manual handling.

In the same test, the recovery request was rejected, recover() confirmed the refund had succeeded, and the next recovery directly returned the recorded result—only 1 payment was sent in total.

Idempotency key is a unique identifier used in distributed systems to ensure that repeated execution of the same operation produces only one effect. Payment providers (such as Stripe and PayPal) typically require clients to include a unique key in the request header; the server associates that key with the processing result of the first request and caches it for a period of time. Subsequent requests with the same key will not execute the charge again but will directly return the cached result (or return a conflict error). The validity period of the idempotency key is defined by the provider; after the window expires, the same key may be treated as a brand-new request. Therefore, clients must ensure the key remains unchanged across retries and restarts, and understand the provider's retention window, in order to safely achieve "at-least-once calls, exactly-once effects."

The Idempotency Key Expiration Trap and Adapter Status

Idempotency keys are not valid forever. For example, Stripe retains idempotency keys for 24 hours. Therefore, the reconciler must declare its own window period; NotFound results outside the window are treated as "status unknown" rather than "safe to retry." This detail is often overlooked but is key to avoiding duplicate payments.

Currently, Sevenbox's LangGraph adapter sevenbox_node does not yet support durable mode, so the above logic needs to be written inside the node function. The user also noted that their LangGraph test simulates crashes by throwing exceptions, while the crash tests in the repository directly kill the real process, which is closer to a production environment.

Write-Ahead Logging (WAL) is a classic technique in database systems for ensuring atomicity and durability: before modifying actual data, the modification intent is first written to a log and persisted; upon crash recovery, a consistent state is restored by replaying or rolling back the log. The solution in this article places "recording the attempt" before "executing the call," which is exactly the application of WAL thinking to AI agent side-effect management—first persist the operation intent (including the idempotency key), then execute the external call; upon recovery, determine whether the operation has completed based on the persisted record, thereby avoiding duplicate side effects. This pattern does not rely on querying the external service's state, but uses the local log as the source of truth, complementing the reconciler's remote confirmation.

Implications for AI Agent Engineering

This case reveals a core challenge in deploying AI agents: LLM reasoning can be retried, but side effects cannot. Any node involving external state changes—payments, sending emails, creating tickets, calling third-party APIs—needs idempotent design. LangGraph's checkpoint mechanism provides recovery capability, but developers must ensure the idempotency of side effects themselves.

From an engineering perspective, placing "recording the attempt" before "executing the call" (write-ahead) is a general pattern, consistent with the WAL (write-ahead logging) approach in databases. For teams building decision audit layers, such mechanisms not only prevent duplicate payments but also leave a complete trail for post-hoc auditing. As AI agents move from demos to production, these "boring" reliability designs will determine whether systems can truly be entrusted with funds and critical operations.

One-line Summary

LangGraph node crash recovery resends payments; durable side effects and idempotency keys are needed to prevent duplicate charges.

Paragraph Summary

The article reproduces the problem of LangGraph agents resending payments when recovering from a crash in a payment node: checkpoints save the state before node execution, so recovery re-runs the entire node. The solution is to persist the attempt record to SQLite before the side-effect call and generate an idempotency key (current_operation_id) that remains unchanged across retries, combined with a reconciler to confirm the operation status upon recovery—if completed, return the recorded result; if not completed, retry with the same key; if status is unknown, involve manual handling. The article also points out that idempotency keys have expiration periods (e.g., 24 hours for Stripe), and NotFound outside the window should be treated as status unknown; currently, Sevenbox's LangGraph adapter does not yet support durable mode and must be implemented within the node function. The core takeaway: LLM reasoning can be retried, but side effects must be idempotent.