Safe Retries Without an Idempotency Key: Using Message-ID to Make Gmail Sends Reconcilable

Every article about idempotency ends the same way: "Just add an idempotency key." That advice holds up right until you actually start sending email. Gmail API's messages.send has no idempotency key parameter — you POST a raw message, get back a message ID, or the connection drops and you get nothing. At that point, you can only determine from the outside: did this email actually get sent?

Blind retries cause duplicate sends. Not retrying may silently drop a real notification. This is the worst kind of fork, because customers can see both sides.

This article is drawn from a build-in-public practice by an n8n workflow author. The core idea is: let the message carry its own reconciliation handle, enabling safe retries for Gmail sends with zero provider idempotency support.

The Core Insight: Message-ID Is a Key You Already Control

Every email has a Message-ID header. RFC 5322 requires it to be globally unique, and Gmail stores it, keeps it searchable, and preserves the SENT label on saved copies. So if you set the Message-ID yourself instead of letting Gmail generate it randomly, you turn your inbox into a lookup table that can answer: "Have I already sent this?"

The critical rule: Message-ID must be deterministically derived from the job itself — never from a clock.

// n8n Code node - 从稳定的 job key 确定性生成 Message-ID
const crypto = require('crypto');
const jobKey = $json.customer_id + ':' + $json.event_type + ':' + $json.order_id;
const hash = crypto.createHash('sha256').update(jobKey).digest('hex').slice(0, 32);
const messageId = '<job-' + hash + '@inbound.yourdomain.com>';

const raw = [
  'From: "Support" <support@yourdomain.com>',
  'To: ' + $json.to,
  'Subject: ' + $json.subject,
  'Message-ID: ' + messageId,
  'MIME-Version: 1.0',
  'Content-Type: text/plain; charset="UTF-8"',
  '',
  $json.body,
].join('\r\n');

return { json: { raw: Buffer.from(raw).toString('base64url'), messageId } };

Two properties are essential: the key is derived from the source event (customer id + event type + order id), so the same logical send always produces the same Message-ID; and it contains no $now, no $execution.id, no freshly generated UUID.

There's also a hard requirement: you must send raw MIME yourself — using the raw field of Gmail's users.messages.send. If you let the Gmail node construct the message for you, Gmail will assign its own Message-ID and you lose this handle. In n8n, this means using an HTTP Request node (or the Gmail node's raw field), not the friendly "Send" input.

RFC 5322's specification for Message-ID requires the format <local-part@domain>, where local-part should be globally unique. This header was originally designed for email threading and deduplication — not distributed system reconciliation — but its storage and searchability happen to satisfy the latter. Gmail's rfc822msgid: search operator hits this field directly with exact matching rather than full-text fuzzy search, giving it the precision required as a lookup key.

It's worth noting that RFC 5322 does not prohibit senders from setting their own Message-ID, and Gmail won't reject raw MIME messages with a custom one. However, if you use Gmail's higher-level API (like n8n's Gmail node in "Send" mode), the Gmail server will overwrite your value on receipt — which is exactly why you must submit the complete MIME directly via the raw field.

Recovery Steps: Ask the Mailbox, Don't Guess

With a controllable key, your sweeper can reconcile rather than blindly resend:

GET /gmail/v1/users/me/messages?q=rfc822msgid:<job-abc123@inbound.yourdomain.com>

• Hit (and the message has the SENT label): the send happened. Mark the job done, do not resend.
• Miss: the send did not happen. Retry with the same deterministic Message-ID.

The mechanism is that simple. No provider support needed, no extra services, no exactly-once dependencies. Just a key you inject and a query you can run.

This approach of "querying external state" rather than "relying on local records" is a variant of the Idempotent Consumer Pattern in distributed systems. Traditional solutions typically rely on a deduplication table in a database, using the idempotency key as the primary key and INSERT OR IGNORE or ON CONFLICT DO NOTHING to prevent duplicate processing. This approach is different: it uses Gmail's SENT mailbox directly as that deduplication table — requiring no additional storage infrastructure and avoiding consistency issues between the dedup table and the actual send (because the "record" and the "effect" are the same object). The tradeoff is a dependency on the eventual consistency of Gmail's search index, which is exactly what the "Search Index Is Not Instant" section below addresses.

Three Critical Caveats That Make or Break This

This isn't a free lunch. Skip these three points and you'll ship a bug worse than the one you're fixing.

The Search Index Is Not Instant

Gmail's search index can lag by seconds to minutes. A "not found" result immediately after a crash is ambiguous — it's not evidence that the message wasn't sent. So don't treat the first negative result as a signal to "retry immediately." Delay your sweeper, let the index catch up, then scan again. Only retry if it's still missing after that window. If a channel can't tolerate this delay, it should go to review, not automatic retry.

Gmail's search is powered by an asynchronous indexing pipeline. After a message is accepted by users.messages.send, it isn't immediately visible to q=rfc822msgid:... queries. In production, this delay is typically seconds to minutes, but there's no official SLA upper bound. This means "not found" (HTTP 200 + empty list) and "truly not sent" are indistinguishable states in the short window after a crash — the classic "ambiguous failure" scenario in distributed systems.

In practice, set the sweeper's wait window to a conservative value (e.g., 3–5 minutes), and after the first negative result wait again and re-query rather than immediately triggering a retry. For channels with zero tolerance for delay, this window itself is a signal: that operation isn't suited for automated retry and should route to a human review queue.

A Hit Proves Existence, Not Uniqueness

Finding a copy in your own mailbox is evidence you sent it, not a server-side guarantee that a second copy doesn't exist. To assert uniqueness, you must count: if the same Message-ID was delivered twice, the query will return two hits. So assert hits == 1, not hits >= 1. A count of two means a duplicate effect — that should go to review. That's exactly the signal you were trying to detect in the first place.

Record Effects Where Effects Happen

Count side effects at the provider, or record actual effect lines with Message-ID in your relay's acceptance log — never count in n8n. n8n's execution count tells you nothing about whether the email actually left.

Not Every Channel Deserves the Same Guarantee

Once this works, resist the urge to apply it everywhere. First ask: what's the actual cost of one duplicate on each channel?

• Customer emails (receipts, status updates): duplicates are mildly annoying, but missing is worse. At-least-once with a deduplication hint is the right tradeoff — and the Message-ID trick makes that tradeoff safe.
• Anything involving money or irreversible actions: force Unknown → review and let a human decide. The mechanism still gives that person a queryable handle, but never auto-recover.

Write this decision down per provider, because exactly-once is a property of the receiving system — callers can't impose it. Your workflow's job is to be honest about which kind of system it's talking to.

How to Test This

Like any crash-boundary test: put a relay in front of the provider that supports three modes.

1. fail-before-forward — Gmail never received it. Retry must be safe, and the resend must produce exactly one effect (query finds nothing, retry fires, the second attempt is the only one).
2. forward-then-drop-response — Gmail accepted it, the relay dropped the response. This is the ambiguous category. The sweeper must find the message by Message-ID and mark it done — not resend.
3. forward-and-respond — happy path. Ends in done, sweeper must not touch it.

Assert three things, all in the acceptance log rather than n8n: retrying the same key → exactly one effect; dropped response → no duplicate; ambiguous job → falls into review and is never auto-retried. And report mock results as mock results — a green mock test is not a Gmail guarantee until you've watched a real ambiguous send complete reconciliation yourself.

Closing Thoughts

You don't need a provider to hand you an idempotency key before sends can be safe to retry. What you need is: a key you control, a place that remembers it, and the discipline to ask rather than guess. For Gmail, that place is the mailbox itself, and that key is the Message-ID you set at send time.

n8n is the executor; the mailbox is the source of truth. Let each do its job, and duplicate emails stop happening.