Idempotency keys are the cheapest reliability feature you can ship

Every payment integration eventually ships the same bug. A client sends POST /charge, the request succeeds, the response times out in transit, the client retries, and the customer is billed twice. Nobody wrote incorrect logic. The bug lives in the gap between "the work happened" and "the caller learned that the work happened", and no amount of careful coding closes that gap on its own.

Idempotency keys close it. The idea is small enough to explain in one sentence: the caller attaches a unique key to a request, and the server promises that the same key will produce the same outcome exactly once, no matter how many times it arrives.

Why retries are not optional

If you run anything over a network, retries are already happening. HTTP clients retry. Load balancers retry. Your own SDK retries on connection reset. Kubernetes restarts pods mid-request and the work is delivered again by a queue redelivery you did not configure.

So the question is never "should we handle duplicates". The question is whether you handle them deliberately or discover them in a support ticket at 2am with a customer asking why their card was charged three times.

The four lines that make it work

The storage design is what people get wrong. An idempotency record needs to capture three things: the key, the request fingerprint, and the result. The result matters as much as the key, because a retry must return the original response, not just refuse to repeat the work.

create table idempotency_keys (
  key            text primary key,
  request_hash   text        not null,
  status         text        not null default 'in_progress', -- in_progress | done
  response_body  jsonb,
  response_code  int,
  created_at     timestamptz not null default now(),
  expires_at     timestamptz not null
);

create index on idempotency_keys (expires_at);

The primary key on key is the whole trick. It converts "have I seen this before?" from an application-level check into a database guarantee, which means it stays correct when two requests arrive at the same millisecond on different machines.

The part nobody warns you about

The naive flow — read the row, do the work, write the row — has a race. Two concurrent retries both read "no record", both charge the card, and both write. The primary key does not save you, because neither transaction tried to insert until after the damage was done.

The fix is to make the insert the claim:

def claim(key: str, request_hash: str) -> Claim:
    try:
        db.execute(
            "insert into idempotency_keys (key, request_hash, expires_at)"
            " values (%s, %s, now() + interval '24 hours')",
            (key, request_hash),
        )
        db.commit()
        return Claim.ACQUIRED
    except UniqueViolation:
        row = db.fetch_one("select * from idempotency_keys where key = %s", (key,))
        if row["request_hash"] != request_hash:
            # Same key, different body: the caller has a bug. Refuse loudly.
            raise Conflict("idempotency key reused with a different payload")
        if row["status"] == "done":
            return Claim.REPLAY(row["response_code"], row["response_body"])
        return Claim.IN_PROGRESS

Three outcomes, and each one deserves a different HTTP status. ACQUIRED means do the work. REPLAY means return the stored response verbatim, with the original status code — a replayed 201 must not become a 200. IN_PROGRESS means a concurrent attempt is running right now, and the correct answer is 409 Conflict with a Retry-After, not a second charge.

That request_hash comparison is the detail that separates a robust implementation from a dangerous one. Without it, a buggy client that reuses a key for a different payload gets the previous response silently, and you have created a data corruption mechanism that looks like a reliability feature.

Expiry is a product decision, not a technical one

Keys cannot live forever. The window should cover the longest plausible retry chain: client-side backoff, a queue redelivery, and a human clicking the button again after lunch. Twenty-four hours covers all of that for most products. Card networks care much less about your key than you do — the money moving twice is the problem, not the row.

Set expires_at when you insert, delete on a schedule, and index the expiry column so the cleanup job is a range scan instead of a table scan. If you skip the index, the cleanup query becomes the next incident.

What it costs

Forty lines, one table, one index, one cron job. Compare that against the cost of a duplicate-charge incident: refunds, a reconciliation script written under pressure, a support conversation you cannot automate, and a customer who now checks their statement.

Reconciliation is work you do after the fact with incomplete information. Idempotency keys are work you do before the fact with complete information. Ship the keys.

I write about production failures in Postgres, queues, and distributed systems.

Subscribe by email · RSS · Bluesky