Idempotency #

In modern systems — especially backend, microservices, and distributed systems — the network can never really be trusted. Requests fail halfway through, clients retry, load balancers try again, message queues redeliver the same event. In this environment, one question becomes critical: what happens if the same operation is executed more than once? If the answer is “could double charge”, “could double order”, or “could corrupt data” — your system isn’t idempotent yet. This article covers idempotency from its conceptual roots, why retries are a certainty rather than an exception, how to implement idempotency keys, and concrete patterns for databases, Redis, and message queues.

What Is Idempotency? #

Idempotency is a property of an operation where running it multiple times with the same input produces the same effect as running it once. It doesn’t mean the operation can’t be repeated — it means repetition doesn’t add new side effects.

In mathematics, this is written as:

f(x) = f(f(x))

In software terms: whether the same request is sent once or a hundred times, the system ends up in the same final state.

The easiest illustration: an elevator button. Pressing the button for floor 5 once or ten times doesn’t make the elevator go to floor 5 ten times. The effect is the same — the elevator goes to floor 5. That’s idempotency.

flowchart LR
    subgraph NonIdempotent["❌ Not Idempotent"]
        A1["1st Request\nPOST /balance/add 100"] --> B1[Balance +100]
        A2["2nd Request\nPOST /balance/add 100"] --> B2[Balance +200]
        A3["3rd Request\nPOST /balance/add 100"] --> B3[Balance +300]
    end
    subgraph Idempotent["✅ Idempotent"]
        C1["1st Request\nPUT /balance/set 100"] --> D1[Balance = 100]
        C2["2nd Request\nPUT /balance/set 100"] --> D2[Balance = 100]
        C3["3rd Request\nPUT /balance/set 100"] --> D3[Balance = 100]
    end

Why Idempotency Matters #

The main reason: the real world isn’t ideal. Engineers often write code as if every request will arrive exactly once, be processed once, and succeed. Reality is very different.

Here are scenarios that happen every day in production systems:

In distributed systems, retry isn’t an edge case — it’s the default behavior. Every HTTP library, every message broker, every orchestrator has built-in retry mechanisms. That means your system will receive duplicate requests. The only question is whether it’s ready for them.

sequenceDiagram
    participant Client
    participant Network
    participant Server
    participant DB

    Client->>Network: POST /payments (Rp 500.000)
    Network->>Server: Request received
    Server->>DB: INSERT payment
    DB-->>Server: OK
    Note over Network: Timeout! Response never arrives
    Server--xClient: Response lost
    Client->>Network: Retry POST /payments (Rp 500.000)
    Network->>Server: Request received again
    Server->>DB: INSERT payment AGAIN
    DB-->>Server: OK
    Server-->>Client: 200 OK
    Note over Client,DB: User pays 2x — without knowing it

Idempotency in HTTP Methods #

HTTP has classified methods by their idempotency semantics since the early RFCs. But this is often misunderstood as a technical guarantee, when it’s actually a semantic contract — the responsibility for the implementation still lies with the engineer.

Idempotency isn’t determined by the HTTP method alone, but by its implementation. A PUT /users/123 that calls UPDATE users SET login_count = login_count + 1 is still not idempotent even though it uses PUT. Idempotency lives in the business logic, not in the method.

A common source of confusion: DELETE /orders/456 sent twice. The first request deletes the order and returns 200 OK. The second gets 404 Not Found. Is this idempotent? Yes — because the system’s final state is the same: order 456 doesn’t exist.

Idempotency vs Deduplication #

These two concepts are often conflated even though they serve different purposes:

flowchart TD
    A[Duplicate Request Arrives] --> B{Deduplication?}
    B -- Yes --> C["Block the request\nbefore it's processed"]
    B -- No --> D{Idempotency?}
    D -- Yes --> E["Process the request\nbut produce the same effect"]
    D -- No --> F["❌ Double Effect\nDouble charge, double order"]
    C --> G[Response: request already exists]
    E --> H[Response: same result as before]

Ideally a system uses both: deduplication as a performance optimization, idempotency as the safety guarantee.

Idempotency Key #

The most common technique for implementing idempotency on non-idempotent operations (like POST) is the Idempotency Key — a unique identifier the client sends with every request.

How It Works #

sequenceDiagram
    participant Client
    participant Server
    participant Store as Key Store\n(DB / Redis)

    Client->>Server: POST /payments\nIdempotency-Key: uuid-abc-123
    Server->>Store: Check key uuid-abc-123
    Store-->>Server: Not found
    Server->>Server: Process payment
    Server->>Store: Save key + result
    Server-->>Client: 200 OK {payment_id: 789}

    Note over Client: Timeout, retry!

    Client->>Server: POST /payments\nIdempotency-Key: uuid-abc-123
    Server->>Store: Check key uuid-abc-123
    Store-->>Server: Found! Result: {payment_id: 789}
    Server-->>Client: 200 OK {payment_id: 789}
    Note over Client,Store: No double charge

The client is responsible for generating a unique key per business operation — usually a UUID v4. The server is responsible for storing the key and its response, and returning the old response if the same key comes in again.

Header Format #

POST /payments HTTP/1.1
Content-Type: application/json
Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000

{
  "amount": 500000,
  "recipient_id": "user-123"
}

Rules for a Good Key #

✓ UUID v4 — random, not predictable
✓ Generated by the client — the server must not generate keys for clients
✓ One key per business operation — not per HTTP call
✗ Timestamp alone — can collide
✗ Auto-increment — predictable, guessable
✗ Combination of business fields without UUID — too easy to collide

Implementing Idempotency #

Database-Based #

The simplest approach with the strongest consistency guarantee: store the idempotency key in a database table with a unique constraint.

-- Create the idempotency store table
CREATE TABLE idempotency_keys (
    key         VARCHAR(255) PRIMARY KEY,
    response    JSONB NOT NULL,
    status_code INT NOT NULL,
    created_at  TIMESTAMP DEFAULT NOW(),
    expires_at  TIMESTAMP
);

-- Index for cleaning up expired keys
CREATE INDEX idx_expires_at ON idempotency_keys(expires_at);

// ANTI-PATTERN: no idempotency check — double charge can happen
func processPayment(req PaymentRequest) (*Payment, error) {
    payment := Payment{
        Amount:      req.Amount,
        RecipientID: req.RecipientID,
    }
    return db.CreatePayment(payment)
}

// CORRECT: idempotency check before processing, save the result after success
func processPayment(req PaymentRequest, idempotencyKey string) (*Payment, error) {
    // Check whether this key has already been processed
    existing, err := db.FindIdempotencyKey(idempotencyKey)
    if err == nil && existing != nil {
        // Key exists — return the previous result without reprocessing
        var payment Payment
        json.Unmarshal(existing.Response, &payment)
        return &payment, nil
    }

    // Process the payment
    payment := Payment{
        Amount:      req.Amount,
        RecipientID: req.RecipientID,
    }
    created, err := db.CreatePayment(payment)
    if err != nil {
        return nil, err
    }

    // Save the key + result for future requests
    responseBytes, _ := json.Marshal(created)
    db.SaveIdempotencyKey(IdempotencyKey{
        Key:       idempotencyKey,
        Response:  responseBytes,
        ExpiresAt: time.Now().Add(24 * time.Hour),
    })

    return created, nil
}

For operations involving money or critical data, use a database transaction that wraps both the business operation and the idempotency key storage together. This prevents the race condition where two requests with the same key arrive at the same time.

Cache-Based (Redis) #

For high-throughput operations, Redis is more efficient than a database because the SET NX (set if not exists) operation is atomic.

// ANTI-PATTERN: plain set — can be overwritten by concurrent requests
redisClient.Set(ctx, key, response, ttl)

// CORRECT: SET NX — atomic, only sets if the key doesn't exist yet
func processWithRedisIdempotency(key string, ttl time.Duration, fn func() (interface{}, error)) (interface{}, error) {
    // Try to fetch an existing result
    cached, err := redisClient.Get(ctx, key).Result()
    if err == nil {
        // Key found, return the cached result
        var result interface{}
        json.Unmarshal([]byte(cached), &result)
        return result, nil
    }

    // Process the operation
    result, err := fn()
    if err != nil {
        return nil, err
    }

    // Save with SET NX so concurrent requests can't overwrite
    resultBytes, _ := json.Marshal(result)
    redisClient.SetNX(ctx, key, string(resultBytes), ttl)

    return result, nil
}

Comparing the database vs Redis approaches:

Message Queue — Idempotent Consumers #

In event-driven architecture, consumers must be idempotent by design because message brokers guarantee at-least-once delivery, not exactly-once.

// ANTI-PATTERN: consumer processes directly without checking for duplicates
func handleOrderEvent(event OrderEvent) error {
    return db.CreateShipment(Shipment{
        OrderID: event.OrderID,
        Address: event.ShippingAddress,
    })
    // If the event is redelivered → duplicate shipment!
}

// CORRECT: check event_id before processing, skip if already processed
func handleOrderEvent(event OrderEvent) error {
    // Check whether this event has already been processed
    alreadyProcessed, err := db.IsEventProcessed(event.EventID)
    if err != nil {
        return err
    }
    if alreadyProcessed {
        // Not an error — this is expected behavior, just skip
        log.Info("event already processed, skipping", "event_id", event.EventID)
        return nil
    }

    // Process inside a transaction: create shipment + mark event as processed
    return db.Transaction(func(tx *DB) error {
        if err := tx.CreateShipment(Shipment{
            OrderID: event.OrderID,
            Address: event.ShippingAddress,
        }); err != nil {
            return err
        }
        return tx.MarkEventProcessed(event.EventID)
    })
}

flowchart TD
    A[Event Arrives from Queue] --> B[Read event_id]
    B --> C{"event_id already\nin processed_events?"}
    C -- Yes --> D[Skip — log and ack]
    C -- No --> E[Process event in a DB transaction]
    E --> F[Insert operation result]
    F --> G[Insert event_id into processed_events]
    G --> H{"Transaction\nsuccessful?"}
    H -- Yes --> I[Ack the queue]
    H -- No --> J[Rollback — event will be redelivered]
    J --> A

Idempotency and Database Transactions #

Idempotency and transactions are two different concepts that complement each other, not replace each other.

The critical case: a database transaction has committed, but the response never reached the client because of a network drop. A transaction can’t help here — the idempotency key is what saves you.

sequenceDiagram
    participant Client
    participant Server
    participant DB

    Client->>Server: POST /transfer (key=X)
    Server->>DB: BEGIN TRANSACTION
    Server->>DB: Debit account A
    Server->>DB: Credit account B
    Server->>DB: COMMIT ✓
    Note over Server,Client: Network drop!
    Server--xClient: Response lost

    Client->>Server: Retry POST /transfer (key=X)
    Server->>DB: Check idempotency_keys WHERE key=X
    DB-->>Server: Found! Transaction already succeeded
    Server-->>Client: 200 OK — transfer already done

Common Implementation Mistakes #

// ✗ Mistake 1: Relying on the client not to retry
// Never assume this — every client retries

// ✗ Mistake 2: Idempotency key without TTL — storage balloons
db.SaveIdempotencyKey(key, response)  // no expiry
// ✓ Always set a sensible TTL (24 hours for most cases)
db.SaveIdempotencyKey(key, response, expiresAt: time.Now().Add(24*time.Hour))

// ✗ Mistake 3: Queue consumer not idempotent
func consume(event Event) {
    db.Insert(...)  // inserts directly without a duplicate check
}
// ✓ Always check event_id before processing
func consume(event Event) {
    if db.IsProcessed(event.ID) { return nil }
    // process...
}

// ✗ Mistake 4: Race condition in the idempotency check
existing := db.Find(key)   // check
if existing == nil {       // gap here! another request can slip in
    db.Save(key, result)   // save — possibly a duplicate
}
// ✓ Use an atomic operation: INSERT ON CONFLICT or SET NX
db.Exec("INSERT INTO idempotency_keys ... ON CONFLICT (key) DO NOTHING")

// ✗ Mistake 5: Idempotency key generated by the server, not the client
func createPayment(req Request) {
    key := uuid.New()  // server-generated — client retries get a different key!
    // ...
}
// ✓ The key must come from the client; the server only receives and validates it
func createPayment(req Request, idempotencyKey string) {
    // ...
}

Idempotency Implementation Checklist #

DESIGN:
  □ All retryable write operations have been identified
  □ Key storage strategy has been decided (DB vs Redis)
  □ TTL for idempotency keys has been set
  □ Key format is documented in the API spec

IMPLEMENTATION:
  □ Idempotency check happens before the business operation
  □ Key + result stored in a single atomic operation
  □ Race conditions handled with INSERT ON CONFLICT or SET NX
  □ Message queue consumers check event_id

ERROR HANDLING:
  □ Expired keys return a clear error
  □ Key with a different payload is rejected (key conflict)
  □ Logging when a duplicate request is detected

TESTING:
  □ Test scenario: first request succeeds
  □ Test scenario: duplicate request with the same key
  □ Test scenario: two concurrent requests with the same key
  □ Test scenario: expired key, client retries with a new key

Summary #

• Idempotency is the safety guarantee for retries — the same operation can be executed many times without additional side effects.
• Retries are a certainty — networks, load balancers, and message queues all retry; your system must be ready to receive them.
• HTTP methods aren’t a guarantee — PUT and DELETE are semantically idempotent, but the implementation is still the engineer’s responsibility.
• Idempotency Key — the most common technique: the client sends a unique UUID per operation, the server stores key + result, duplicate requests get the old result.
• Database for critical data — use INSERT ON CONFLICT DO NOTHING for an atomic idempotency check that’s safe against race conditions.
• Redis for high throughput — SET NX is atomic and very fast, great for high-volume APIs.
• Queue consumers must be idempotent — store processed event_ids and skip if found again.
• Idempotency complements transactions — transactions guarantee atomicity within one operation, idempotency guarantees safety across requests.
• TTL is mandatory — always set an expiry on idempotency keys so storage doesn’t grow without bound.

← Previous: Clean Code   Next: Race Condition →