Async Processing #

In modern software — from web applications and mobile backends to distributed systems — async processing is no longer just a performance optimization, but an architectural requirement. The biggest mistake many teams make is treating async as something to add later when the system starts slowing down. But large systems don’t become async — they’re designed async from the start. This article covers async processing from its roots: why the synchronous model has limits that can’t be broken with bigger hardware, three forms of async with different characteristics, four foundational principles to understand, and how to decide precisely when async is the solution and when it becomes a new source of problems.

What Is Async Processing? #

Async processing (asynchronous processing) is a processing pattern where a task is not executed in a way that blocks the main execution flow of the program. A request doesn’t have to wait for a process to finish before continuing to the next execution — the result can be returned later, by another worker, in another process, or even on another machine.

The fundamental difference between synchronous and asynchronous:

flowchart TD
    subgraph Sync["Synchronous — Blocking"]
        S1[Request Arrives] --> S2["Process A\nvalidation"]
        S2 --> S3["Process B\nsave to DB"]
        S3 --> S4["Process C\nsend email\n⏳ wait for SMTP"]
        S4 --> S5["Process D\ngenerate PDF\n⏳ wait for render"]
        S5 --> S6["Response to User\n~3000ms"]
    end
    subgraph Async["Asynchronous — Non-Blocking"]
        A1[Request Arrives] --> A2["Process A\nvalidation"]
        A2 --> A3["Process B\nsave to DB"]
        A3 --> A4["Enqueue: send email\nEnqueue: generate PDF"]
        A4 --> A5["Response to User\n~80ms"]
        A4 -.->|background| A6[Worker: send email]
        A4 -.->|background| A7[Worker: generate PDF]
    end

With async processing, the user gets a response in milliseconds, while heavy processes are completed in the background without blocking other requests.

Why the Synchronous Model Has Limits #

Async processing was born from the real limitations of synchronous systems that can’t be solved by simply adding CPU or memory.

Thread Blocking Is the Root of the Problem #

In the traditional synchronous model, one request locks one thread for the duration of the process. That thread sits idle while waiting for I/O — database queries, HTTP calls to external APIs, reading files from disk — but still consumes memory and a slot in the thread pool.

sequenceDiagram
    participant User
    participant Thread as Thread Pool\n(20 threads)
    participant DB
    participant SMTP

    User->>Thread: Registration request (Thread 1 allocated)
    Thread->>DB: INSERT user (50ms wait)
    DB-->>Thread: OK
    Thread->>SMTP: Send email (800ms wait!)
    Note over Thread: Thread 1 idle for 800ms\nCan't serve other requests
    SMTP-->>Thread: OK
    Thread-->>User: Response (850ms total)
    Note over Thread: With 20 threads and 800ms/request\ncan only handle ~25 requests/second

Under high traffic, the thread pool runs out, new requests queue up, latency multiplies, and eventually the system collapses — not because of a lack of CPU, but because threads sit idle waiting for slow I/O.

Concrete Impact in Production Systems #

Four Foundational Principles of Async Processing #

1. Decoupling #

The caller doesn’t need to know how or when the task finishes. This separates responsibility between the one requesting the work and the one doing it.

// ANTI-PATTERN: tight coupling — the caller must wait until the email is sent
func registerUser(req RegistrationRequest) error {
    user := createUser(req)
    db.Save(user)
    
    // The request must wait for the email to finish — unnecessary coupling
    err := emailService.SendWelcomeEmail(user.Email)
    if err != nil {
        // Do we roll back the user just because the email failed?
        return err
    }
    return nil
}

// CORRECT: decoupled — registration and email are separate concerns
func registerUser(req RegistrationRequest) error {
    user := createUser(req)
    db.Save(user)
    
    // Publish an event — don't care when or how the email is sent
    eventBus.Publish(UserRegisteredEvent{
        UserID: user.ID,
        Email:  user.Email,
    })
    return nil // immediate response, no waiting for the email
}

// The email is sent by a separate handler, can retry, can scale independently
func (h *EmailHandler) HandleUserRegistered(event UserRegisteredEvent) {
    emailService.SendWelcomeEmail(event.Email)
}

2. Non-Blocking Execution #

The main thread is free to serve other requests while processes run in the background. This is the key to horizontal scalability.

flowchart LR
    subgraph Blocking["Blocking"]
        T1["Thread 1\nRequest A"] --> W1["⏳ Wait for DB\n200ms"]
        T2["Thread 2\nRequest B"] --> W2["⏳ Wait for DB\n200ms"]
        T3["Thread 3\nRequest C"] --> W3["⏳ Wait for DB\n200ms"]
        T4["Thread 4\nIdle — no threads left!"]
    end
    subgraph NonBlocking["Non-Blocking"]
        EL["Event Loop\nOne Thread"] --> R1["Request A\nregister callback"]
        EL --> R2["Request B\nregister callback"]
        EL --> R3["Request C\nregister callback"]
        EL --> R4[Request D]
        EL --> R5[Request E...]
        DB["(DB)"] -.->|result A arrives| EL
        DB -.->|result B arrives| EL
    end

3. Eventual Completion #

The result isn’t available instantly — it will finish “at some point” in the future. This requires a shift in how you think about data consistency: not strong consistency (always up to date), but eventual consistency (consistent eventually).

// ANTI-PATTERN: assuming an async task is done when the response is sent
func uploadFile(file File) UploadResponse {
    jobID := queue.Enqueue(ResizeImageJob{File: file})
    
    return UploadResponse{
        JobID:    jobID,
        ThumbURL: "/thumbs/" + file.ID + ".jpg", // might not exist yet!
        Status:   "completed", // WRONG — this is still pending
    }
}

// CORRECT: express honest state, provide a way to poll or a webhook
func uploadFile(file File) UploadResponse {
    jobID := queue.Enqueue(ResizeImageJob{File: file})
    
    return UploadResponse{
        JobID:      jobID,
        Status:     "processing",             // honest about the current state
        StatusURL:  "/jobs/" + jobID,         // client can poll the status
        WebhookURL: "/jobs/" + jobID + "/webhook", // or wait for a notification
    }
}

4. State Awareness #

Because the process runs separately from the request, state must be stored explicitly. If a worker crashes, the state must not disappear with it.

// ANTI-PATTERN: state only in memory — lost on crash
var pendingJobs = make(map[string]Job) // not persistent!

func enqueueJob(job Job) string {
    id := uuid.New().String()
    pendingJobs[id] = job // crash = all jobs lost
    go processJob(job)
    return id
}

// CORRECT: state stored in persistent storage before processing
func enqueueJob(job Job) (string, error) {
    // Save to the DB first — before the queue, before processing
    id := uuid.New().String()
    if err := db.Create(&JobRecord{
        ID:        id,
        Payload:   mustMarshal(job),
        Status:    "pending",
        CreatedAt: time.Now(),
    }); err != nil {
        return "", err
    }
    
    // Only then publish to the queue — if the queue fails, the job still exists in the DB
    queue.Publish(JobMessage{ID: id})
    return id, nil
}

Three Forms of Async Processing #

Async processing doesn’t come in just one form. Each form has different characteristics, strengths, and limits.

Form 1: Async at the Code Level #

Async tasks run in the same process using goroutines, coroutines, async/await, or futures/promises. This is the lightest form of async — suitable for fast I/O-bound tasks.

// ANTI-PATTERN: sequential — total latency = A + B + C
func getDashboardData(userID string) DashboardData {
    profile  := fetchProfile(userID)   // 100ms
    orders   := fetchOrders(userID)    // 150ms
    notifs   := fetchNotifs(userID)    // 80ms
    // Total: 330ms — even though it could be 150ms
    return buildDashboard(profile, orders, notifs)
}

// CORRECT: concurrent goroutines — total latency = max(A, B, C)
func getDashboardData(ctx context.Context, userID string) DashboardData {
    type result struct {
        profile Profile
        orders  []Order
        notifs  []Notification
        err     error
    }

    profileCh := make(chan result, 1)
    ordersCh  := make(chan result, 1)
    notifsCh  := make(chan result, 1)

    go func() {
        p, err := fetchProfile(ctx, userID)
        profileCh <- result{profile: p, err: err}
    }()
    go func() {
        o, err := fetchOrders(ctx, userID)
        ordersCh <- result{orders: o, err: err}
    }()
    go func() {
        n, err := fetchNotifs(ctx, userID)
        notifsCh <- result{notifs: n, err: err}
    }()

    // Collect all results — total ~150ms not 330ms
    pr := <-profileCh
    or := <-ordersCh
    nr := <-notifsCh

    return buildDashboard(pr.profile, or.orders, nr.notifs)
}

Characteristics:

Form 2: Background Worker-Based Async #

Tasks are sent to a queue and processed by a worker running separately — possibly a different process, a different server, or even a cloud function. This is the most common form of async for heavy business operations.

flowchart LR
    A[API Handler] --> B["Validate\nRequest"]
    B --> C["Save to DB\nstatus: pending"]
    C --> D[Publish to Queue]
    D --> E["Response 202\nAccepted"]
    D -.->|async| F["Worker 1\nprocess job"]
    D -.->|async| G["Worker 2\nprocess job"]
    D -.->|async| H["Worker 3\nprocess job"]
    F --> I["Update DB\nstatus: completed"]
    G --> I
    H --> I
    I -.-> J["Webhook / Notif\nto client"]

// ANTI-PATTERN: generating a report inside the request — the user waits for minutes
func generateReport(w http.ResponseWriter, r *http.Request) {
    report := heavyReportGeneration(r.Context(), getParams(r)) // could take 2 minutes!
    w.Write(report.Bytes()) // HTTP timeout before it finishes
}

// CORRECT: enqueue the job, return a job ID, client polls or waits for a webhook
func generateReport(w http.ResponseWriter, r *http.Request) {
    params := getParams(r)
    
    jobID, err := jobQueue.Enqueue(ReportJob{
        UserID: currentUser(r).ID,
        Params: params,
    })
    if err != nil {
        http.Error(w, "failed to queue job", 500)
        return
    }
    
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusAccepted) // 202, not 200
    json.NewEncoder(w).Encode(map[string]string{
        "job_id":     jobID,
        "status":     "processing",
        "status_url": "/reports/jobs/" + jobID,
    })
}

// Separate worker — can scale independently of the API
func (w *ReportWorker) Process(job ReportJob) error {
    report, err := heavyReportGeneration(context.Background(), job.Params)
    if err != nil {
        return err // the queue will retry
    }
    
    storage.Save(job.ID, report)
    db.UpdateJobStatus(job.ID, "completed")
    notifyUser(job.UserID, job.ID)
    return nil
}

Characteristics:

Form 3: Event-Driven Async Processing #

The system publishes events when something happens, and anyone interested can react — without the publisher knowing who its consumers are. This is the most loosely coupled form of async.

flowchart LR
    P["Order Service\nPublish: OrderPaid"]
    P --> Q[("(Event Bus\nKafka / Pub/Sub)")]
    Q --> C1["Invoice Service\ncreate invoice"]
    Q --> C2["Notification Service\nsend email + push"]
    Q --> C3["Loyalty Service\nadd points"]
    Q --> C4["Analytics Service\nlog to warehouse"]

// ANTI-PATTERN: the publisher must know and call all downstream services directly
func processPayment(orderID string) error {
    order := db.FindOrder(orderID)
    markOrderAsPaid(order)
    
    // Tight coupling — if one downstream fails, roll everything back?
    invoiceService.CreateInvoice(order)    // coupling!
    emailService.SendReceipt(order)        // coupling!
    loyaltyService.AddPoints(order)        // coupling!
    analyticsService.LogPayment(order)     // coupling!
    return nil
}

// CORRECT: publish one event, consumers react independently
func processPayment(orderID string) error {
    order := db.FindOrder(orderID)
    markOrderAsPaid(order)
    
    // One event, many consumers — the publisher doesn't know who's listening
    eventBus.Publish("order.paid", OrderPaidEvent{
        OrderID:   order.ID,
        UserID:    order.UserID,
        Amount:    order.TotalAmount,
        PaidAt:    time.Now(),
    })
    return nil
}

// Each service has its own consumer, independent, can fail without affecting others
func (s *InvoiceService) OnOrderPaid(event OrderPaidEvent) error {
    return s.createInvoice(event.OrderID)
}

func (s *NotificationService) OnOrderPaid(event OrderPaidEvent) error {
    return s.sendReceiptEmail(event.UserID, event.OrderID)
}

Characteristics:

Async in Modern Architectures #

Async processing is the foundation of almost every large-scale system architecture.

flowchart TD
    subgraph Microservices["Microservices"]
        MS1[Order Service] -->|async event| MS2[Payment Service]
        MS2 -->|async event| MS3[Notification Service]
        MS1 -->|async event| MS4[Inventory Service]
    end
    subgraph Serverless["Serverless"]
        TR["Trigger\nS3 / Queue / HTTP"] -->|event| FN["Function\nruns on-demand"]
    end
    subgraph HighTraffic["High-Traffic System"]
        CLI[Client] --> LB[Load Balancer]
        LB --> API[API Server]
        API --> Q[("(Queue\nBuffer)")]
        Q --> W1[Worker Pool]
        Q --> W2[Worker Pool]
    end

Positive Impacts and Challenges #

Positive Impacts #

flowchart LR
    ASYNC[Async Processing] --> SC["Scalability\nWorkers scale independently"]
    ASYNC --> RS["Resilience\nIsolated failures, can retry"]
    ASYNC --> PF["Performance\nLower user latency"]
    ASYNC --> UX["Better UX\nNo UI freeze"]
    ASYNC --> DC["Decoupling\nServices independent of each other"]

Challenges to Anticipate #

Common Engineer Mistakes #

// ✗ Mistake 1: Assuming async = parallel
// A goroutine that doesn't truly run concurrently isn't parallel
go func() {
    result = heavyCompute() // still CPU-bound, not faster
}()

// ✗ Mistake 2: Async for everything — including things that don't need it
eventBus.Publish(UserNameUpdatedEvent{...})
// Username validation doesn't need async — the result must be instant

// ✗ Mistake 3: Not storing state before enqueuing
queue.Publish(job) // if the queue crashes, the job is lost
db.Save(job)       // too late — it should be DB first, queue second

// ✓ The correct order: DB first, queue second
db.Save(&JobRecord{ID: jobID, Status: "pending"})
queue.Publish(JobMessage{ID: jobID})

// ✗ Mistake 4: Non-idempotent consumer — a message can be received 2x
func handleJob(job Job) {
    db.Create(job.Result) // duplicate if the message is redelivered!
}
// ✓ Check before processing
func handleJob(job Job) {
    if db.Exists(job.ID) { return nil }
    db.Create(job.Result)
}

// ✗ Mistake 5: No visibility into job status
queue.Publish(job) // the user doesn't know whether it succeeded or not
// ✓ Store a queryable job status
db.Save(&Job{ID: jobID, Status: "pending"})
// expose GET /jobs/{id} for polling

When to Use Async Processing #

Use Async If: #

Don’t Use Async If: #

Async processing moves complexity from inside the request to outside the request. If your system doesn’t yet have job monitoring, retry mechanisms, a DLQ, and good distributed tracing — async just moves the problem somewhere harder to see.

Async Processing Implementation Checklist #

DESIGN:
  □ Identify operations that don't need to finish before the response
  □ Decide the async form: code-level, background worker, or event-driven
  □ Define the contract between publisher and consumer (event schema)
  □ Set the SLA completion time — when is a job considered too slow?

IMPLEMENTATION:
  □ Store job state in persistent storage before enqueuing to the queue
  □ Idempotent consumers — check job_id / event_id before processing
  □ Error handling: retry policy + DLQ for jobs that keep failing
  □ Expose a job status endpoint (/jobs/{id}) for client polling

OBSERVABILITY:
  □ Correlation ID propagated from the initial request through the async chain
  □ Structured logging at every stage: enqueue, dequeue, complete, fail
  □ Metrics: job throughput, queue depth, consumer lag, error rate
  □ Alerts if queue depth is too high or consumer lag grows

TESTING:
  □ Unit tests for consumers with directly injected events
  □ Integration tests with a real queue (or test containers)
  □ Test scenarios: job fails → retry → DLQ
  □ Test scenarios: duplicate message → idempotency skip
  □ Load tests: validate the queue isn't a bottleneck during traffic spikes

Summary #

• Async Processing is an architectural decision, not an optimization — large systems don’t become async, they’re designed async from the start.
• Four foundational principles: decoupling (the caller doesn’t care when it finishes), non-blocking (threads free for other requests), eventual completion (results available in the future), state awareness (state must persist so it isn’t lost).
• Three forms of async: code-level (goroutines/coroutines, within one process), background workers (queue + separate workers, long-running), event-driven (publish-subscribe, loosely coupled multi-consumer).
• Decoupling is the biggest benefit — the publisher doesn’t need to know who consumes its events; every service can grow and fail independently.
• Idempotency is a hard requirement — message brokers guarantee at-least-once delivery; consumers must be ready to receive the same event more than once.
• Save state to the DB before enqueuing to the queue — this ordering is crucial so jobs aren’t lost if the queue or worker crashes.
• Eventual consistency must be communicated to users — don’t return “completed” status when the task is still “processing”; use 202 Accepted and provide a polling endpoint.
• Observability must exist before async — correlation IDs, job status tracking, and queue depth monitoring are prerequisites, not nice-to-haves.
• Don’t make everything async — critical validation, atomic transactions, and simple CRUD are better kept synchronous.

← Previous: Reactive Programming   Next: Event-Driven →