Broken Pipe #

One morning, an engineer sees thousands of error lines in production logs: BrokenPipeError: [Errno 32] Broken pipe. Nothing crashed, the application keeps running, but the error feels alarming — something must be wrong. After investigation, the cause turns out to be simple: a monitoring script reading another process’s output with | head -5 — taking the first 5 lines then stopping — caused an error in the sending process because the reader had already closed the pipe.

Broken pipe is one of the most common errors on Unix/Linux but is often misunderstood. It isn’t always a bug sign — sometimes it’s expected behavior. Understanding it correctly helps engineers distinguish what needs handling, what can be ignored, and how to prevent this error from leaking into logs that should be clean.

What a Pipe Is and Why It Can Be “Broken” #

A pipe is an inter-process communication mechanism in Unix — a way to connect one process’s output to another process’s input. When you write ls | grep txt, the shell creates a pipe: ls’s output flows into grep’s input.

Pipe visualization:

flowchart TD
    subgraph Normal["Normal Condition"]
        direction LR
        A1["Process A (Writer)<br>write('data')"] --> P1["Pipe Buffer"]
        P1 --> B1["Process B (Reader)<br>read()"]
    end

    subgraph Broken["Broken Pipe"]
        direction LR
        A2["Process A (Writer)<br>write('data')<br>(← SIGPIPE/EPIPE)"] --> P2["Pipe (closed)"]
        P2 -. "no reader" .-> B2["Process B (Reader)<br>(already exited)"]
    end

A writer trying to write to a pipe with no readers left:

  • The kernel sends the SIGPIPE signal to the writer.
  • Or write() returns -1 with errno = EPIPE.

Broken pipes can happen in three main contexts:

Context 1: Shell pipes
  $ long-running-command | head -5
  head reads 5 lines → closes the pipe → long-running-command gets SIGPIPE
  → Normal and expected

Context 2: Network sockets
  An HTTP client sends a request → the server starts sending a response
  The client closes the connection before the response finishes
  → The server gets a "broken pipe" when trying to write to the socket
  → Also normal — clients may disconnect anytime

Context 3: File descriptors
  Process A writes to a file descriptor
  Process B, which should read it, has crashed or closed its fd
  → Process A gets EPIPE

SIGPIPE vs EPIPE: Two Ways the System Notifies #

SIGPIPE (signal):
  → Sent to processes by the kernel when writing to pipes/sockets
    with no readers
  → Default behavior: TERMINATE the process (dies immediately)
  → Can be ignored: signal(SIGPIPE, SIG_IGN)
  → Can be handled: signal(SIGPIPE, my_handler)

EPIPE (errno):
  → The error code returned by write()/send() calls
  → Happens when SIGPIPE is ignored and write() fails
  → Or on non-blocking sockets
  → Must be explicitly handled in code

The sequence of events:
  1. A process tries write() to a broken pipe/socket
  2. The kernel sends SIGPIPE to the process
  3. If SIGPIPE is unhandled → the process terminates
  4. If SIGPIPE is ignored → write() returns -1 with errno = EPIPE
  5. Applications must check errno and handle the error

Handling Across Languages and Contexts #

Python #

package main

import (
	"errors"
	"fmt"
	"log"
	"os"
	"os/signal"
	"syscall"
)

// Approach 1: Ignore SIGPIPE (common for CLI tools)
// Prevents process termination when the pipe is closed by the reader
func ignoreSigpipe() {
	signal.Ignore(syscall.SIGPIPE)
}

// Approach 2: Handle broken pipes explicitly
func safeWrite(data string) {
	_, err := fmt.Println(data)
	if err != nil {
		if errors.Is(err, syscall.EPIPE) {
			// The reader already closed the pipe — this is OK for CLI tools
			// Exit with the appropriate code
			os.Exit(0)
		}
		log.Fatalf("write error: %v", err)
	}
}

// Approach 3: For scripts with large piped output
// This is the most robust pattern for CLI tools
func main() {
	for item := range generateLargeOutput() {
		fmt.Println(item)
	}
}
// Handling client disconnects when sending a response
func sendResponse(conn net.Conn, data []byte) {
	_, err := conn.Write(data)
	if err != nil {
		if errors.Is(err, syscall.EPIPE) ||
			errors.Is(err, syscall.ECONNRESET) ||
			errors.Is(err, syscall.ECONNABORTED) {
			// The client closed the connection before the response finished sending
			// This is normal — log at DEBUG level, not ERROR
			log.Printf("DEBUG: client disconnected before response completed: %v", err)
			return
		}
		// Unexpected socket errors — log as ERROR
		log.Printf("ERROR: socket error: %v", err)
	}
}

Go #

package main

import (
    "errors"
    "io"
    "net"
    "syscall"
    "log"
    "os"
)

// Check whether an error is a broken pipe
func isBrokenPipe(err error) bool {
    if err == nil {
        return false
    }
    // Check the various broken pipe error forms in Go
    if errors.Is(err, syscall.EPIPE) {
        return true
    }
    if errors.Is(err, io.ErrClosedPipe) {
        return true
    }
    // For network errors
    var netErr *net.OpError
    if errors.As(err, &netErr) {
        if errors.Is(netErr.Err, syscall.EPIPE) {
            return true
        }
    }
    return false
}

// An HTTP handler handling client disconnects
func handleRequest(w http.ResponseWriter, r *http.Request) {
    data := generateLargeResponse()

    _, err := w.Write(data)
    if err != nil {
        if isBrokenPipe(err) {
            // Client disconnect — log debug only
            log.Printf("DEBUG: client disconnected: %v", err)
            return
        }
        // Other errors worth attention
        log.Printf("ERROR: write failed: %v", err)
    }
}

// Writing to stdout with broken pipe handling
// (for CLI tools whose output gets piped)
func writeToStdout(lines []string) {
    for _, line := range lines {
        _, err := fmt.Println(line)
        if err != nil {
            if isBrokenPipe(err) {
                // The reader closed the pipe — exit normally
                os.Exit(0)
            }
            log.Fatalf("Write error: %v", err)
        }
    }
}

Node.js #

// Node.js-style broken pipe handling in Go

// CLI tools — writing to stdout that gets piped
func writeToStdout(lines []string) {
	for _, line := range lines {
		_, err := fmt.Println(line)
		if err != nil {
			if errors.Is(err, syscall.EPIPE) {
				// Output piped to a process that already finished (e.g. | head -5)
				// Exit with code 0 — this is expected behavior
				os.Exit(0)
			}
			// Other errors — propagate
			log.Fatalf("write error: %v", err)
		}
	}
}

// HTTP server — client disconnects while streaming responses
func streamLargeData(w http.ResponseWriter, r *http.Request) {
	stream := generateLargeDataStream()

	// The client closed the connection — stop generating data
	ctx := r.Context()
	go func() {
		<-ctx.Done()
		stream.Close()
		log.Printf("DEBUG: client disconnected, stream destroyed")
	}()

	if err := stream.WriteTo(w); err != nil {
		if errors.Is(err, syscall.EPIPE) || errors.Is(err, syscall.ECONNRESET) {
			// The client closed the connection — expected for streaming
			log.Printf("DEBUG: client disconnected: %v", err)
			return
		}
		// Other errors — propagate
		log.Printf("ERROR: stream error: %v", err)
	}
}

// HTTP server — handling writes after client disconnects
func handleRequest(w http.ResponseWriter, r *http.Request) {
	// Go surfaces broken pipes as write errors — no pre-check of the socket needed
	_, err := w.Write(generateResponse())
	if err != nil {
		if errors.Is(err, syscall.EPIPE) || errors.Is(err, syscall.ECONNRESET) {
			log.Printf("DEBUG: client disconnected: %v", err)
		} else {
			log.Printf("ERROR: response error: %v", err)
		}
	}
}

Broken Pipes in Web Servers: Client Disconnects #

Broken pipes most often appear in web servers because of client disconnects. This is very normal and usually doesn’t indicate a problem:

Common client-disconnect scenarios causing broken pipes:

  1. Users closing browser tabs before pages finish loading
     → The server tries to send a response → the client socket is closed → EPIPE

  2. Mobile users losing signal mid-request
     → TCP connections break → servers get ECONNRESET or EPIPE

  3. Load balancer timeouts before servers finish generating responses
     → Load balancers close connections → servers get EPIPE

  4. Impatient clients (client-side timeouts)
     → Client timeouts → servers get EPIPE

  5. API consumers with incorrect implementations
     → Clients send requests but immediately close connections

  All of these scenarios are NORMAL and don't indicate bugs.
  The important thing: log at DEBUG level, not ERROR.
  Don't alert on-call engineers for broken pipes from client disconnects.
package main

import (
	"io"
	"log"
	"os"
	"strings"
)

// Filtering broken pipes from the app server error log.
// Go has no WSGI server like Gunicorn; this is the equivalent
// of a logging filter: drop messages mentioning "broken pipe".
type brokenPipeFilter struct {
	inner io.Writer
}

func (f brokenPipeFilter) Write(p []byte) (int, error) {
	if strings.Contains(strings.ToLower(string(p)), "broken pipe") {
		return len(p), nil // silently drop
	}
	return f.inner.Write(p)
}

func main() {
	// Route all log output through the filter
	log.SetOutput(brokenPipeFilter{inner: os.Stderr})
}

Graceful Shutdowns and Broken Pipes #

When applications receive shutdown signals (SIGTERM), they need to finish in-flight requests before closing. If not done correctly, active clients will get broken pipes.

package main

import (
	"log"
	"os"
	"os/signal"
	"sync/atomic"
	"syscall"
	"time"
)

type GracefulServer struct {
	shutdown       chan struct{}
	inShutdown     atomic.Bool
	activeRequests atomic.Int64
}

// Handle SIGTERM for graceful shutdowns
func NewGracefulServer() *GracefulServer {
	s := &GracefulServer{shutdown: make(chan struct{})}

	ch := make(chan os.Signal, 1)
	signal.Notify(ch, syscall.SIGTERM, syscall.SIGINT)
	go func() {
		<-ch
		s.startShutdown()
	}()

	return s
}

func (s *GracefulServer) startShutdown() {
	// Start a graceful shutdown when receiving SIGTERM
	log.Println("Received shutdown signal, starting graceful shutdown...")
	s.inShutdown.Store(true)
	close(s.shutdown)
}

// Handle one request with tracking for graceful shutdowns
func (s *GracefulServer) handleRequest(request Request) Response {
	if s.inShutdown.Load() {
		// Reject new requests during shutdown
		return Response{Status: 503, Body: "Service shutting down"}
	}
	s.activeRequests.Add(1)
	defer s.activeRequests.Add(-1)

	return processRequest(request)
}

// Wait for all requests to finish before exiting.
// Prevents broken pipes to active clients.
func (s *GracefulServer) waitForShutdown(timeout time.Duration) {
	<-s.shutdown // wait for the shutdown signal

	deadline := time.Now().Add(timeout)
	for time.Now().Before(deadline) {
		if s.activeRequests.Load() == 0 {
			break
		}
		log.Printf("Waiting for %d active requests...", s.activeRequests.Load())
		time.Sleep(500 * time.Millisecond)
	}

	if s.activeRequests.Load() > 0 {
		log.Printf("Shutdown timeout: %d requests still active", s.activeRequests.Load())
	}
	log.Println("Server shutdown complete")
}
# Kubernetes deployments — graceful shutdown configuration

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      # Give time for graceful shutdowns before pods are killed
      terminationGracePeriodSeconds: 60  # default 30 seconds

      containers:
        - name: app
          lifecycle:
            preStop:
              exec:
                # Wait 10 seconds before the container starts shutting down
                # Gives the load balancer time to remove the pod from rotation
                command: ["/bin/sleep", "10"]

Debugging Broken Pipes #

# Detecting and investigating broken pipes

# 1. Check logs for broken pipes
grep -i "broken pipe\|EPIPE\|SIGPIPE" /var/log/app/error.log

# 2. Check whether a process receives SIGPIPE
strace -e signal -p <PID> 2>&1 | grep SIGPIPE

# 3. Monitor a process's file descriptors
lsof -p <PID>   # view all open fds
# Check whether any pipes have an end that no longer exists

# 4. Check network connection states
ss -tp          # TCP connections with associated processes
# Pay attention to the CLOSE_WAIT state — can be a broken pipe source

# 5. Monitor in real-time with SystemTap or eBPF
# (advanced — requires kernel support)
# bpftrace -e 'kprobe:pipe_write { ... }'

# 6. Reproduce broken pipes controllably for testing
# Terminal 1: run a writer
python3 -c "
import time, sys
for i in range(100):
    print(f'line {i}')
    time.sleep(0.1)
" | head -3

# Terminal 2: observe the behavior
# head -3 reads 3 lines then exits
# The Python script gets SIGPIPE/BrokenPipeError

Broken Pipes vs Similar Errors #

EPIPE vs ECONNRESET vs ECONNABORTED:

  EPIPE (Broken pipe):
  → Happens when WRITING to closed pipes/sockets
  → The reader already closed the pipe end
  → Common in: servers writing responses to already-disconnected clients

  ECONNRESET (Connection reset by peer):
  → Happens when TCP connections are reset by remotes
  → Remotes send RST packets
  → Common in: client crashes, firewalls cutting connections, NAT timeouts

  ECONNABORTED (Connection aborted):
  → Connections aborted by local stacks
  → Often happens when accept() is called on already-RST'd connections

  ETIMEDOUT (Connection timed out):
  → No remote response within the specified time
  → Can indicate network problems or slow servers

  All of these can appear together:
  → Client disconnects → ECONNRESET from the TCP perspective
  → The server tries to write → EPIPE/SIGPIPE
  → Both come from the same event but look different in code

Anti-Patterns to Avoid #

package main

import (
	"errors"
	"fmt"
	"log"
	"os"
	"os/signal"
	"syscall"
)

// ✗ Anti-pattern 1: logging broken pipes as ERROR
// Creates thousands of false positives in logs and unnecessary alerts
func antiPattern1(err error) {
	log.Printf("ERROR: Broken pipe error: %v", err) // DON'T for client disconnects
}

// ✓ Solution: log as DEBUG for expected client disconnects
func solution1(err error) {
	log.Printf("DEBUG: Client disconnected (EPIPE) - normal behavior: %v", err)
}

// ✗ Anti-pattern 2: not handling SIGPIPE, letting processes crash
// CLI tools without signal handling crash when output is piped to | head
func generateReport() {
	for _, line := range hugeDataset {
		_, err := fmt.Println(line)
		if err != nil {
			log.Fatal(err) // crashes with SIGPIPE if | head -10 is used
		}
	}
}

// ✓ Solution: handle broken pipes in CLI tools
func generateReportFixed() {
	for _, line := range hugeDataset {
		_, err := fmt.Println(line)
		if err != nil {
			if errors.Is(err, syscall.EPIPE) {
				os.Exit(0)
			}
			log.Fatal(err)
		}
	}
}

// ✗ Anti-pattern 3: globally ignoring SIGPIPE without consideration
// signal.Ignore(syscall.SIGPIPE) in all code
// Can keep processes running endlessly when nobody is reading
// and writing loops without stop into /dev/null

// ✓ Solution: explicitly handle EPIPE after SIG_IGN
func ignoreSigpipe() {
	signal.Ignore(syscall.SIGPIPE)
	// Now you MUST check write() return values and errno
}

// ✗ Anti-pattern 4: no graceful shutdowns
// Pods killed immediately → all active requests get broken pipes
// ✓ Solution: terminationGracePeriodSeconds + preStop hooks in Kubernetes

// ✗ Anti-pattern 5: alerting on-call for every broken pipe
// Broken pipes from client disconnects can number thousands per day in healthy apps
// ✓ Solution: distinguish "broken pipes from client disconnects" (normal)
//   from "broken pipes from internal bugs" (needs alerts)

Broken Pipe Handling Checklist #

CLI TOOLS:
  □ BrokenPipeError handled for tools whose output can be piped
  □ Exit with code 0 on broken pipes (not errors)
  □ stderr closed before exiting to avoid secondary errors

WEB SERVERS:
  □ Client disconnects (EPIPE/ECONNRESET) logged as DEBUG, not ERROR
  □ No on-call alerts for broken pipes from client disconnects
  □ Response streaming stopped on client disconnects (don't keep generating)

GRACEFUL SHUTDOWNS:
  □ SIGTERM handlers exist and work
  □ Active requests finish before processes exit
  □ Kubernetes terminationGracePeriodSeconds configured per needs
  □ preStop hooks give load balancers time to remove pods

NETWORK PROGRAMMING:
  □ write()/send() return values always checked
  □ EPIPE handled differently from other errors (no re-raises, no error logs)
  □ Socket error handlers exist for all maintained connections

DEBUGGING:
  □ Logs contain enough context to distinguish normal broken pipes from bugs
  □ Monitoring distinguishes "client disconnects" from "internal errors"
  □ No noise in error logs from expected broken pipes

Summary #

  • Broken pipes happen when writers try writing to pipes or sockets already closed by readers — the kernel sends SIGPIPE to writers, or write() returns EPIPE if SIGPIPE is ignored.
  • Broken pipes in web servers from client disconnects are NORMAL — users closing tabs, mobile users losing signals, load balancer timeouts — all cause broken pipes that don’t indicate bugs.
  • Log broken pipes from client disconnects as DEBUG, not ERROR — thousands of broken pipes per day in healthy applications are common. Logging as ERROR only creates confusing noise.
  • CLI tools with piped output must handle BrokenPipeError — when | head -5 or | less closes pipes, tools must exit with code 0, not crash or print errors.
  • SIGPIPE’s default behavior is process termination — make sure CLI tools and server daemons needing to survive broken pipes explicitly handle or ignore SIGPIPE.
  • Graceful shutdowns prevent broken pipes to active clients — when receiving SIGTERM, wait for all requests to finish before closing connections. Kubernetes terminationGracePeriodSeconds controls the waiting duration.
  • Distinguish EPIPE, ECONNRESET, and ECONNABORTED — all signal problematic connections but from different perspectives. All three usually come from the same event (client disconnects).
  • Don’t alert on-call for every broken pipe — build alerting that distinguishes expected broken pipes (client disconnects) from unexpected ones (internal bugs).
  • After ignoring SIGPIPE, always check errno after write() — if SIGPIPE is ignored, failing write() calls return -1 with errno EPIPE. Code not checking this will keep running as if writes succeeded.
  • Streaming responses must stop on client disconnects — don’t keep generating data when nobody is reading. This wastes CPU and can pile up data in buffers that never get sent.
#

← Previous: Micro Frontend   Next: Circuit Breaker

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact