Input Validation #

Every piece of data entering the system from outside — from forms, URL parameters, API request bodies, HTTP headers, uploaded files — must be considered unsafe until proven otherwise. This isn’t about paranoia; it’s about the reality that systems can’t control what clients send.

Well-behaved browsers send valid data. Attacker-controlled browsers, tools like Burp Suite or curl, or automated scripts can send anything — SQL injection payloads, XSS scripts, files disguised as images, sizes beyond expectations, or data types different from what’s expected. Systems that don’t strictly validate input are systems that blindly trust all of this.

Input validation isn’t only about security. It’s about system integrity: ensuring the data entering the database, processed by business logic, and returned to other users is valid, consistent, and won’t break the system.

Why Server-Side Validation Is Mandatory #

Client-side validation (JavaScript) provides good UX — instant feedback when users fill out forms. But it can’t be trusted for security.

Why client-side validation isn't enough:

  1. It can be bypassed entirely:
     curl -X POST https://api.example.com/users \
       -H "Content-Type: application/json" \
       -d '{"email": "not-an-email", "age": -999}'
     → No JavaScript runs, frontend validation is ignored

  2. Browser developer tools:
     Attackers can edit JavaScript in the browser to skip validation
     or send requests directly from the Network tab

  3. Automated tools:
     SQLMap, Burp Suite, and other fuzzing tools send
     thousands of requests with various payloads without going through the UI

  4. Mobile apps and third-party clients:
     APIs accessed from mobile apps or third parties
     don't go through your frontend at all

  Principle: client-side validation for UX,
           server-side validation for security and integrity

flowchart LR
    subgraph Client
        A["Browser/App"] --> B["Client Validation\nfor UX"]
        B --> C[Send Request]
    end

    subgraph Server["Server — Mandatory Validation"]
        D[Receive Request] --> E[Validate Input]
        E --> |Invalid| F["400 Bad Request\nClear message"]
        E --> |Valid| G["Sanitization/Encoding"]
        G --> H[Business Logic]
        H --> I["Database/Storage"]
    end

    C --> D

Whitelists vs Blacklists: The Fundamental Principle #

Two validation approaches: whitelists (allow what’s explicitly known to be safe) and blacklists (reject what’s known to be dangerous). Whitelists are always safer.

// ANTI-PATTERN: blacklists — trying to forbid what's known to be dangerous
func validateUsernameBlacklist(username string) bool {
    forbiddenChars := []string{"<", ">", "\"", "'", ";", "--", "DROP", "SELECT"}
    lower := strings.ToLower(username)
    for _, forbidden := range forbiddenChars {
        if strings.Contains(lower, strings.ToLower(forbidden)) {
            return false
        }
    }
    return true
}

// Problems with blacklists:
// → Never complete — there's always a bypass technique
// → Attackers use encodings: %3C for <, &#60; for <, \u003c for <
// → Attackers use case variations, Unicode tricks, etc.

// CORRECT: whitelists — only allow what's known to be safe
func validateUsernameWhitelist(username string) bool {
    // Only letters, numbers, underscores, and dashes
    // 3-30 characters long
    matched, _ := regexp.MatchString(`^[a-zA-Z0-9_-]{3,30}$`, username)
    return matched
}

// If it doesn't match the pattern → rejected
// No matter what bypass technique is used

Whitelists vs Blacklists in various contexts:

  Usernames:
  ✓ Whitelist: [a-zA-Z0-9_-], 3-30 characters
  ✗ Blacklist: forbid <, >, ;, etc. (easily bypassed)

  Uploaded file types:
  ✓ Whitelist: only jpg, png, gif, pdf
  ✗ Blacklist: forbid exe, php, js (can use .phtml, .php5, etc.)

  HTML in rich text editors:
  ✓ Whitelist: only <b>, <i>, <p>, <a href="..."> with restricted attributes
  ✗ Blacklist: forbid <script>, onerror= (too many bypass ways)

  URL redirects:
  ✓ Whitelist: only registered domains
  ✗ Blacklist: forbid javascript:, data: (many other encodings exist)

Validation Per Data Type #

Strings: Length, Format, and Characters #

// CreateUserRequest — validation rules
type CreateUserRequest struct {
    Username    string
    Email       string
    DisplayName string
    Bio         string
    Website     string
}

// Validate checks every rule manually (stdlib only)
func (req *CreateUserRequest) Validate() error {
    // Minimum and maximum lengths
    if len(req.Username) < 3 || len(req.Username) > 30 {
        return errors.New("username must be 3-30 characters")
    }
    if len(req.Email) > 254 { // RFC 5321 limit
        return errors.New("email too long")
    }
    if len(req.DisplayName) < 1 || len(req.DisplayName) > 100 {
        return errors.New("display_name must be 1-100 characters")
    }
    if len(req.Bio) > 500 {
        return errors.New("bio too long")
    }
    if len(req.Website) > 2000 {
        return errors.New("website too long")
    }

    // Username: only letters, numbers, underscores, and dashes
    if !regexp.MustCompile(`^[a-zA-Z0-9_-]+$`).MatchString(req.Username) {
        return errors.New("username may only contain letters, numbers, underscores, and dashes")
    }

    // Email: basic format validation — use a library for complete validation
    if !regexp.MustCompile(`^[^@]+@[^@]+\.[^@]+$`).MatchString(req.Email) {
        return errors.New("invalid email format")
    }
    req.Email = strings.ToLower(req.Email) // normalize: lowercase the email

    // Website: must use http or https and have a host
    parsed, err := url.Parse(req.Website)
    if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") || parsed.Host == "" {
        return errors.New("website must use http or https")
    }
    return nil
}

// Usage on an endpoint
func createUser(w http.ResponseWriter, r *http.Request) {
    var req CreateUserRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        writeJSON(w, http.StatusBadRequest, map[string]string{"error": "Invalid JSON"})
        return
    }
    if err := req.Validate(); err != nil {
        writeJSON(w, http.StatusBadRequest, map[string]string{"error": err.Error()})
        return
    }

    // The data is validated — safe to process
    user := User.Create(req.Username, req.Email, req.DisplayName)
    writeJSON(w, http.StatusCreated, map[string]string{"user_id": user.ID})
}

Numbers: Ranges, Types, and Business Limits #

// TransferRequest — money validation
type TransferRequest struct {
    ToAccount   string
    Amount      float64
    Description string
}

func (req *TransferRequest) Validate() error {
    // Account numbers: digits only, 10-20 characters
    if len(req.ToAccount) < 10 || len(req.ToAccount) > 20 {
        return errors.New("to_account must be 10-20 characters")
    }
    for _, c := range req.ToAccount {
        if c < '0' || c > '9' {
            return errors.New("account numbers may only contain digits")
        }
    }

    // Amount: > 0 and <= 100 million, at most 2 decimals
    if req.Amount <= 0 || req.Amount > 100_000_000 {
        return errors.New("amount must be > 0 and <= 100 million")
    }
    if math.Round(req.Amount*100) != req.Amount*100 {
        return errors.New("transfer amounts may have at most 2 decimals")
    }
    return nil
}

// Validation often missed for numbers:
type ProductRequest struct {
    Price float64
}

func (req *ProductRequest) Validate() error {
    if req.Price < 0 {
        return errors.New("price must not be negative")
    }
    const maxPrice = 1_000_000_000 // 1 billion
    if req.Price > maxPrice {
        return fmt.Errorf("price exceeds the maximum limit of %d", maxPrice)
    }
    return nil
}

// Pagination — prevents per_page=999999 causing high server load
type PaginationRequest struct {
    Page    int // >= 1
    PerPage int // 1-100
}

func (req *PaginationRequest) Validate() error {
    if req.Page < 1 {
        return errors.New("page must be >= 1")
    }
    if req.PerPage < 1 || req.PerPage > 100 {
        return errors.New("per_page must be 1-100")
    }
    return nil
}

Emails: Correct Validation #

// ANTI-PATTERN: email regexes that are too simple or too complex
// "Perfect" email regexes are very long and hard to read
// Even complete RFC 5322 email validation isn't practical

// CORRECT: use a battle-tested library (net/mail is part of the standard library)
import "net/mail"

func validateAndNormalizeEmail(email string) (string, error) {
    // mail.ParseAddress performs:
    // - Format checks (has @, valid domain, etc.)
    // - Normalization (the address part is parsed structurally)
    parsed, err := mail.ParseAddress(email)
    if err != nil {
        return "", fmt.Errorf("invalid email: %w", err)
    }

    // Normalize: lowercase the domain part
    parts := strings.SplitN(parsed.Address, "@", 2)
    return strings.ToLower(parts[0]) + "@" + strings.ToLower(parts[1]), nil
}

// Examples:
// "[email protected]" → "[email protected]"
// "[email protected]" → "[email protected]" (valid)
// "not-an-email" → error
// "@nodomain" → error

File Uploads: Comprehensive Validation #

// Whitelist of allowed MIME types (extension per MIME type)
var allowedImageTypes = map[string][]string{
    "image/jpeg": {".jpg", ".jpeg"},
    "image/png":  {".png"},
    "image/gif":  {".gif"},
    "image/webp": {".webp"},
}

var allowedDocumentTypes = map[string][]string{
    "application/pdf": {".pdf"},
    "application/vnd.openxmlformats-officedocument.wordprocessingml.document": {".docx"},
}

const maxFileSize = 10 * 1024 * 1024 // 10MB

// validateFileUpload comprehensively validates file uploads.
// Returns: (valid bool, mimeType string, errors []string)
func validateFileUpload(header *multipart.FileHeader, allowedTypes map[string][]string) (bool, string, []string) {
    var errors []string

    // 1. Check the file size
    if header.Size == 0 {
        errors = append(errors, "File must not be empty")
    } else if header.Size > maxFileSize {
        errors = append(errors, fmt.Sprintf("Maximum file size is %dMB", maxFileSize/1024/1024))
    }

    // 2. Detect the MIME type from file CONTENT, not the extension
    // or the Content-Type header (both can be forged by attackers)
    f, err := header.Open()
    if err != nil {
        return false, "", append(errors, "Cannot read file")
    }
    defer f.Close()

    headerBytes := make([]byte, 2048)
    n, _ := f.Read(headerBytes)
    detectedMime := http.DetectContentType(headerBytes[:n])

    if _, ok := allowedTypes[detectedMime]; !ok {
        return false, "", append(errors,
            "File type not allowed. Detected as: "+detectedMime+
                ". Allowed types: "+fmt.Sprint(maps.Keys(allowedTypes)))
    }

    // 3. Validate the extension (optional but useful)
    if filename := header.Filename; filename != "" {
        ext := strings.ToLower(path.Ext(filename))
        allowedExtensions := allowedTypes[detectedMime]
        if !slices.Contains(allowedExtensions, ext) {
            errors = append(errors, "Extension '"+ext+"' doesn't match the detected file type")
        }
    }

    return len(errors) == 0, detectedMime, errors
}

JSON Schema Validation #

For APIs accepting JSON, schema validation ensures the structure and data types match expectations before processing.

// With Go — typed structs give schema validation at the JSON level
type OrderItemRequest struct {
    ProductID int    `json:"product_id"`
    Quantity  int    `json:"quantity"`
    Notes     string `json:"notes,omitempty"`
}

type CreateOrderRequest struct {
    Items             []OrderItemRequest `json:"items"`
    ShippingAddressID int               `json:"shipping_address_id"`
    UsePoints         bool              `json:"use_points"`
    PromoCode         string            `json:"promo_code,omitempty"`
}

// Manual rule validation on top of the typed schema
func (req *CreateOrderRequest) Validate() error {
    // 1 to 100 items
    if len(req.Items) < 1 || len(req.Items) > 50 {
        return errors.New("items must contain 1-50 entries")
    }
    if req.ShippingAddressID <= 0 {
        return errors.New("shipping_address_id must be > 0")
    }

    // Promo codes are only uppercase letters and numbers
    if req.PromoCode != "" && !regexp.MustCompile(`^[A-Z0-9]{3,20}$`).MatchString(req.PromoCode) {
        return errors.New("invalid promo code format")
    }

    // No duplicate products allowed in one order
    seen := map[int]bool{}
    for _, item := range req.Items {
        if item.ProductID <= 0 {
            return errors.New("product_id must be > 0")
        }
        if item.Quantity < 1 || item.Quantity > 100 {
            return errors.New("quantity must be 1-100")
        }
        if seen[item.ProductID] {
            return errors.New("no duplicate products allowed in one order")
        }
        seen[item.ProductID] = true
    }
    return nil
}

// IMPORTANT: reject unknown fields
// decoder.DisallowUnknownFields() makes unknown fields a decoding error

Sanitization vs Encoding vs Validation: An Important Difference #

These three concepts are often used interchangeably even though they differ.

Validation — does the input meet the rules?
  → Decision: accept or reject
  → Done before processing
  → Example: is this a valid email?

Sanitization — clean the input of dangerous content
  → Modify the input: remove or escape dangerous elements
  → Use for rich text (HTML with restricted tags)
  → Example: remove <script> from user-entered HTML

Encoding/Escaping — transform special characters to be safe in a given context
  → Done at OUTPUT time, not input
  → Different contexts need different encodings
  → Example: & → &amp; when rendering into HTML

The correct order:
  1. Validate the input (reject if invalid)
  2. Process and store (use parameterized queries, don't concatenate)
  3. Sanitize if needed (for rich text)
  4. Encode at output (per rendering context)

Common mistakes:
  ✗ Sanitizing input to prevent SQL injection
    → Use parameterized queries, not input sanitization
  ✗ Encoding input before storing it in the database
    → Encode at output, not at storage
    → Database data should be in its original form

Edge Cases Often Missed #

// Edge case 1: empty strings vs None vs whitespace
func validateRequiredString(value string) (string, error) {
    if value == "" {
        return "", errors.New("field is required")
    }
    stripped := strings.TrimSpace(value)
    if stripped == "" {
        return "", errors.New("field must not be only whitespace")
    }
    return stripped, nil
}

// Edge case 2: numbers in strings vs numbers
// An API receives {"quantity": "10"} when an integer is expected
// Decode into an int — "10" (a string) is rejected by encoding/json
type OrderItem struct {
    Quantity int `json:"quantity"`
    // json.Decoder.UseNumber + strict typing rejects string values
}

// Edge case 3: overly long arrays/lists — a DoS vector
type SearchRequest struct {
    Tags []string `json:"tags"`
    // Without a limit: users can send 10,000 tags → server overload
}

func (req *SearchRequest) Validate() error {
    if len(req.Tags) > 20 {
        return errors.New("too many tags (max 20)")
    }
    return nil
}

// Edge case 4: unbounded nested object depth
// JSON {"a":{"b":{"c":{"d":{"e":{...}}}}}} thousands of levels
// Causes stack overflows in some parsers
func validateJSONDepth(data any, maxDepth, currentDepth int) error {
    if currentDepth > maxDepth {
        return fmt.Errorf("JSON structure too deep (max %d levels)", maxDepth)
    }
    switch v := data.(type) {
    case map[string]any:
        for _, value := range v {
            if err := validateJSONDepth(value, maxDepth, currentDepth+1); err != nil {
                return err
            }
        }
    case []any:
        for _, item := range v {
            if err := validateJSONDepth(item, maxDepth, currentDepth+1); err != nil {
                return err
            }
        }
    }
    return nil
}

// Edge case 5: integer overflow
// Older platforms: 32-bit integer max = 2,147,483,647
// If users send quantity = 2147483648 → overflow bug
type QuantityRequest struct {
    Quantity int `json:"quantity"` // with an explicit upper bound:
}

func (req *QuantityRequest) Validate() error {
    if req.Quantity < 1 || req.Quantity > 2_147_483_647 { // 32-bit int max
        return errors.New("quantity out of range")
    }
    return nil
}

// Edge case 6: path traversal in filenames
func validateFilename(filename string) (string, error) {
    // Only take the filename, without the path
    safeName := path.Base(filepath.ToSlash(filename))
    if safeName == "" || strings.HasPrefix(safeName, ".") {
        return "", errors.New("invalid filename")
    }
    // Remove dangerous characters
    re := regexp.MustCompile(`[^\w\-.]`)
    safeName = re.ReplaceAllString(safeName, "_")
    return safeName, nil
}

// validateFilename("../../etc/passwd") → "passwd"
// validateFilename(".htaccess") → error
// validateFilename("file<>name.pdf") → "file__name.pdf"

Error Messages That Are Useful but Don’t Leak Information #

// ANTI-PATTERN: overly verbose error messages
// → Leaks system information to attackers
{
    "error": "Column 'email' cannot be null in table 'users'",
    "sql": "INSERT INTO users (username, email) VALUES (?, NULL)"
}

// ANTI-PATTERN: overly generic error messages
// → Users don't know what to fix
{"error": "Invalid request"}

// CORRECT: informative error messages for users without leaking internals
// A 400 response with useful details
{
    "error": "Validation failed",
    "details": [
        {
            "field": "email",
            "message": "Invalid email format. Example: [email protected]"
        },
        {
            "field": "username",
            "message": "Username may only contain letters, numbers, underscores, and dashes"
        }
    ]
}

// A consistent error response implementation
type FieldError struct {
    Field   string `json:"field"`
    Message string `json:"message"`
}

func validationErrorResponse(errors []FieldError) (int, map[string]any) {
    // Standardize validation error responses.
    return 400, map[string]any{
        "error":   "Validation failed",
        "details": errors,
    }
}

Validation at Multiple Layers #

Effective validation happens at several layers, each with different responsibilities:

Layer 1 — API/Controller Layer:
  → Format and type validation (is this an email? is this an integer?)
  → Schema validation (are all required fields present?)
  → Size limits (string lengths, item counts, file sizes)
  → This is the "front door" — reject before entering business logic

Layer 2 — Business Logic Layer:
  → Business rule validation (enough stock? enough balance? valid promo code?)
  → Cross-field relationship validation (start date < end date)
  → Contextual validation (is this user allowed to perform this operation?)

Layer 3 — Database Layer (last resort):
  → NOT NULL, UNIQUE, FOREIGN KEY constraints
  → CHECK constraints for value ranges
  → Type enforcement (integer columns don't accept strings)
  → This is the safety net — shouldn't be needed often if upper layers work

Principle: the earlier an error is detected, the cheaper it costs.

Anti-Patterns to Avoid #

// ✗ Anti-pattern 1: relying on client-side validation alone
// No server-side validation — only browser JavaScript validation
// curl straight to the API: no validation at all

// ✗ Anti-pattern 2: blacklists for SQL injection prevention
func sanitizeInput(value string) string {
    return strings.ReplaceAll(strings.ReplaceAll(strings.ReplaceAll(value, "'", "''"), ";", ""), "--", "")
}
// Incomplete, easily bypassed, and not the right solution
// ✓ Solution: parameterized queries — no sanitization needed for SQL

// ✗ Anti-pattern 3: no input size limits
func search(w http.ResponseWriter, r *http.Request) {
    query := r.FormValue("query")
    // No query length limit
    // Attackers can send a 100MB string as a query
    results := searchDB(query)
}
// ✓ Solution: always limit input lengths

// ✗ Anti-pattern 4: trusting Content-Type headers for file types
func uploadFile(w http.ResponseWriter, r *http.Request) {
    contentType := r.Header.Get("Content-Type") // forgeable!
    if strings.HasPrefix(contentType, "image/jpeg") {
        saveAsImage(r)
    }
}
// ✓ Solution: detect MIME types from file content (magic bytes)

// ✗ Anti-pattern 5: storing already-encoded data in the database
// Users send: <script>alert(1)</script>
// Validation: convert to &lt;script&gt;alert(1)&lt;/script&gt;
// Store in the database: &lt;script&gt;alert(1)&lt;/script&gt; ← wrong!
// When displayed: double-encoding, corrupted data
// ✓ Solution: store raw data in the database, encode at output

// ✗ Anti-pattern 6: no validation on optional fields
type UserProfile struct {
    Name string
    Bio  string // bio isn't validated! users can send a 10MB bio
}

// ✓ Solution: optional fields still need limits
type UserProfile struct {
    Name string // min 1, max 100
    Bio  string // max 500
}

Input Validation Checklist #

FUNDAMENTALS:
  □ All validation done server-side, not depending on clients
  □ Whitelist approach used — only allow what's known to be safe
  □ Validation happens at the right layers (API, business logic, database)
  □ Default behavior: reject unknown or schema-mismatched input

FORMAT & TYPE:
  □ Data types validated (integer, string, boolean, date, etc.)
  □ Formats validated (email, URL, phone numbers, UUIDs, etc.)
  □ Value ranges validated for numbers (min, max)
  □ Lengths validated for strings (min_length, max_length)
  □ Item counts validated for arrays/lists (min_items, max_items)

BUSINESS:
  □ Business rules validated (enough stock, enough balance, valid status)
  □ Cross-field relationships validated (date_start < date_end)
  □ Unique constraints validated (email not registered, username available)

FILE UPLOADS:
  □ MIME types detected from file content, not extensions or Content-Type headers
  □ File sizes limited
  □ Filenames sanitized (removing path traversal, dangerous characters)
  □ Files stored outside the web root
  □ Image files structurally validated (not just MIME types)

EDGE CASES:
  □ Empty and whitespace-only strings handled
  □ Null/None handled per requirements
  □ Path traversal prevented for inputs used as filenames
  □ Nested JSON depth limited
  □ Integer overflow prevented with explicit limits
  □ Unbounded arrays prevented with max_items

ERROR HANDLING:
  □ Error messages useful to users without leaking internal information
  □ 400 responses with details on problematic fields
  □ Validation failures logged for monitoring (but no sensitive data logged)

DATABASE:
  □ Database constraints as the last safety net
  □ NOT NULL for required fields
  □ CHECK constraints for value ranges
  □ UNIQUE constraints for fields that must be unique

Summary #

Server-side validation is mandatory — client-side validation is only for UX — anything clients send can be forged. Curl, Burp Suite, and automated scripts don’t go through frontend validation.
Whitelists are always safer than blacklists — blacklists are never complete. There’s always a bypass technique not yet on the list. Whitelists define what’s allowed, not what’s forbidden.
Validation happens at multiple layers — the API layer for formats and types, the business logic layer for business rules, the database layer as the safety net. The earlier errors are detected, the cheaper they are.
Validation, sanitization, and encoding are three different things — validation decides accept or reject, sanitization cleans dangerous content from rich text, encoding transforms special characters at output. Don’t mix their roles.
Data types must be strictly enforced — “10” isn’t 10. Using schema validation libraries like Pydantic or jsonschema ensures correct types before data is processed.
File MIME types must be detected from content, not extensions — `file.jpg` can contain PHP, `file.pdf` can contain executables. Magic bytes can’t be forged at the content level.
Optional fields still need validation — a None bio is valid, but a None bio isn’t the same as a 10MB string bio. max_length is still needed even for non-required fields.
Limit everything — string lengths, array item counts, file sizes, and nested JSON depths. Without these limits, one request can drain server resources.
Store raw data in the database, encode at output — encoding at storage time causes double-encoding and data corruption. Encode based on the context where data will be displayed.
Good error messages help users without leaking internal information — stack traces, database table names, and SQL queries must never appear in user-visible error responses.
#

← Previous: Authorization Next: Remote Code Execution