Configuration Manager #

Every application needs configuration — database URLs, API keys, worker thread counts, timeouts, feature flags. What separates applications that are easy to operate from those that become nightmares is how that configuration is managed. Configuration scattered across places, differing between developers, containing secrets committed to git, or lacking mechanisms to change without redeploying are signs of fragile systems.

Configuration Manager — in this context not referring to a specific tool, but to the practices and patterns for systematically managing application configuration — is the foundation of system operability. When configuration is managed correctly, changing production behavior can be done without deployment, secrets are safely stored in vaults instead of git, and onboarding new developers doesn’t require the ritual of “asking the senior for the configuration.”

The Problems with Unmanaged Configuration #

The most common configuration problem patterns:

  1. Hardcoded in source code:
     DATABASE_URL = "postgres://admin:***@localhost/mydb"
     → Passwords in git history forever
     → Must change code to change configuration

  2. Configuration differing between developers:
     Developer A uses port 5432, Developer B uses 5433
     → "Works on my machine" unrelated to the application
     → Hard-to-reproduce bugs

  3. Secrets in committed .env files:
     # .env — DON'T COMMIT THIS FILE
     API_KEY=sk-...  ← but it's already in git history
     → Git history stores all file versions, including old ones

  4. Configuration differing between production and staging:
     → Bugs that only appear in production
     → Nobody knows what the actual production configuration is

  5. No configuration validation:
     The app starts without a DB_URL → crashes with a confusing error
     → Should be: "DB_URL is required" at startup

  6. Secret rotation that can't happen without downtime:
     Change an API key → update .env on all servers → restart all instances
     → An error-prone manual process
flowchart TD
    A[Application Configuration] --> B{What type is it?}
    B --> C["Non-secret\nPublic URLs, timeouts, feature flags"]
    B --> D["Secret\nPasswords, API keys, private keys"]

    C --> E["Environment Variables\nor Config Files"]
    D --> F["Secret Managers\nVault, AWS Secrets Manager"]

    E --> G["Version control\n.env.example without values"]
    F --> H["Never committed\nInjected at runtime"]

    G --> I[Available to all developers]
    H --> J[Only authorized processes can access]

Twelve-Factor App: Config #

The Twelve-Factor App methodology defines one principle most fundamental to configuration: store configuration in the environment, not in code. Anything that might differ between deployments (dev, staging, production) must be in environment variables, not hardcoded.

package main

import (
	"fmt"
	"log"
	"os"
	"strconv"
	"strings"
)

// ANTI-PATTERN: configuration in code
type BadConfig struct {
	DatabaseURL string
	RedisURL    string
	SecretKey   string
	Debug       bool
	MaxWorkers  int
}

// CORRECT: configuration from the environment
type Config struct {
	DatabaseURL string
	RedisURL    string
	SecretKey   string
	Debug       bool
	MaxWorkers  int
	LogLevel    string
}

// Load and validate all configuration from the environment
func fromEnv() (*Config, error) {
	var errors []string

	require := func(name string) string {
		value := os.Getenv(name)
		if value == "" {
			errors = append(errors, fmt.Sprintf("Required environment variable '%s' is not set", name))
		}
		return value
	}
	optional := func(name, def string) string {
		value := os.Getenv(name)
		if value == "" {
			return def
		}
		return value
	}

	maxWorkers, _ := strconv.Atoi(optional("MAX_WORKERS", "4"))
	config := &Config{
		DatabaseURL: require("DATABASE_URL"),
		RedisURL:    optional("REDIS_URL", "redis://localhost:6379"),
		SecretKey:   require("SECRET_KEY"),
		Debug:       strings.ToLower(optional("DEBUG", "false")) == "true",
		MaxWorkers:  maxWorkers,
		LogLevel:    strings.ToUpper(optional("LOG_LEVEL", "INFO")),
	}

	// Fail fast: if anything is missing, crash at startup with a clear message
	if len(errors) > 0 {
		return nil, fmt.Errorf("incomplete configuration:\n  - %s", strings.Join(errors, "\n  - "))
	}

	return config, nil
}

// Load once at startup
func main() {
	config, err := fromEnv()
	if err != nil {
		log.Fatal(err)
	}
	_ = config
}

Environment Variables: Practical Guidance #

# .env.example — commit this file to git (without sensitive values)
# This documents the configuration the application needs

# Database (REQUIRED)
DATABASE_URL=postgres://user:***@host:5432/dbname

# Redis (REQUIRED for caching and sessions)
REDIS_URL=redis://localhost:6379

# Security (REQUIRED)
SECRET_KEY=generate-with-openssl-rand-hex-32

# Application settings
DEBUG=false
LOG_LEVEL=INFO
MAX_WORKERS=4
PORT=8000

# External services (REQUIRED for the payment feature)
PAYMENT_GATEWAY_URL=https://api.payment-gateway.com
PAYMENT_API_KEY=your-api-key-here

# Optional: Email service
SMTP_HOST=smtp.mailtrap.io
SMTP_PORT=587
SMTP_USER=
SMTP_PASS=
# .gitignore — MAKE SURE .env is here
.env
.env.local
.env.*.local
*.secret

# .env — NOT committed to git
# This file exists on every developer machine and server
# Created from .env.example and filled with actual values

DATABASE_URL=postgres://myuser:***@localhost/mydb_dev
REDIS_URL=redis://localhost:6379
SECRET_KEY=a8f5f167f44f4964e6c998dee827110c
DEBUG=true
LOG_LEVEL=DEBUG
// Go — using godotenv
func loadEnv() {
	// Load the .env file if it exists (for local development)
	// In production, environment variables already come from the system
	_ = godotenv.Load()
}

Per-Environment Configuration #

Per-environment configuration strategies:

  Approach: Overlay / inheritance
  Default values in code (lowest priority)
  ↓
  Config files (environment-specific, not secrets)
  ↓
  Environment variables (highest priority, override everything)

  Dev:        each developer's local .env
  Staging:    environment variables from secret managers (auto-injected)
  Production: environment variables from secret managers (auto-injected)

  No committed config files contain actual values
package main

import (
	"encoding/json"
	"os"
	"strings"
)

// A simple layered config implementation
// Configuration with priorities:
// 1. Environment variables (highest)
// 2. config.{environment}.json
// 3. config.default.json (lowest)
type LayeredConfig struct {
	config map[string]interface{}
}

func NewLayeredConfig() *LayeredConfig {
	env := os.Getenv("APP_ENV")
	if env == "" {
		env = "development"
	}
	c := &LayeredConfig{config: map[string]interface{}{}}
	c.loadFile("config.default.json")
	c.loadFile("config." + env + ".json")
	c.loadEnvVars()
	return c
}

func (c *LayeredConfig) loadFile(filename string) {
	data, err := os.ReadFile(filename)
	if err != nil {
		return
	}
	var parsed map[string]interface{}
	if json.Unmarshal(data, &parsed) == nil {
		mergeMaps(c.config, parsed)
	}
}

func (c *LayeredConfig) loadEnvVars() {
	// Override with relevant environment variables
	mappings := map[string]string{
		"DATABASE_URL": "database.url",
		"REDIS_URL":    "redis.url",
		"LOG_LEVEL":    "logging.level",
		"MAX_WORKERS":  "server.max_workers",
	}
	for envKey, configKey := range mappings {
		if value := os.Getenv(envKey); value != "" {
			setNested(c.config, strings.Split(configKey, "."), value)
		}
	}
}

// Get with dot notation: get("database.url")
func (c *LayeredConfig) get(key string) interface{} {
	keys := strings.Split(key, ".")
	var d interface{} = c.config
	for _, k := range keys {
		m, ok := d.(map[string]interface{})
		if !ok {
			return nil
		}
		d = m[k]
	}
	return d
}

func mergeMaps(dst, src map[string]interface{}) {
	for k, v := range src {
		if vm, ok := v.(map[string]interface{}); ok {
			if dm, ok := dst[k].(map[string]interface{}); ok {
				mergeMaps(dm, vm)
				continue
			}
		}
		dst[k] = v
	}
}

func setNested(m map[string]interface{}, keys []string, value string) {
	for _, k := range keys[:len(keys)-1] {
		if _, ok := m[k].(map[string]interface{}); !ok {
			m[k] = map[string]interface{}{}
		}
		m = m[k].(map[string]interface{})
	}
	m[keys[len(keys)-1]] = value
}

Secret Management #

Secrets — passwords, API keys, private keys — must never be in source code or committed config files. Use dedicated secret managers.

HashiCorp Vault #

package main

import (
	"os"

	vault "github.com/hashicorp/vault/api"
)

// getVaultClient returns an AppRole-authenticated Vault client
func getVaultClient() (*vault.Client, error) {
	client, err := vault.NewClient(vault.DefaultConfig())
	if err != nil {
		return nil, err
	}
	client.SetAddress(os.Getenv("VAULT_ADDR"))

	// AppRole login
	secret, err := client.Logical().Write("auth/approle/login", map[string]interface{}{
		"role_id":   os.Getenv("VAULT_ROLE_ID"),
		"secret_id": os.Getenv("VAULT_SECRET_ID"),
	})
	if err != nil {
		return nil, err
	}
	client.SetToken(secret.Auth.ClientToken)
	return client, nil
}

// getDatabaseCredentials returns dynamic secrets — new credentials each time, with limited TTLs
func getDatabaseCredentials() (map[string]interface{}, error) {
	client, err := getVaultClient()
	if err != nil {
		return nil, err
	}
	secret, err := client.Logical().Read("database/creds/myapp-role")
	if err != nil {
		return nil, err
	}
	return map[string]interface{}{
		"username": secret.Data["username"],
		"password": secret.Data["password"],
		"lease_id": secret.LeaseID,
	}, nil
}

// getStaticSecret returns static secrets from the Vault KV store (v2)
func getStaticSecret(path string) (map[string]interface{}, error) {
	client, err := getVaultClient()
	if err != nil {
		return nil, err
	}
	secret, err := client.KVv2("secret").Get(nil, path)
	if err != nil {
		return nil, err
	}
	return secret.Data, nil
}

AWS Secrets Manager #

package main

import (
	"context"
	"encoding/json"
	"os"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/secretsmanager"
)

// getSecret returns a secret from AWS Secrets Manager.
// Cached per process — refresh by creating a new instance or restarting.
func getSecret(secretName string) (map[string]string, error) {
	cfg, err := config.LoadDefaultConfig(context.Background(),
		config.WithRegion(envOr("AWS_DEFAULT_REGION", "ap-southeast-1")))
	if err != nil {
		return nil, err
	}
	client := secretsmanager.NewFromConfig(cfg)

	output, err := client.GetSecretValue(context.Background(), &secretsmanager.GetSecretValueInput{
		SecretId: aws.String(secretName),
	})
	if err != nil {
		return nil, err
	}

	var parsed map[string]string
	if err := json.Unmarshal([]byte(*output.SecretString), &parsed); err != nil {
		return nil, err
	}
	return parsed, nil
}

// Usage
func getDatabaseConfig() (map[string]string, error) {
	env := os.Getenv("APP_ENV")
	return getSecret("myapp/" + env + "/database")
	// Returns: {"host": "...", "port": "5432", "username": "...", "password": "..."}
}

func envOr(key, fallback string) string {
	if v := os.Getenv(key); v != "" {
		return v
	}
	return fallback
}
AWS Secrets Manager supports automatic rotation — database passwords can rotate every 30 days without downtime. Make sure applications don’t cache credentials too long (use cache TTLs < 1 hour) to get the latest values after rotation.

Feature Flags #

Feature flags separate deployment from release — code can be deployed anytime, features are enabled separately. Useful for canary releases, A/B testing, and kill switches.

package main

import (
	"context"
	"encoding/json"
	"time"

	"github.com/redis/go-redis/v9"
)

var redisClient = redis.NewClient(&redis.Options{
	Addr: "localhost:6379",
})

const cacheTTL = 60 * time.Second

// isEnabled checks the flag with rollout percentage and whitelist
func isEnabled(ctx context.Context, flagName string, userID int64) bool {
	flag := getFlag(ctx, flagName)
	if flag == nil || flag["enabled"] != true {
		return false
	}

	// Rollout percentage: enabled for X% of users
	rolloutPct := 100
	if v, ok := flag["rollout_percentage"].(float64); ok {
		rolloutPct = int(v)
	}

	// User whitelists are always enabled
	if userID > 0 && containsID(flag["user_whitelist"], userID) {
		return true
	}

	// Deterministic bucketing: the same user always gets the same result
	if userID > 0 && rolloutPct < 100 {
		return userID%100 < int64(rolloutPct)
	}

	return rolloutPct == 100
}

// getFlag reads from the cache, falling back to the database
func getFlag(ctx context.Context, flagName string) map[string]interface{} {
	cacheKey := "feature_flag:" + flagName
	cached, err := redisClient.Get(ctx, cacheKey).Result()
	if err == nil {
		var flag map[string]interface{}
		if json.Unmarshal([]byte(cached), &flag) == nil {
			return flag
		}
	}

	// Get from the database if not in the cache
	flag := queryFlagFromDB(flagName)
	if flag == nil {
		return nil
	}

	flagData := map[string]interface{}{
		"enabled":            flag.enabled,
		"rollout_percentage": flag.rolloutPercentage,
		"user_whitelist":     flag.userWhitelist,
	}
	encoded, _ := json.Marshal(flagData)
	redisClient.Set(ctx, cacheKey, encoded, cacheTTL)
	return flagData
}

// Middleware-style guard for handlers (equivalent of the Python decorator)
func featureFlag(flagName string, next func(userID int64)) func(userID int64) {
	return func(userID int64) {
		if !isEnabled(context.Background(), flagName, userID) {
			writeJSON(map[string]string{"error": "Feature not available"}, 404)
			return
		}
		next(userID)
	}
}

// Usage
// http.HandleFunc("/api/new-checkout", featureFlag("new_checkout_flow", handleNewCheckout))

// Inside regular functions
func getRecommendations(userID int64) []string {
	if isEnabled(context.Background(), "ml_recommendations", userID) {
		return mlService.GetRecommendations(userID)
	}
	return getPopularProducts() // fallback
}

func containsID(list interface{}, userID int64) bool {
	items, ok := list.([]interface{})
	if !ok {
		return false
	}
	for _, item := range items {
		if id, ok := item.(float64); ok && int64(id) == userID {
			return true
		}
	}
	return false
}

Startup Configuration Validation #

Failing fast at startup prevents applications from running with wrong configuration and crashing mid-way with confusing errors.

package main

import (
	"fmt"
	"net/url"
	"os"
	"strconv"
)

// Validate all configuration at startup
type ConfigValidator struct {
	errors   []string
	warnings []string
}

func (v *ConfigValidator) require(name string) *ConfigValidator {
	if os.Getenv(name) == "" {
		v.errors = append(v.errors, fmt.Sprintf("MISSING: %s is required", name))
	}
	return v
}

func (v *ConfigValidator) requireURL(name string) *ConfigValidator {
	value := os.Getenv(name)
	if value == "" {
		v.errors = append(v.errors, fmt.Sprintf("MISSING: %s is required", name))
		return v
	}
	parsed, err := url.Parse(value)
	if err != nil || parsed.Scheme == "" || parsed.Host == "" {
		v.errors = append(v.errors, fmt.Sprintf("INVALID_URL: %s = '%s'", name, value))
	}
	return v
}

func (v *ConfigValidator) requireInt(name string, minVal, maxVal *int) *ConfigValidator {
	value := os.Getenv(name)
	if value == "" {
		v.errors = append(v.errors, fmt.Sprintf("MISSING: %s is required", name))
		return v
	}
	intVal, err := strconv.Atoi(value)
	if err != nil {
		v.errors = append(v.errors, fmt.Sprintf("NOT_INTEGER: %s = '%s'", name, value))
		return v
	}
	if minVal != nil && intVal < *minVal {
		v.errors = append(v.errors, fmt.Sprintf("INVALID: %s = %d (min: %d)", name, intVal, *minVal))
	}
	if maxVal != nil && intVal > *maxVal {
		v.errors = append(v.errors, fmt.Sprintf("INVALID: %s = %d (max: %d)", name, intVal, *maxVal))
	}
	return v
}

func (v *ConfigValidator) warnIfDefault(name, dangerousValue string) *ConfigValidator {
	if os.Getenv(name) == dangerousValue {
		v.warnings = append(v.warnings,
			fmt.Sprintf("WARNING: %s is still using an unsafe default value", name))
	}
	return v
}

func (v *ConfigValidator) validate() {
	for _, warning := range v.warnings {
		fmt.Printf("⚠️  %s\n", warning)
	}
	if len(v.errors) > 0 {
		fmt.Printf("\n❌ Invalid configuration:\n\n")
		for _, e := range v.errors {
			fmt.Printf("  ✗ %s\n", e)
		}
		fmt.Printf("\nSee .env.example for guidance.\n")
		os.Exit(1)
	}
	fmt.Println("✓ Configuration valid")
}

func newConfigValidator() *ConfigValidator { return &ConfigValidator{} }
func intPtr(v int) *int                    { return &v }

// Call at the very beginning of the application, before anything else
func validateConfig() {
	newConfigValidator().
		requireURL("DATABASE_URL").
		requireURL("REDIS_URL").
		require("SECRET_KEY").
		requireInt("MAX_WORKERS", intPtr(1), intPtr(64)).
		requireInt("PORT", intPtr(1024), intPtr(65535)).
		warnIfDefault("SECRET_KEY", "changeme").
		warnIfDefault("SECRET_KEY", "your-secret-key-here").
		validate()
}

Anti-Patterns to Avoid #

package main

// ✗ Anti-pattern 1: secrets in source code
const apiKey = "sk-..."
const databasePassword = "supersecretpassword"
// ✓ Solution: environment variables + secret managers

// ✗ Anti-pattern 2: .env committed to git
// .gitignore doesn't contain .env
// ✓ Solution: .env in .gitignore FROM THE START, commit .env.example instead

// ✗ Anti-pattern 3: no configuration validation
// The app starts without a DATABASE_URL → crashes when a request arrives
// ✓ Solution: validate and fail fast at startup with clear messages

// ✗ Anti-pattern 4: .env.example not up-to-date
// Production has 10 env vars not present in .env.example
// New developers debug for 2 hours
// ✓ Solution: .env.example always updated alongside configuration changes

// ✗ Anti-pattern 5: feature flags without cleanup mechanisms
// Flags at 100% rollout for 6 months are never removed
// Old code and unnecessary conditionals keep piling up
// ✓ Solution: feature flags have expiry dates, cleaned up routinely

// ✗ Anti-pattern 6: caching secrets too long
// Secrets fetched at startup, cached forever
// Secret rotation in Vault has no effect
// ✓ Solution: cache secrets with short TTLs (< 1 hour)

Configuration Manager Checklist #

CONFIGURATION STRUCTURE:
  □ .env.example exists in the repository with all required variables
  □ .env is in .gitignore (and never committed)
  □ All variables documented (name, type, description, example values)
  □ Clear separation between non-secrets and secrets

SECRET MANAGEMENT:
  □ No hardcoded secrets in source code
  □ No secrets in config files committed to git
  □ Secrets fetched from secret managers (Vault, AWS Secrets Manager, etc.)
  □ Secret rotation possible without downtime
  □ Audit logs for secret access available

VALIDATION:
  □ Configuration validated at application startup (fail fast)
  □ Clear error messages for missing or invalid configuration
  □ Unsafe default values produce warnings

PER ENVIRONMENT:
  □ Production configuration can't "leak" into development
  □ Clear mechanisms for changing configuration in each environment
  □ Staging configuration as close to production as possible

FEATURE FLAGS:
  □ Feature flags exist for risky new features
  □ Rollout percentage and user whitelist mechanisms exist
  □ Old flags at 100% rollout cleaned up regularly
  □ Kill switches exist for problematic features

OPERATIONAL:
  □ Changing configuration doesn't require deployments (except structural changes)
  □ Configuration changes audited (who changed what when)
  □ Configuration rollbacks possible quickly

Summary #

  • Configuration in the environment, not in code — the most fundamental twelve-factor principle. Anything differing between deployments must be in environment variables, not hardcoded.
  • .env.example is the contract, .env is the implementation — .env.example is committed to git as documentation of what’s needed. .env is never committed — it contains actual values specific to each environment.
  • Secrets need more treatment than regular environment variables — use secret managers (Vault, AWS Secrets Manager) for passwords, API keys, and private keys. Secret managers provide audit logs, automatic rotation, and controlled access.
  • Validate configuration at startup, not at use time — failing fast with clear error messages is far better than crashing mid-way with confusing errors.
  • Feature flags separate deployment from release — code can be deployed anytime, features enabled separately. This enables canary releases, A/B testing, and kill switches without code rollbacks.
  • Cache secrets with short TTLs — secrets cached forever can’t be rotated. Fetch fresh regularly or use secret manager automatic rotation mechanisms.
  • Keep .env.example always up-to-date — new developers missing required configuration will debug for hours. A complete .env.example saves everyone time.
  • Feature flags have lifecycles — flags created for gradual rollouts must be removed after 100% rollout. Accumulated flags are technical debt slowing teams down.
  • Production configuration must change without deployments — timeouts, worker counts, and rate limit thresholds changeable via environment variables without full CI/CD cycles.
  • All secret access must be audited — who accessed which secret and when must be recorded. This is critical for compliance and incident investigations.
#

← Previous: Infrastructure as Code   Next: Serverless

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact