Infrastructure as Code #

Imagine two teams that both need to deploy a new server for a staging environment. The first team logs into the AWS console, clicks around creating a VPC, subnets, security groups, EC2 instances, RDS, configuring them one by one. This process takes two hours, is error-prone, and nobody can reproduce it exactly — the result is always slightly different from other environments. The second team runs one command: terraform apply. In 15 minutes, the entire environment stands up with a configuration identical to production.

Infrastructure as Code (IaC) is the practice of defining and managing infrastructure through code files that can be version-controlled, reviewed, tested, and run repeatedly with consistent results. It’s not just about automation — it’s about treating infrastructure with the same discipline as application code: review before deployment, test before production, roll back when there are problems, and complete audit trails.

Why IaC Isn’t Just Automation #

Before IaC, infrastructure was managed manually or with ad-hoc scripts. The problem wasn’t speed — it was consistency, reliability, and collaboration.

Manual infrastructure problems vs IaC:

  Manual / ClickOps:
  ✗ "Works on my environment" — production differs from staging
  ✗ Nobody can reproduce the exact same setup
  ✗ Changes undocumented — who changed what and when?
  ✗ Disaster recovery takes a long time and results aren't guaranteed identical
  ✗ No code review — big changes go straight to production
  ✗ "Snowflake servers" — unique, irreplaceable servers

  IaC:
  ✓ Identical environments: dev = staging = production (except size)
  ✓ Reproducible: destroy and rebuild in minutes with the same result
  ✓ Version control: every change in git with author and timestamp
  ✓ Code review: infrastructure changes reviewed like application code
  ✓ Disaster recovery: re-run IaC code to rebuild from scratch
  ✓ Immutable servers: replace, don't patch

flowchart LR
    subgraph Before["Before IaC"]
        A1[Engineer] -->|manual clicks| B1[AWS Console]
        B1 --> C1["Unique server\nsnowflake server"]
        C1 --> D1[Can't be reproduced]
    end

    subgraph After["With IaC"]
        A2[Engineer] -->|git commit| B2[Repository]
        B2 -->|PR + review| C2["CI/CD Pipeline"]
        C2 -->|terraform apply| D2[Infrastructure]
        D2 --> E2[Identical and reproducible]
    end

Declarative vs Imperative: Two IaC Approaches #

There are two IaC paradigms important to understand before choosing tools.

Imperative (Procedural):
  "Do the following steps..."
  You define HOW to reach the desired state.

  Example (Bash script):
  aws ec2 create-vpc --cidr-block 10.0.0.0/16
  aws ec2 create-subnet --vpc-id vpc-xxx --cidr-block 10.0.1.0/24
  aws ec2 run-instances --image-id ami-xxx --instance-type t3.micro

  Problems:
  → Not idempotent — running twice creates two VPCs, two subnets, two instances
  → Doesn't know the current state — can't "sync" if manual changes exist
  → Execution order matters and is easy to get wrong

Declarative:
  "The state I want is..."
  You define WHAT should exist; the tool determines HOW.

  Example (Terraform):
  resource "aws_vpc" "main" {
    cidr_block = "10.0.0.0/16"
  }
  resource "aws_subnet" "public" {
    vpc_id     = aws_vpc.main.id
    cidr_block = "10.0.1.0/24"
  }

  Advantages:
  → Idempotent — run 10 times, same result
  → State management — Terraform knows what already exists
  → Automatic dependency resolution — subnets know they need VPCs first

Terraform: Declarative Cloud Infrastructure #

Terraform is the most popular IaC tool for provisioning cloud resources. It works by defining the desired state and computing the diff between the desired state and the existing state.

# main.tf — a simple but production-ready infrastructure example

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  # Backend for storing state — DON'T store state locally for teams
  backend "s3" {
    bucket         = "myapp-terraform-state"
    key            = "production/terraform.tfstate"
    region         = "ap-southeast-1"
    dynamodb_table = "terraform-state-lock"  # for locking
    encrypt        = true
  }
}

provider "aws" {
  region = var.aws_region

  default_tags {
    tags = {
      Project     = var.project_name
      Environment = var.environment
      ManagedBy   = "terraform"
    }
  }
}

# Variables — parameters that can differ per environment
variable "aws_region" {
  description = "AWS region for deployment"
  type        = string
  default     = "ap-southeast-1"
}

variable "environment" {
  description = "Environment name: dev, staging, or production"
  type        = string

  validation {
    condition     = contains(["dev", "staging", "production"], var.environment)
    error_message = "Environment must be one of: dev, staging, production."
  }
}

variable "project_name" {
  description = "Project name"
  type        = string
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.micro"
}

# Networking
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.project_name}-${var.environment}-vpc"
  }
}

resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "${var.project_name}-${var.environment}-igw"
  }
}

resource "aws_subnet" "public" {
  count             = 2  # multi-AZ for high availability
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.${count.index + 1}.0/24"
  availability_zone = data.aws_availability_zones.available.names[count.index]

  map_public_ip_on_launch = true

  tags = {
    Name = "${var.project_name}-${var.environment}-public-${count.index + 1}"
    Tier = "public"
  }
}

resource "aws_subnet" "private" {
  count             = 2
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.${count.index + 10}.0/24"
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = {
    Name = "${var.project_name}-${var.environment}-private-${count.index + 1}"
    Tier = "private"
  }
}

# Security Group for the application
resource "aws_security_group" "app" {
  name        = "${var.project_name}-${var.environment}-app-sg"
  description = "Security group for the application server"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port       = 443
    to_port         = 443
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
    description     = "HTTPS from the ALB"
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
    description = "Allow all outbound"
  }

  tags = {
    Name = "${var.project_name}-${var.environment}-app-sg"
  }
}

# Outputs for use by other resources or other teams
output "vpc_id" {
  description = "VPC ID"
  value       = aws_vpc.main.id
}

output "public_subnet_ids" {
  description = "Public subnet IDs"
  value       = aws_subnet.public[*].id
}

output "private_subnet_ids" {
  description = "Private subnet IDs"
  value       = aws_subnet.private[*].id
}

The Correct Terraform Workflow #

# Standard Terraform workflow

# 1. Format the code for consistency
terraform fmt -recursive

# 2. Validate syntax and configuration
terraform validate

# 3. See changes BEFORE applying — MUST be reviewed!
terraform plan -out=tfplan

# The plan output shows:
# + = new resources to be created
# ~ = resources to be modified
# - = resources to be deleted (BE CAREFUL!)
# -/+ = resources to be replaced (destroy + create)

# Example plan output:
# Plan: 5 to add, 2 to change, 0 to destroy.

# 4. Apply only after the plan is reviewed
terraform apply tfplan

# 5. In CI/CD: always plan first, apply after manual approval
# Never auto-apply to production without review

# Other useful commands:
terraform state list          # list all managed resources
terraform state show aws_vpc.main  # detail one resource
terraform import aws_s3_bucket.legacy bucket-name  # import existing resources
terraform taint aws_instance.app  # mark for recreation on the next apply
terraform graph | dot -Tsvg > graph.svg  # visualize dependencies

Modularization: Reusable Infrastructure Components #

Modules are the way to create reusable infrastructure components. Instead of redefining the same VPC in every project, create a VPC module consumed with different parameters.

# modules/vpc/main.tf — a reusable module

variable "project_name" { type = string }
variable "environment"  { type = string }
variable "vpc_cidr"     { type = string }
variable "public_subnets"  { type = list(string) }
variable "private_subnets" { type = list(string) }

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
  # ...VPC configuration...
}

resource "aws_subnet" "public" {
  count      = length(var.public_subnets)
  cidr_block = var.public_subnets[count.index]
  # ...
}

output "vpc_id"             { value = aws_vpc.this.id }
output "public_subnet_ids"  { value = aws_subnet.public[*].id }
output "private_subnet_ids" { value = aws_subnet.private[*].id }

# environments/production/main.tf — consuming the module

module "vpc" {
  source = "../../modules/vpc"

  project_name    = "myapp"
  environment     = "production"
  vpc_cidr        = "10.0.0.0/16"
  public_subnets  = ["10.0.1.0/24", "10.0.2.0/24"]
  private_subnets = ["10.0.10.0/24", "10.0.11.0/24"]
}

module "app_server" {
  source = "../../modules/ec2"

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnet_ids
  # ...
}

Ansible: Configuration Management #

Ansible fills a different role from Terraform. Terraform is for provisioning (creating servers), Ansible is for configuration management (configuring existing servers).

# playbook.yml — application server configuration

---
- name: Configure Application Server
  hosts: app_servers
  become: true  # run as root

  vars:
    app_user: appuser
    app_dir: /opt/myapp
    node_version: "20"

  tasks:
    - name: Update package cache
      apt:
        update_cache: yes
        cache_valid_time: 3600

    - name: Install required packages
      apt:
        name:
          - git
          - curl
          - nginx
          - certbot
          - python3-certbot-nginx
        state: present

    - name: Create application user
      user:
        name: "{{ app_user }}"
        system: yes
        shell: /bin/bash
        home: "{{ app_dir }}"
        create_home: yes

    - name: Install Node.js
      block:
        - name: Download NodeSource setup script
          get_url:
            url: "https://deb.nodesource.com/setup_{{ node_version }}.x"
            dest: /tmp/nodesource_setup.sh
            mode: '0755'

        - name: Run NodeSource setup
          command: /tmp/nodesource_setup.sh

        - name: Install Node.js
          apt:
            name: nodejs
            state: present

    - name: Configure Nginx
      template:
        src: templates/nginx.conf.j2
        dest: /etc/nginx/sites-available/myapp
        mode: '0644'
      notify: Reload Nginx

    - name: Enable Nginx site
      file:
        src: /etc/nginx/sites-available/myapp
        dest: /etc/nginx/sites-enabled/myapp
        state: link
      notify: Reload Nginx

    - name: Configure application environment
      template:
        src: templates/env.j2
        dest: "{{ app_dir }}/.env"
        owner: "{{ app_user }}"
        mode: '0600'

  handlers:
    - name: Reload Nginx
      service:
        name: nginx
        state: reloaded

# inventory/production.yml — inventory file

all:
  children:
    app_servers:
      hosts:
        app01.example.com:
          ansible_user: ubuntu
          ansible_ssh_private_key_file: ~/.ssh/production.pem
        app02.example.com:
          ansible_user: ubuntu
          ansible_ssh_private_key_file: ~/.ssh/production.pem

    db_servers:
      hosts:
        db01.example.com:
          ansible_user: ubuntu

  vars:
    environment: production

Immutable Infrastructure #

Immutable infrastructure is the principle that servers are never modified after deployment — if changes are needed, create new servers and replace the old ones.

Mutable vs Immutable Infrastructure:

  Mutable (traditional):
  Server → patch → patch → patch → patch
  → Servers "age" — accumulating undocumented changes
  → "Configuration drift" — servers differ from what they should be
  → Hard to debug because you don't know exactly what's on the server
  → Hard rollbacks — must undo changes one by one

  Immutable:
  Image v1 → Image v2 → Image v3
       Server v1 replaced by a fresh Server v2 from Image v2
  → No configuration drift
  → Rollback = use the previous image version
  → Identical to other environments using the same image
  → Blue-green deployments become natural

# Packer for creating immutable AMIs (Amazon Machine Images)
# packer.pkr.hcl

packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
  }
}

variable "app_version" {
  type    = string
  default = "latest"
}

source "amazon-ebs" "app" {
  ami_name      = "myapp-${var.app_version}-{{timestamp}}"
  instance_type = "t3.micro"
  region        = "ap-southeast-1"

  source_ami_filter {
    filters = {
      name                = "ubuntu/images/hvm-ssd/ubuntu-22.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]  # Canonical
  }

  ssh_username = "ubuntu"

  tags = {
    Name        = "myapp-${var.app_version}"
    Version     = var.app_version
    BuildDate   = "{{timestamp}}"
    ManagedBy   = "packer"
  }
}

build {
  sources = ["source.amazon-ebs.app"]

  # Install dependencies
  provisioner "shell" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y nodejs npm nginx",
      "sudo npm install -g pm2",
    ]
  }

  # Deploy the application into the image
  provisioner "file" {
    source      = "dist/"
    destination = "/opt/myapp"
  }

  # Configuration
  provisioner "ansible" {
    playbook_file = "playbooks/configure-app.yml"
  }
}

GitOps: Infrastructure from Git #

GitOps is the practice of using Git as the single source of truth for infrastructure state. Infrastructure changes can only happen through pull requests to the repository.

# .github/workflows/terraform.yml — GitOps workflow

name: Terraform

on:
  push:
    branches: [main]
    paths: ['infra/**']
  pull_request:
    branches: [main]
    paths: ['infra/**']

permissions:
  contents: read
  pull-requests: write  # for posting plans to PR comments

jobs:
  terraform:
    name: Terraform Plan & Apply
    runs-on: ubuntu-latest

    defaults:
      run:
        working-directory: infra/production

    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.6.0

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ap-southeast-1

      - name: Terraform Init
        run: terraform init

      - name: Terraform Format Check
        run: terraform fmt -check -recursive

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        id: plan
        run: terraform plan -no-color -out=tfplan
        continue-on-error: true  # still post to the PR even if the plan fails

      # Post the plan to the PR comment for review
      - name: Post Plan to PR
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const output = `#### Terraform Plan 📋
            \`\`\`
            ${{ steps.plan.outputs.stdout }}
            \`\`\`
            *Planned by: @${{ github.actor }}*`;

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })            

      # Apply only on main after PRs are merged
      - name: Terraform Apply
        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
        run: terraform apply -auto-approve tfplan

Testing Infrastructure #

Infrastructure also needs testing — ensuring servers are reachable, services are running, configurations are correct, and security groups match expectations.

// test_infrastructure.go — tests using the standard library
// and the AWS SDK (aws-sdk-go-v2 style) for cloud assertions.

package infrastructure

import (
    "context"
    "encoding/json"
    "net/http"
    "os/exec"
    "testing"
    "time"

    "github.com/aws/aws-sdk-go-v2/service/ec2"
    "github.com/aws/aws-sdk-go-v2/service/rds"
)

// terraformOutputs runs `terraform output -json` in infra/staging
// and returns the raw values for use in tests.
func terraformOutputs(t *testing.T) map[string]json.RawMessage {
    t.Helper()

    out, err := exec.Command("terraform", "output", "-json").Output()
    if err != nil {
        t.Fatalf("terraform output failed: %v", err)
    }

    var decoded map[string]struct {
        Value json.RawMessage `json:"value"`
    }
    if err := json.Unmarshal(out, &decoded); err != nil {
        t.Fatalf("decoding terraform output: %v", err)
    }

    outputs := make(map[string]json.RawMessage, len(decoded))
    for name, entry := range decoded {
        outputs[name] = entry.Value
    }
    return outputs
}

// TestVPCExists verifies that the VPC has been created.
func TestVPCExists(t *testing.T) {
    ctx := context.Background()
    ec2Client := ec2.New(ec2.Options{Region: "ap-southeast-1"})
    outputs := terraformOutputs(t)

    var vpcID string
    if err := json.Unmarshal(outputs["vpc_id"], &vpcID); err != nil {
        t.Fatalf("vpc_id output is not a string: %v", err)
    }

    resp, err := ec2Client.DescribeVpcs(ctx, &ec2.DescribeVpcsInput{
        VpcIds: []string{vpcID},
    })
    if err != nil {
        t.Fatalf("describing VPC: %v", err)
    }
    if len(resp.Vpcs) != 1 || resp.Vpcs[0].State != ec2.VpcStateAvailable {
        t.Fatalf("VPC %s is not available", vpcID)
    }
}

// TestSecurityGroupRules verifies that security groups have the
// expected rules — in particular that SSH is not public.
func TestSecurityGroupRules(t *testing.T) {
    ctx := context.Background()
    ec2Client := ec2.New(ec2.Options{Region: "ap-southeast-1"})
    outputs := terraformOutputs(t)

    var sgID string
    if err := json.Unmarshal(outputs["app_security_group_id"], &sgID); err != nil {
        t.Fatalf("app_security_group_id output is not a string: %v", err)
    }

    resp, err := ec2Client.DescribeSecurityGroups(ctx, &ec2.DescribeSecurityGroupsInput{
        GroupIds: []string{sgID},
    })
    if err != nil {
        t.Fatalf("describing security group: %v", err)
    }

    for _, perm := range resp.SecurityGroups[0].IpPermissions {
        if perm.FromPort != nil && *perm.FromPort == 22 {
            for _, ip := range perm.IpRanges {
                if *ip.CidrIp == "0.0.0.0/0" {
                    t.Fatal("SSH must not be open to the internet")
                }
            }
        }
    }
}

// TestApplicationEndpoint verifies that the application is reachable.
func TestApplicationEndpoint(t *testing.T) {
    outputs := terraformOutputs(t)

    var albDNS string
    if err := json.Unmarshal(outputs["alb_dns_name"], &albDNS); err != nil {
        t.Fatalf("alb_dns_name output is not a string: %v", err)
    }

    client := &http.Client{Timeout: 10 * time.Second}
    resp, err := client.Get("https://" + albDNS + "/health")
    if err != nil {
        t.Fatalf("health check failed: %v", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        t.Fatalf("unexpected status: %d", resp.StatusCode)
    }
    var body struct {
        Status string `json:"status"`
    }
    if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
        t.Fatalf("decoding health response: %v", err)
    }
    if body.Status != "ok" {
        t.Fatalf("health status is %q", body.Status)
    }
}

// TestDatabaseNotPubliclyAccessible verifies that databases are
// not reachable from the internet.
func TestDatabaseNotPubliclyAccessible(t *testing.T) {
    ctx := context.Background()
    rdsClient := rds.New(rds.Options{Region: "ap-southeast-1"})

    resp, err := rdsClient.DescribeDBInstances(ctx, &rds.DescribeDBInstancesInput{})
    if err != nil {
        t.Fatalf("describing DB instances: %v", err)
    }

    for _, db := range resp.DBInstances {
        if db.PubliclyAccessible != nil && *db.PubliclyAccessible {
            t.Fatalf("database %s must not be publicly accessible",
                *db.DBInstanceIdentifier)
        }
    }
}

Anti-Patterns to Avoid #

# ✗ Anti-pattern 1: state files locally or in git
# terraform.tfstate committed to the repository
# State contains sensitive data (passwords, keys)
# Can't be used by teams simultaneously (race conditions)
# ✓ Solution: remote backends (S3 + DynamoDB for locking)

# ✗ Anti-pattern 2: hardcoded credentials in code
resource "aws_db_instance" "main" {
  username = "admin"
  password = "hardcoded-password-123"  # VERY DANGEROUS
}
# ✓ Solution: use variables + secret managers
resource "aws_db_instance" "main" {
  username = var.db_username
  password = data.aws_secretsmanager_secret_version.db_password.secret_string
}

# ✗ Anti-pattern 3: one huge environment without modularization
# All resources in one very long main.tf file
# ✓ Solution: modularize per component (vpc, database, application)
# and per environment (dev, staging, production)

# ✗ Anti-pattern 4: terraform apply without plan reviews
# CI/CD auto-applying without anyone checking the plan
# ✓ Solution: plans must be reviewed, apply only after approval

# ✗ Anti-pattern 5: no consistent tagging
resource "aws_instance" "app" {
  # No tags
}
# ✓ Solution: consistent tags on all resources for cost tracking and ownership

Infrastructure as Code Checklist #

STRUCTURE & ORGANIZATION:
  □ Clearly structured repositories (modules/, environments/, etc.)
  □ Modules for reusable components
  □ Separate environments: dev, staging, production
  □ Variable files per environment (terraform.tfvars)
  □ Consistent tagging on all resources

STATE MANAGEMENT:
  □ Remote backends configured (S3, GCS, or Terraform Cloud)
  □ State locking enabled (DynamoDB for S3 backends)
  □ State never committed to git
  □ State files backed up regularly

SECURITY:
  □ No hardcoded credentials or secrets
  □ Secrets from environment variables or secret managers
  □ State files encrypted (encrypt = true in backends)
  □ Least privilege for IAM roles used by Terraform/Ansible

WORKFLOW:
  □ All changes through pull requests
  □ terraform plans reviewed before terraform applies
  □ Production applies require manual approvals
  □ terraform fmt and terraform validate in CI
  □ Plan outputs posted to PR comments for easy review

TESTING:
  □ Infrastructure tested after deployment (health checks, security checks)
  □ Tests verifying critical security group rules
  □ Tests verifying databases aren't publicly accessible
  □ Integration tests for applications running on the infrastructure

DOCUMENTATION:
  □ Every variable has a clear description
  □ Every output has a clear description
  □ READMEs explaining usage and requirements
  □ Architecture documented (diagrams or explanations)

Summary #

  • IaC isn’t just automation — it’s about consistency and collaboration — infrastructure that can be reviewed, tested, version-controlled, and reproduced is trustworthy infrastructure.
  • Declarative is safer than imperative — defining what should exist (Terraform) is safer and more idempotent than defining how to create it (scripts). Declarative tools know the current state and only make needed changes.
  • Remote state is mandatory for teams — state files locally or in git are recipes for race conditions and secret leaks. Use S3 + DynamoDB or Terraform Cloud.
  • Plans must be reviewed before applies — terraform plan shows exactly what will change, including resources to be deleted. Applying without plan reviews is gambling with production infrastructure.
  • Modularization makes infrastructure maintainable — VPCs, databases, applications each as reusable modules. Copy-pasted configuration is infrastructure technical debt.
  • Immutable infrastructure eliminates configuration drift — servers never modified after deployment are always in a known state. Rollbacks are as easy as using the previous image version.
  • GitOps makes git the single source of truth — no infrastructure changes outside pull requests. Every change is documented, reviewed, and revertible.
  • Infrastructure needs testing like code — verify security group rules are correct, databases aren’t publicly accessible, and applications are reachable after deployment.
  • Secrets never exist in IaC code — variables containing passwords, API keys, or tokens must come from environment variables or secret managers, never hardcoded.
  • Consistent tagging is a long-term investment — Environment, Project, ManagedBy tags on all resources ease cost tracking, troubleshooting, and ownership clarity.
#

← Previous: OOM Killer   Next: Configuration Manager

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact