Terraform for advanced users: HCL syntax and masterclass

Master HCL syntax: from advanced data structures and built-in functions through for-expressions for professional Terraform development.

Reading time: 45 min

You have Terraform fundamentals in place and a first project running. The next step is HashiCorp Configuration Language (HCL) in full: advanced HCL features and professional syntax techniques that make Terraform configurations maintainable, reusable and robust.

Building on Terraform fundamentals, the language features that separate simple Terraform scripts from professional infrastructure definitions come next: advanced data structures, complex validation, sensitive data handling and Terraform's built-in functions. Each section pairs theory with examples you can apply directly.

⚠️ Important notes and prerequisites: The material is aimed at experienced Linux administrators, junior DevOps engineers and IT professionals with solid Linux and networking knowledge. If you are comfortable on the command line and already understand cloud concepts such as virtual machines, networks and IAM, you are in the right place.

Terraform newcomers are welcome, but Linux beginners should complete basic tutorials first so you can focus on the essentials.

📋 Additional prerequisites:

  • Basic understanding of Terraform concepts (providers, resources, state)
  • Experience with the Terraform workflow (init, plan, apply)
  • Practical knowledge of basic HCL syntax
  • First experience with variables and outputs
As usual you will find special markers throughout:
* 💡 Tips and notes: For a traceable, efficient way of working
* ⚠️ Warnings and pitfalls: For destructive or changing commands
* 🔧 Practical examples: To follow along directly
* ❗ Typical sources of error: With an explanation of the cause

🎯 Goals: After working through the material you command the advanced HCL features that professional Terraform configurations need. You understand complex data structures, can implement robust validation, and know how to manage sensitive data safely. Built-in functions become tools for elegant, reusable infrastructure definitions.

The outcome is HCL you can use to write maintainable, secure and efficient Terraform configurations — and a base for more complex topics such as state management, AWS infrastructures and team workflows in later parts of the series.

From advanced syntax features through built-in functions, the path is toward Terraform code that is not only functional but elegant and professional.

Advanced HCL syntax

The fundamentals already covered first HCL syntax in practice. HCL is more than syntax — it is the language in which you express the entire infrastructure. Advanced HCL features let you implement complex logic, build dynamic configurations and write maintainable infrastructure definitions.

Complex data structures (objects, sets, maps)

What makes HCL a powerful configuration language? HCL combines the simplicity of a configuration language with the flexibility of a programming language. Unlike plain JSON or YAML, HCL offers functions, variables and conditional logic. That lets you reuse configurations written once across different scenarios.

Why is understanding the advanced syntax decisive? Without advanced HCL features you write repetitive code that is hard to maintain. With the advanced functions you create dynamic, reusable configurations that can adapt to different environments. That lowers the error rate and makes the infrastructure more scalable.

What should you watch for with advanced HCL features? The power of the advanced syntax can also produce complex, hard-to-read configuration. Balance between functionality and readability is decisive. Every piece of complexity must have a clear benefit.

What do you use advanced HCL syntax for? Mainly for multi-environment configurations, dynamic resource creation, complex validation and automating infrastructure patterns that repeat.

What are complex data structures in HCL? Besides the simple data types (string, number, bool), HCL offers complex structures such as objects, sets and maps. They let you organise hierarchical data and keep complex configurations readable.

Why do complex data structures matter? They solve variable explosion. Instead of defining dozens of individual variables, you organise related data in logical structures. That makes the configuration more maintainable and reduces sources of error.

What must you watch for with complex data structures? Type consistency is decisive. HCL is type-safe, which means you cannot simply mix different data types. Nested structures can also raise debugging complexity.

What do you use complex data structures for? For configurations that belong together logically: database configurations, network settings, application configs and multi-environment definitions.

Object data type in detail:

Objects are structured data types with named attributes. Each attribute can have a different data type. They are ideal for configurations that have several related properties.


# Advanced object definition for database configuration
variable "database_config" {
  description = "Complete database configuration for different environments"
  type = object({
    # Base configuration
    engine         = string
    engine_version = string
    instance_class = string

    # Storage configuration
    storage_type      = string
    allocated_storage = number
    max_storage      = number
    storage_encrypted = bool

    # Backup configuration
    backup_enabled         = bool
    backup_retention_days  = number
    backup_window         = string
    maintenance_window    = string

    # Monitoring configuration
    monitoring_enabled    = bool
    monitoring_interval   = number
    performance_insights  = bool

    # Network configuration
    subnet_group_name     = string
    security_group_ids    = list(string)
    publicly_accessible   = bool

    # Tags and metadata
    tags = map(string)

    # Advanced options
    parameters = map(string)
  })

  validation {
    condition = contains(["mysql", "postgres", "mariadb"], var.database_config.engine)
    error_message = "Database engine must be mysql, postgres, or mariadb."
  }

  validation {
    condition = var.database_config.allocated_storage >= 20 && var.database_config.allocated_storage <= 1000
    error_message = "Allocated storage must be between 20 and 1000 GB."
  }

  validation {
    condition = var.database_config.backup_retention_days >= 0 && var.database_config.backup_retention_days <= 35
    error_message = "Backup retention must be between 0 and 35 days."
  }
}

# Environment-specific database configurations
locals {
  database_configs = {
    dev = {
      engine         = "mysql"
      engine_version = "8.0"
      instance_class = "db.t3.micro"

      storage_type      = "gp2"
      allocated_storage = 20
      max_storage      = 50
      storage_encrypted = false

      backup_enabled        = false
      backup_retention_days = 0
      backup_window        = "03:00-04:00"
      maintenance_window   = "sun:04:00-sun:05:00"

      monitoring_enabled   = false
      monitoring_interval  = 0
      performance_insights = false

      subnet_group_name   = "dev-db-subnet-group"
      security_group_ids  = ["sg-dev-db"]
      publicly_accessible = false

      tags = {
        Environment = "development"
        BackupPolicy = "none"
        CostCenter  = "engineering"
      }

      parameters = {
        "innodb_buffer_pool_size" = "128M"
        "max_connections"         = "100"
      }
    }

    staging = {
      engine         = "mysql"
      engine_version = "8.0"
      instance_class = "db.t3.small"

      storage_type      = "gp3"
      allocated_storage = 50
      max_storage      = 200
      storage_encrypted = true

      backup_enabled        = true
      backup_retention_days = 7
      backup_window        = "03:00-04:00"
      maintenance_window   = "sun:04:00-sun:05:00"

      monitoring_enabled   = true
      monitoring_interval  = 60
      performance_insights = true

      subnet_group_name   = "staging-db-subnet-group"
      security_group_ids  = ["sg-staging-db"]
      publicly_accessible = false

      tags = {
        Environment = "staging"
        BackupPolicy = "weekly"
        CostCenter  = "engineering"
      }

      parameters = {
        "innodb_buffer_pool_size" = "256M"
        "max_connections"         = "200"
      }
    }

    prod = {
      engine         = "mysql"
      engine_version = "8.0"
      instance_class = "db.r5.large"

      storage_type      = "gp3"
      allocated_storage = 200
      max_storage      = 1000
      storage_encrypted = true

      backup_enabled        = true
      backup_retention_days = 30
      backup_window        = "03:00-04:00"
      maintenance_window   = "sun:04:00-sun:05:00"

      monitoring_enabled   = true
      monitoring_interval  = 15
      performance_insights = true

      subnet_group_name   = "prod-db-subnet-group"
      security_group_ids  = ["sg-prod-db-primary", "sg-prod-db-backup"]
      publicly_accessible = false

      tags = {
        Environment = "production"
        BackupPolicy = "daily"
        CostCenter  = "operations"
        Compliance  = "required"
      }

      parameters = {
        "innodb_buffer_pool_size" = "1024M"
        "max_connections"         = "500"
        "slow_query_log"         = "1"
        "long_query_time"        = "2"
      }
    }
  }

  # Current environment configuration
  current_db_config = local.database_configs[var.environment]
}

# RDS instance with object configuration
resource "aws_db_instance" "main" {
  identifier = "${var.project_name}-${var.environment}-db"

  # Engine configuration
  engine         = local.current_db_config.engine
  engine_version = local.current_db_config.engine_version
  instance_class = local.current_db_config.instance_class

  # Storage configuration
  storage_type      = local.current_db_config.storage_type
  allocated_storage = local.current_db_config.allocated_storage
  max_allocated_storage = local.current_db_config.max_storage
  storage_encrypted = local.current_db_config.storage_encrypted

  # Backup configuration
  backup_retention_period = local.current_db_config.backup_retention_days
  backup_window          = local.current_db_config.backup_window
  maintenance_window     = local.current_db_config.maintenance_window

  # Monitoring configuration
  monitoring_interval                = local.current_db_config.monitoring_interval
  monitoring_role_arn               = local.current_db_config.monitoring_enabled ? aws_iam_role.rds_monitoring[0].arn : null
  performance_insights_enabled       = local.current_db_config.performance_insights
  performance_insights_retention_period = local.current_db_config.performance_insights ? 7 : null

  # Network configuration
  db_subnet_group_name   = local.current_db_config.subnet_group_name
  vpc_security_group_ids = local.current_db_config.security_group_ids
  publicly_accessible    = local.current_db_config.publicly_accessible

  # Database configuration
  db_name  = "${var.project_name}_${var.environment}"
  username = var.db_username
  password = var.db_password

  # Tags
  tags = merge(
    local.current_db_config.tags,
    {
      Name = "${var.project_name}-${var.environment}-db"
    }
  )

  # Lifecycle management
  lifecycle {
    prevent_destroy = true
  }
}

# Parameter group for database optimisation
resource "aws_db_parameter_group" "main" {
  family = "${local.current_db_config.engine}${substr(local.current_db_config.engine_version, 0, 3)}"
  name   = "${var.project_name}-${var.environment}-params"

  dynamic "parameter" {
    for_each = local.current_db_config.parameters
    content {
      name  = parameter.key
      value = parameter.value
    }
  }

  tags = local.current_db_config.tags
}

💡 Object optimisation: Use objects for configurations that belong together logically. That makes your Terraform configuration not only clearer but also more type-safe.

Set data type extended:

Sets are collections of unique values with no particular order. They are ideal for configurations where order is irrelevant but uniqueness matters.


# Advanced set configuration for security groups
variable "security_rules" {
  description = "Security group rules configuration"
  type = set(object({
    type        = string
    from_port   = number
    to_port     = number
    protocol    = string
    cidr_blocks = list(string)
    description = string
  }))

  default = [
    {
      type        = "ingress"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = ["10.0.0.0/16"]
      description = "SSH access from VPC"
    },
    {
      type        = "ingress"
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTP access from internet"
    },
    {
      type        = "ingress"
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTPS access from internet"
    },
    {
      type        = "egress"
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
      description = "All outbound traffic"
    }
  ]
}

# Environment-specific security rules
locals {
  # Base rules for all environments
  base_security_rules = [
    {
      type        = "egress"
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
      description = "All outbound traffic"
    }
  ]

  # Development environment - less restrictive
  dev_security_rules = [
    {
      type        = "ingress"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "SSH access for development"
    },
    {
      type        = "ingress"
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTP access"
    },
    {
      type        = "ingress"
      from_port   = 8080
      to_port     = 8080
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "Development server"
    }
  ]

  # Production environment - restrictive
  prod_security_rules = [
    {
      type        = "ingress"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = ["10.0.0.0/16"]
      description = "SSH access from VPC only"
    },
    {
      type        = "ingress"
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTP access"
    },
    {
      type        = "ingress"
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTPS access"
    }
  ]

  # Merge environment-specific rules
  environment_rules = var.environment == "prod" ? local.prod_security_rules : local.dev_security_rules

  # Final rule set
  final_security_rules = toset(concat(local.base_security_rules, local.environment_rules))
}

# Security group with dynamic rules
resource "aws_security_group" "app" {
  name_prefix = "${var.project_name}-${var.environment}-app-"
  description = "Security group for ${var.project_name} application"
  vpc_id      = var.vpc_id

  # Dynamic rule creation
  dynamic "ingress" {
    for_each = { for rule in local.final_security_rules : "${rule.type}-${rule.from_port}-${rule.to_port}" => rule if rule.type == "ingress" }
    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
      description = ingress.value.description
    }
  }

  dynamic "egress" {
    for_each = { for rule in local.final_security_rules : "${rule.type}-${rule.from_port}-${rule.to_port}" => rule if rule.type == "egress" }
    content {
      from_port   = egress.value.from_port
      to_port     = egress.value.to_port
      protocol    = egress.value.protocol
      cidr_blocks = egress.value.cidr_blocks
      description = egress.value.description
    }
  }

  tags = {
    Name = "${var.project_name}-${var.environment}-app-sg"
    Environment = var.environment
  }

  lifecycle {
    create_before_destroy = true
  }
}

⚠️ Set pitfall: Sets have no order. If you need the order of the elements, use a list instead.

Map data type for complex configurations:

Maps are key-value pairs that can contain more complex structures as values. They are ideal for configurations that must be addressed by keys.


# Advanced map configuration for multi-region deployment
variable "region_configurations" {
  description = "Configuration for different AWS regions"
  type = map(object({
    # Region-specific settings
    availability_zones = list(string)
    cidr_block        = string

    # Compute configuration
    instance_types = object({
      web = string
      app = string
      db  = string
    })

    # Auto Scaling configuration
    scaling_config = object({
      min_size         = number
      max_size         = number
      desired_capacity = number
      scale_up_threshold   = number
      scale_down_threshold = number
    })

    # Backup configuration
    backup_config = object({
      enabled = bool
      retention_days = number
      cross_region_copy = bool
      destination_region = string
    })

    # Monitoring configuration
    monitoring_config = object({
      enabled = bool
      detailed_monitoring = bool
      log_retention_days = number
      alert_endpoints = list(string)
    })

    # Compliance settings
    compliance_config = object({
      encryption_required = bool
      audit_logging = bool
      vpc_flow_logs = bool
      cloudtrail_enabled = bool
    })

    # Cost optimisation
    cost_config = object({
      spot_instances_enabled = bool
      reserved_instances = bool
      auto_shutdown = bool
      shutdown_schedule = string
    })
  }))

  default = {
    us-east-1 = {
      availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
      cidr_block        = "10.0.0.0/16"

      instance_types = {
        web = "t3.small"
        app = "t3.medium"
        db  = "db.t3.small"
      }

      scaling_config = {
        min_size             = 2
        max_size             = 10
        desired_capacity     = 3
        scale_up_threshold   = 70
        scale_down_threshold = 30
      }

      backup_config = {
        enabled = true
        retention_days = 30
        cross_region_copy = true
        destination_region = "us-west-2"
      }

      monitoring_config = {
        enabled = true
        detailed_monitoring = true
        log_retention_days = 90
        alert_endpoints = ["alerts-east@company.com"]
      }

      compliance_config = {
        encryption_required = true
        audit_logging = true
        vpc_flow_logs = true
        cloudtrail_enabled = true
      }

      cost_config = {
        spot_instances_enabled = false
        reserved_instances = true
        auto_shutdown = false
        shutdown_schedule = "never"
      }
    }

    us-west-2 = {
      availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
      cidr_block        = "10.1.0.0/16"

      instance_types = {
        web = "t3.micro"
        app = "t3.small"
        db  = "db.t3.micro"
      }

      scaling_config = {
        min_size             = 1
        max_size             = 5
        desired_capacity     = 2
        scale_up_threshold   = 80
        scale_down_threshold = 20
      }

      backup_config = {
        enabled = true
        retention_days = 7
        cross_region_copy = false
        destination_region = ""
      }

      monitoring_config = {
        enabled = true
        detailed_monitoring = false
        log_retention_days = 30
        alert_endpoints = ["alerts-west@company.com"]
      }

      compliance_config = {
        encryption_required = false
        audit_logging = false
        vpc_flow_logs = false
        cloudtrail_enabled = false
      }

      cost_config = {
        spot_instances_enabled = true
        reserved_instances = false
        auto_shutdown = true
        shutdown_schedule = "0 22 * * MON-FRI"
      }
    }
  }
}

# Apply regional configuration
locals {
  current_region_config = var.region_configurations[var.aws_region]
}

# VPC with regional configuration
resource "aws_vpc" "main" {
  cidr_block           = local.current_region_config.cidr_block
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.project_name}-${var.environment}-vpc"
    Region = var.aws_region
  }
}

# Subnets based on regional configuration
resource "aws_subnet" "public" {
  count = length(local.current_region_config.availability_zones)

  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(local.current_region_config.cidr_block, 8, count.index)
  availability_zone       = local.current_region_config.availability_zones[count.index]
  map_public_ip_on_launch = true

  tags = {
    Name = "${var.project_name}-${var.environment}-public-${count.index + 1}"
    Type = "public"
  }
}

# Launch template with regional configuration
resource "aws_launch_template" "app" {
  name_prefix   = "${var.project_name}-${var.environment}-app-"
  image_id      = data.aws_ami.ubuntu.id
  instance_type = local.current_region_config.instance_types.app

  # Spot instances when configured
  instance_market_options {
    market_type = local.current_region_config.cost_config.spot_instances_enabled ? "spot" : null

    dynamic "spot_options" {
      for_each = local.current_region_config.cost_config.spot_instances_enabled ? [1] : []
      content {
        spot_instance_type = "one-time"
      }
    }
  }

  # Monitoring configuration
  monitoring {
    enabled = local.current_region_config.monitoring_config.detailed_monitoring
  }

  # Advanced EBS configuration
  block_device_mappings {
    device_name = "/dev/sda1"
    ebs {
      volume_size = 20
      volume_type = "gp3"
      encrypted   = local.current_region_config.compliance_config.encryption_required
      iops        = 3000
    }
  }

  user_data = base64encode(templatefile("${path.module}/userdata.sh", {
    region = var.aws_region
    environment = var.environment
    monitoring_enabled = local.current_region_config.monitoring_config.enabled
    log_retention = local.current_region_config.monitoring_config.log_retention_days
  }))

  tag_specifications {
    resource_type = "instance"
    tags = {
      Name = "${var.project_name}-${var.environment}-app"
      Region = var.aws_region
      CostOptimized = local.current_region_config.cost_config.spot_instances_enabled
    }
  }
}

# Auto Scaling group with regional configuration
resource "aws_autoscaling_group" "app" {
  name                = "${var.project_name}-${var.environment}-app-asg"
  vpc_zone_identifier = aws_subnet.public[*].id

  min_size         = local.current_region_config.scaling_config.min_size
  max_size         = local.current_region_config.scaling_config.max_size
  desired_capacity = local.current_region_config.scaling_config.desired_capacity

  launch_template {
    id      = aws_launch_template.app.id
    version = "$Latest"
  }

  # Auto shutdown for cost optimisation
  dynamic "tag" {
    for_each = local.current_region_config.cost_config.auto_shutdown ? [1] : []
    content {
      key                 = "AutoShutdown"
      value               = "true"
      propagate_at_launch = true
    }
  }

  tag {
    key                 = "Name"
    value               = "${var.project_name}-${var.environment}-app"
    propagate_at_launch = true
  }
}

Data structure comparison and use cases:

Data type Characteristics Performance Use case Debugging difficulty
Object Structure with named attributes High Configuration groups Low
Set Unique values, no order Medium Security groups, tags Medium
Map Key-value pairs High Environment configs Medium
List Ordered collection Low Sequential data Low

🔧 Practical example - choosing a data structure:


# Choose the right data structure
locals {
  # ✅ Object for related configuration
  database_config = {
    engine = "mysql"
    version = "8.0"
    storage = 100
  }

  # ✅ Set for unique values without order
  allowed_cidrs = toset([
    "10.0.0.0/16",
    "172.16.0.0/12",
    "192.168.0.0/16"
  ])

  # ✅ Map for key-value mappings
  environment_settings = {
    dev  = { instance_type = "t3.micro", count = 1 }
    prod = { instance_type = "t3.large", count = 3 }
  }

  # ✅ List for ordered data
  deployment_stages = [
    "build",
    "test",
    "deploy",
    "verify"
  ]
}

💡 Performance tip: Objects and maps have O(1) access via keys, while lists need O(n) for search.

Escape sequences and string interpolation

What are escape sequences and string interpolation? Escape sequences let you use special characters in strings. String interpolation makes configurations dynamic by embedding variables, functions and expressions directly in strings.

Why do they matter so much in Terraform? Terraform works heavily with strings — from resource names through template files to user-data scripts. Without correct string handling you get syntax errors and security holes.

What must you watch for with string operations? String interpolation can lead to injection vulnerabilities if it is not escaped properly. Complex string operations can also hurt Terraform performance.

What do you use advanced string features for? Dynamic configuration, template generation, JSON/YAML creation and complex naming.


# Advanced string operations
locals {
  # Base variables
  project_name = "ecommerce-platform"
  environment  = "production"
  region       = "us-east-1"

  # Advanced escape sequences
  json_config = jsonencode({
    # Strings with quotation marks
    message = "Welcome to \"${local.project_name}\" platform"

    # Linux paths for logs and configuration
    log_path = "/var/log/${local.project_name}/application.log"
    config_path = "/etc/${local.project_name}/config"
    data_path = "/opt/${local.project_name}/data"

    # Unix socket paths
    socket_path = "/var/run/${local.project_name}/${local.project_name}.sock"

    # Unicode characters
    company_name = "Acme Corp™"

    # Newlines and formatting
    welcome_text = <<-EOT
      Welcome to ${local.project_name}!
      Environment: ${local.environment}
      Region: ${local.region}

      For support, contact: support@company.com
    EOT
  })

  # Complex string interpolation
  resource_naming = {
    # Base naming
    prefix = "${local.project_name}-${local.environment}"

    # Conditional naming
    db_name = "${local.project_name}-${local.environment == "prod" ? "production" : "development"}-db"

    # Timestamp-based names
    backup_name = "${local.project_name}-backup-${formatdate("YYYY-MM-DD-hhmm", timestamp())}"

    # Hash-based unique names
    unique_bucket = "${local.project_name}-${local.environment}-${substr(md5("${local.project_name}-${local.environment}-${timestamp()}"), 0, 8)}"
  }

  # Template variables for different formats
  template_vars = {
    # Base information
    project_name = local.project_name
    environment  = local.environment
    region       = local.region

    # Conditional values
    debug_mode = local.environment != "prod"
    ssl_enabled = local.environment == "prod"

    # Computed values
    instance_count = local.environment == "prod" ? 3 : 1
    storage_size   = local.environment == "prod" ? 100 : 20

    # Formatted strings
    formatted_date = formatdate("YYYY-MM-DD", timestamp())
    formatted_time = formatdate("hh:mm:ss", timestamp())
  }
}

# Advanced heredoc syntax for complex templates
resource "aws_instance" "app" {
  count         = local.template_vars.instance_count
  ami           = data.aws_ami.ubuntu.id
  instance_type = local.environment == "prod" ? "t3.large" : "t3.micro"

  # Complex user data with advanced string features
  user_data = base64encode(templatefile("${path.module}/templates/userdata.sh", merge(local.template_vars, {
    instance_index = count.index
    instance_id    = count.index + 1
    total_instances = local.template_vars.instance_count
  })))

  tags = {
    Name = "${local.resource_naming.prefix}-app-${count.index + 1}"
    Role = "application"
  }
}

# Advanced JSON configuration for CloudWatch
resource "aws_cloudwatch_log_group" "app" {
  name              = "/aws/ec2/${local.resource_naming.prefix}/application"
  retention_in_days = local.environment == "prod" ? 90 : 7

  tags = {
    Name = "${local.resource_naming.prefix}-logs"
  }
}

# S3 bucket policy with complex string interpolation
resource "aws_s3_bucket_policy" "app_data" {
  bucket = aws_s3_bucket.app_data.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "DenyInsecureConnections"
        Effect = "Deny"
        Principal = "*"
        Action = "s3:*"
        Resource = [
          "${aws_s3_bucket.app_data.arn}",
          "${aws_s3_bucket.app_data.arn}/*"
        ]
        Condition = {
          Bool = {
            "aws:SecureTransport" = "false"
          }
        }
      },
      {
        Sid    = "AllowApplicationAccess"
        Effect = "Allow"
        Principal = {
          AWS = aws_iam_role.app_role.arn
        }
        Action = [
          "s3:GetObject",
          "s3:PutObject",
          "s3:DeleteObject"
        ]
        Resource = "${aws_s3_bucket.app_data.arn}/*"
      }
    ]
  })
}

userdata.sh template file:


#!/bin/bash
# Generated by Terraform on ${formatted_date} at ${formatted_time}
# Project: ${project_name}
# Environment: ${environment}
# Instance: ${instance_id}/${total_instances}

# Set environment variables
export PROJECT_NAME="${project_name}"
export ENVIRONMENT="${environment}"
export AWS_REGION="${region}"
export INSTANCE_ID="${instance_id}"
export DEBUG_MODE="${debug_mode}"
export SSL_ENABLED="${ssl_enabled}"

# Log everything
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

echo "Starting initialization for ${project_name} instance ${instance_id}/${total_instances}"

# Update system
apt-get update -y
apt-get upgrade -y

# Install required packages
apt-get install -y \
    curl \
    wget \
    unzip \
    jq \
    awscli \
    docker.io \
    docker-compose

# Configure Docker
systemctl enable docker
systemctl start docker
usermod -a -G docker ubuntu

# Create application directory
mkdir -p /opt/${project_name}
cd /opt/${project_name}

# Create application configuration
cat > config.json << 'EOF'
{
  "project_name": "${project_name}",
  "environment": "${environment}",
  "region": "${region}",
  "instance_id": "${instance_id}",
  "debug_mode": ${debug_mode},
  "ssl_enabled": ${ssl_enabled},
  "storage_size": ${storage_size},
  "database_config": {
    "host": "localhost",
    "port": 3306,
    "name": "${project_name}_${environment}"
  },
  "monitoring": {
    "enabled": true,
    "log_level": "${debug_mode ? "DEBUG" : "INFO"}",
    "metrics_interval": 60
  }
}
EOF

# Environment-specific configuration
%{ if environment == "prod" }
# Production-specific setup
echo "Setting up production environment..."

# SSL certificates
mkdir -p /etc/ssl/certs/${project_name}
# SSL setup would go here

# Performance tuning
echo "net.core.rmem_max = 134217728" >> /etc/sysctl.conf
echo "net.core.wmem_max = 134217728" >> /etc/sysctl.conf
sysctl -p

%{ else }
# Development/staging setup
echo "Setting up development environment..."

# Development tools
apt-get install -y \
    vim \
    htop \
    tree \
    git

# Relaxed security for development
echo "Development mode - relaxed security settings"
%{ endif }

# Install CloudWatch agent
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
dpkg -i amazon-cloudwatch-agent.deb

# Configure CloudWatch agent
cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json << 'EOF'
{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/user-data.log",
            "log_group_name": "/aws/ec2/${project_name}-${environment}/user-data",
            "log_stream_name": "{instance_id}/user-data"
          },
          {
            "file_path": "/opt/${project_name}/logs/application.log",
            "log_group_name": "/aws/ec2/${project_name}-${environment}/application",
            "log_stream_name": "{instance_id}/application"
          }
        ]
      }
    }
  },
  "metrics": {
    "namespace": "${project_name}/${environment}",
    "metrics_collected": {
      "cpu": {
        "measurement": [
          "cpu_usage_idle",
          "cpu_usage_iowait",
          "cpu_usage_user",
          "cpu_usage_system"
        ],
        "metrics_collection_interval": 60
      },
      "disk": {
        "measurement": [
          "used_percent"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "*"
        ]
      },
      "mem": {
        "measurement": [
          "mem_used_percent"
        ],
        "metrics_collection_interval": 60
      }
    }
  }
}
EOF

# Start CloudWatch agent
systemctl enable amazon-cloudwatch-agent
systemctl start amazon-cloudwatch-agent

# Final status
echo "Initialization completed for ${project_name} instance ${instance_id}/${total_instances}"
echo "Environment: ${environment}"
echo "Debug mode: ${debug_mode}"
echo "SSL enabled: ${ssl_enabled}"
echo "Timestamp: $(date)"

Common string interpolation errors:

Problem Symptom Solution
Unescaped quotation marks Error: Invalid JSON Use \"
Wrong template syntax Error: Invalid template ${} for variables
Circular reference Error: Cycle detected Check dependencies
Type mismatch Error: Invalid value Explicit conversion

Comments and code formatting

Why are comments decisive in infrastructure code? Terraform configurations are often managed by teams and maintained for years. Good comments explain not only what is being done but also why particular decisions were made. That matters especially for complex network configurations or security settings.

What should you watch for with comments? Comments can go stale and then mislead. Keep them current and focus on the "why" rather than the "what". The "what" is visible from the code.

What do you use different comment styles for? Single-line comments for short explanations, multi-line for complex architecture decisions, and documentation blocks for modules and important resources.


# ============================================================================
# MULTI-TIER ARCHITECTURE CONFIGURATION
# ============================================================================
# This configuration implements a three-tier architecture pattern:
# 1. Presentation Tier (Public Subnets) - Load Balancers, Bastion Hosts
# 2. Application Tier (Private Subnets) - Application Servers, APIs
# 3. Data Tier (Database Subnets) - RDS, ElastiCache, Data Storage
#
# Architecture Decision: We use separate subnets for each tier to implement
# defense-in-depth security and enable granular network access controls.
#
# Last Updated: 2024-01-15
# Author: DevOps Team
# Review Date: 2024-04-15
# ============================================================================

# VPC Configuration
# Why /16 CIDR: Provides 65,536 IP addresses, sufficient for enterprise growth
# Why enable_dns_hostnames: Required for RDS endpoint resolution
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr  # Default: 10.0.0.0/16
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.project_name}-vpc"
    # Important: This tag is used by AWS Load Balancer Controller
    "kubernetes.io/cluster/${var.project_name}" = "shared"
  }
}

# ============================================================================
# PRESENTATION TIER (PUBLIC SUBNETS)
# ============================================================================
# Public subnets host internet-facing resources like:
# - Application Load Balancers
# - NAT Gateways
# - Bastion Hosts (if needed)
#
# Security Consideration: Only resources that need direct internet access
# should be placed here. Application servers go in private subnets.
# ============================================================================

# Public Subnets - One per Availability Zone
# Why count.index: Ensures subnets are distributed across AZs for high availability
# Why cidrsubnet: Automatically calculates subnet CIDR blocks
resource "aws_subnet" "public" {
  count = length(var.availability_zones)

  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone       = var.availability_zones[count.index]
  map_public_ip_on_launch = true  # Required for NAT Gateway EIPs

  tags = {
    Name = "${var.project_name}-public-${count.index + 1}"
    Type = "public"
    Tier = "presentation"
    # Kubernetes requires this tag for subnet discovery
    "kubernetes.io/role/elb" = "1"
  }
}

# Internet Gateway
# Why dependency: VPC must exist before IGW attachment
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "${var.project_name}-igw"
  }
}

# Public Route Table
# Why single route table: All public subnets need identical routing
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  # Route all traffic to Internet Gateway
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }

  tags = {
    Name = "${var.project_name}-public-rt"
  }
}

# Associate public subnets with route table
resource "aws_route_table_association" "public" {
  count = length(aws_subnet.public)

  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public.id
}

# ============================================================================
# APPLICATION TIER (PRIVATE SUBNETS)
# ============================================================================
# Private subnets host application servers and internal services:
# - Web servers (behind ALB)
# - API servers
# - Background workers
# - Internal services
#
# Security Consideration: These resources cannot be directly accessed from
# the internet. They receive traffic through the load balancer and can
# initiate outbound connections through NAT Gateway.
# ============================================================================

# Private Subnets - Application Tier
# Why separate from public: Defense-in-depth security architecture
# Why +10 offset: Avoids IP conflicts with public subnets
resource "aws_subnet" "private_app" {
  count = length(var.availability_zones)

  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + 10)
  availability_zone = var.availability_zones[count.index]

  tags = {
    Name = "${var.project_name}-private-app-${count.index + 1}"
    Type = "private"
    Tier = "application"
    # Kubernetes requires this tag for internal load balancers
    "kubernetes.io/role/internal-elb" = "1"
  }
}

# ============================================================================
# DATA TIER (DATABASE SUBNETS)
# ============================================================================
# Database subnets are the most restrictive tier:
# - RDS instances
# - ElastiCache clusters
# - Data storage services
#
# Security Consideration: These subnets have no internet access whatsoever.
# They only communicate with application tier through specific ports.
# ============================================================================

# Database Subnets - Most restrictive tier
# Why +20 offset: Clear separation from other tiers
# Why no internet access: Databases should never communicate with internet
resource "aws_subnet" "private_db" {
  count = length(var.availability_zones)

  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + 20)
  availability_zone = var.availability_zones[count.index]

  tags = {
    Name = "${var.project_name}-private-db-${count.index + 1}"
    Type = "private"
    Tier = "data"
  }
}

# ============================================================================
# NAT GATEWAY CONFIGURATION
# ============================================================================
# NAT Gateways enable outbound internet access for private subnets
#
# Architecture Decision: One NAT Gateway per AZ for high availability
# Cost Consideration: NAT Gateways are expensive. For cost optimization,
# consider using a single NAT Gateway (less resilient but cheaper).
# ============================================================================

# Elastic IPs for NAT Gateways
# Why depends_on: EIPs require IGW to be attached first
resource "aws_eip" "nat" {
  count = var.multi_az_nat ? length(var.availability_zones) : 1

  domain = "vpc"
  depends_on = [aws_internet_gateway.main]

  tags = {
    Name = "${var.project_name}-nat-eip-${count.index + 1}"
  }
}

# NAT Gateways
# Why public subnet: NAT Gateway needs internet access to route traffic
# Why conditional count: Supports both single and multi-AZ configurations
resource "aws_nat_gateway" "main" {
  count = var.multi_az_nat ? length(var.availability_zones) : 1

  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id

  depends_on = [aws_internet_gateway.main]

  tags = {
    Name = "${var.project_name}-nat-${count.index + 1}"
  }
}

# ============================================================================
# PRIVATE SUBNET ROUTING
# ============================================================================
# Private subnets need different routing tables because:
# 1. They route internet traffic through NAT Gateway
# 2. Each AZ may have its own NAT Gateway for high availability
# 3. Database subnets may have different routing rules
# ============================================================================

# Route Tables for Private Application Subnets
# Why per-AZ: Each AZ routes through its own NAT Gateway
resource "aws_route_table" "private_app" {
  count = length(var.availability_zones)

  vpc_id = aws_vpc.main.id

  # Route internet traffic through NAT Gateway
  # Why conditional: Supports both single and multi-AZ NAT configurations
  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.main[var.multi_az_nat ? count.index : 0].id
  }

  tags = {
    Name = "${var.project_name}-private-app-rt-${count.index + 1}"
  }
}

# Associate private app subnets with route tables
resource "aws_route_table_association" "private_app" {
  count = length(aws_subnet.private_app)

  subnet_id      = aws_subnet.private_app[count.index].id
  route_table_id = aws_route_table.private_app[count.index].id
}

# Route Tables for Database Subnets
# Why separate: Database subnets may need different routing rules
# Why no internet route: Databases should never access internet directly
resource "aws_route_table" "private_db" {
  count = length(var.availability_zones)

  vpc_id = aws_vpc.main.id

  # No internet route - databases stay internal
  # Future: Add routes for VPC endpoints, peering connections

  tags = {
    Name = "${var.project_name}-private-db-rt-${count.index + 1}"
  }
}

# Associate database subnets with route tables
resource "aws_route_table_association" "private_db" {
  count = length(aws_subnet.private_db)

  subnet_id      = aws_subnet.private_db[count.index].id
  route_table_id = aws_route_table.private_db[count.index].id
}

# ============================================================================
# SECURITY GROUPS
# ============================================================================
# Security groups implement stateful firewall rules at the instance level
# Following principle of least privilege - only allow required traffic
# ============================================================================

# Application Load Balancer Security Group
# Why port 80/443: Standard web traffic
# Why 0.0.0.0/0: Internet-facing load balancer needs public access
resource "aws_security_group" "alb" {
  name_prefix = "${var.project_name}-alb-"
  description = "Security group for Application Load Balancer"
  vpc_id      = aws_vpc.main.id

  # HTTP ingress from internet
  ingress {
    description = "HTTP from internet"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # HTTPS ingress from internet
  ingress {
    description = "HTTPS from internet"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # All outbound traffic (for health checks)
  egress {
    description = "All outbound traffic"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "${var.project_name}-alb-sg"
  }

  # Prevent accidental deletion
  lifecycle {
    create_before_destroy = true
  }
}

# Application Servers Security Group
# Why reference ALB SG: Only allow traffic from load balancer
# Why port 8080: Common application port (adjust as needed)
resource "aws_security_group" "app_servers" {
  name_prefix = "${var.project_name}-app-"
  description = "Security group for application servers"
  vpc_id      = aws_vpc.main.id

  # HTTP from ALB only
  ingress {
    description     = "HTTP from ALB"
    from_port       = 8080
    to_port         = 8080
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }

  # SSH from bastion host (if exists)
  # TODO: Implement bastion host security group
  dynamic "ingress" {
    for_each = var.enable_bastion ? [1] : []
    content {
      description = "SSH from bastion"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = [var.vpc_cidr]  # Only from VPC
    }
  }

  # All outbound traffic (for package updates, API calls)
  egress {
    description = "All outbound traffic"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "${var.project_name}-app-sg"
  }

  lifecycle {
    create_before_destroy = true
  }
}

# Database Security Group
# Why port 3306: MySQL/MariaDB default port
# Why app servers only: Databases should only accept app connections
resource "aws_security_group" "database" {
  name_prefix = "${var.project_name}-db-"
  description = "Security group for database servers"
  vpc_id      = aws_vpc.main.id

  # MySQL/MariaDB from app servers only
  ingress {
    description     = "MySQL from app servers"
    from_port       = 3306
    to_port         = 3306
    protocol        = "tcp"
    security_groups = [aws_security_group.app_servers.id]
  }

  # PostgreSQL support (if needed)
  dynamic "ingress" {
    for_each = var.database_engine == "postgres" ? [1] : []
    content {
      description     = "PostgreSQL from app servers"
      from_port       = 5432
      to_port         = 5432
      protocol        = "tcp"
      security_groups = [aws_security_group.app_servers.id]
    }
  }

  # No outbound rules needed for databases
  # RDS manages its own outbound communication

  tags = {
    Name = "${var.project_name}-db-sg"
  }

  lifecycle {
    create_before_destroy = true
  }
}

/*
 * ARCHITECTURE SUMMARY
 * ====================
 * This configuration creates a secure, scalable three-tier architecture:
 *
 * 1. Public Subnets (Presentation Tier)
 *    - Hosts internet-facing load balancers
 *    - Has direct internet access via Internet Gateway
 *    - Routes: 0.0.0.0/0 -> Internet Gateway
 *
 * 2. Private App Subnets (Application Tier)
 *    - Hosts application servers and APIs
 *    - Internet access via NAT Gateway for updates
 *    - Routes: 0.0.0.0/0 -> NAT Gateway
 *
 * 3. Private DB Subnets (Data Tier)
 *    - Hosts databases and data storage
 *    - No internet access at all
 *    - Routes: Local traffic only
 *
 * Security Groups implement defense-in-depth:
 * - ALB: Accepts internet traffic on 80/443
 * - App Servers: Only accept traffic from ALB
 * - Database: Only accepts traffic from app servers
 *
 * High Availability:
 * - Resources deployed across multiple AZs
 * - Optional multi-AZ NAT Gateways
 * - Database subnets ready for Multi-AZ RDS
 */

🔧 Practical example - code formatting:


# Before terraform fmt (poorly formatted)
resource "aws_security_group" "web" {
name="web-sg"
vpc_id=aws_vpc.main.id
ingress{
from_port=80
to_port=80
protocol="tcp"
cidr_blocks=["0.0.0.0/0"]
}
}

# After terraform fmt (well formatted)
resource "aws_security_group" "web" {
  name   = "web-sg"
  vpc_id = aws_vpc.main.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Comment best practices:

Comment type Use Example
Single-line (#) Short explanations # SSH access from bastion
Multi-line (/\ \/) Architecture documentation Block comments
Header blocks Section separation # ====== SECTION ======
Inline Specific values port = 8080 # Application port
TODO/FIXME Future changes # TODO: Implement bastion host

Formatting tips:

  • Run terraform fmt regularly for consistent formatting
  • Use meaningful indentation for better readability
  • Group related resources with comment blocks
  • Document architecture decisions in detail

Comment pitfalls:

  • Stale comments are worse than no comments
  • Too many comments make the code unreadable
  • Comments should explain the "why", not the "what"
  • Never put secrets or passwords in comments

Debugging techniques for complex syntax:


# Validate Terraform configuration
terraform validate

# Check formatting
terraform fmt -check -diff

# Syntax highlighting in vim
vim -c "syntax on" -c "set ft=terraform" main.tf

# Debug local values
terraform console
> local.current_config
> local.resource_naming.prefix

# Test template functions
terraform console
> templatefile("userdata.sh", {var = "test"})

Advanced HCL syntax lets you create complex, maintainable Terraform configurations. With objects, sets and maps you structure data logically. String interpolation and templates make configurations dynamic. Good comments and formatting keep the work maintainable in a team.

Advanced variables and outputs

What makes advanced variables and outputs? The fundamentals covered simple variable definitions. Next come advanced features such as complex validation, sensitive data handling and conditional outputs. Those functions turn Terraform configuration from static definitions into intelligent, self-validating infrastructure templates.

Why are advanced variable features decisive? Without robust validation and structured outputs you get runtime errors that can be catastrophic in production. Advanced features catch errors at plan time and make infrastructure more predictable and safer.

What must you watch for with advanced variables? Over-complex validation can hurt performance and make the configuration hard to understand. Sensitive variables need extra care with state files and logs.

What do you use advanced variable features for? Enterprise environments where compliance requires strict validation, multi-team projects with complex dependencies, and infrastructures that must manage sensitive data.

Input variables with complex validation

What are complex validations? Validations go beyond simple type checks. They verify business logic, compliance requirements and technical constraints. You can validate regex patterns, ranges, list contents and even external API responses.

Why do validations matter so much? They prevent expensive production mistakes. A wrong CIDR block configuration can cause network outages. Invalid instance types can cause performance problems. Validations catch these issues before apply.

What should you watch for with validations? Validations run on every plan and apply. Complex validations can hurt performance. Error messages should be understandable and action-oriented.

What do you use complex validations for? Compliance checks, security validations, cost control and technical constraints.

🔧 Practical example - advanced variable validation:


# Advanced network configuration with complex validations
variable "network_config" {
  description = "Complete network configuration with validation"
  type = object({
    vpc_cidr = string
    environment = string
    availability_zones = list(string)
    public_subnets = list(string)
    private_subnets = list(string)
    database_subnets = list(string)
    enable_nat_gateway = bool
    enable_vpn_gateway = bool
    dns_domain = string
    tags = map(string)
  })

  # CIDR block validation
  validation {
    condition = can(cidrhost(var.network_config.vpc_cidr, 0))
    error_message = "VPC CIDR must be a valid IPv4 CIDR block (e.g., 10.0.0.0/16)."
  }

  # CIDR size validation
  validation {
    condition = tonumber(split("/", var.network_config.vpc_cidr)[1]) >= 16 && tonumber(split("/", var.network_config.vpc_cidr)[1]) <= 24
    error_message = "VPC CIDR netmask must be between /16 and /24 for proper subnet allocation."
  }

  # Private IP range validation (RFC 1918)
  validation {
    condition = can(regex("^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)", var.network_config.vpc_cidr))
    error_message = "VPC CIDR must use private IP ranges (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16)."
  }

  # Environment validation
  validation {
    condition = contains(["dev", "staging", "prod", "sandbox"], var.network_config.environment)
    error_message = "Environment must be one of: dev, staging, prod, sandbox."
  }

  # Availability zone validation
  validation {
    condition = length(var.network_config.availability_zones) >= 2 && length(var.network_config.availability_zones) <= 6
    error_message = "Must specify between 2 and 6 availability zones for high availability."
  }

  # Subnet count validation
  validation {
    condition = length(var.network_config.public_subnets) == length(var.network_config.availability_zones)
    error_message = "Number of public subnets must match number of availability zones."
  }

  # Subnet CIDR validation
  validation {
    condition = alltrue([
      for subnet in var.network_config.public_subnets :
      can(cidrsubnet(var.network_config.vpc_cidr, 4, 0)) &&
      can(cidrhost(subnet, 0))
    ])
    error_message = "All public subnet CIDRs must be valid and within VPC CIDR range."
  }

  # DNS domain validation
  validation {
    condition = can(regex("^[a-z0-9.-]+\\.[a-z]{2,}$", var.network_config.dns_domain))
    error_message = "DNS domain must be a valid domain name (e.g., example.com)."
  }

  # Tags validation
  validation {
    condition = alltrue([
      for key, value in var.network_config.tags :
      can(regex("^[A-Za-z0-9._-]{1,128}$", key)) &&
      can(regex("^[A-Za-z0-9._\\s-]{1,256}$", value))
    ])
    error_message = "Tag keys and values must follow AWS naming conventions."
  }
}

# Advanced compute configuration with business-logic validation
variable "compute_config" {
  description = "Compute configuration with business logic validation"
  type = object({
    instance_types = map(string)
    min_instances = number
    max_instances = number
    desired_instances = number
    auto_scaling_enabled = bool
    spot_instances_enabled = bool
    spot_max_price = string
    health_check_type = string
    health_check_grace_period = number
    termination_policies = list(string)
  })

  # Instance type validation
  validation {
    condition = alltrue([
      for type in values(var.compute_config.instance_types) :
      can(regex("^[a-z][0-9]+[a-z]*\\.(nano|micro|small|medium|large|xlarge|[0-9]+xlarge)$", type))
    ])
    error_message = "Instance types must be valid AWS instance types (e.g., t3.micro, m5.large)."
  }

  # Scaling limits validation
  validation {
    condition = var.compute_config.min_instances <= var.compute_config.desired_instances && var.compute_config.desired_instances <= var.compute_config.max_instances
    error_message = "Scaling configuration must follow: min_instances <= desired_instances <= max_instances."
  }

  # Spot price validation
  validation {
    condition = var.compute_config.spot_instances_enabled ? can(tonumber(var.compute_config.spot_max_price)) && tonumber(var.compute_config.spot_max_price) > 0 : true
    error_message = "Spot max price must be a positive number when spot instances are enabled."
  }

  # Health check validation
  validation {
    condition = contains(["EC2", "ELB"], var.compute_config.health_check_type)
    error_message = "Health check type must be 'EC2' or 'ELB'."
  }

  # Termination policy validation
  validation {
    condition = alltrue([
      for policy in var.compute_config.termination_policies :
      contains(["OldestInstance", "NewestInstance", "OldestLaunchConfiguration", "ClosestToNextInstanceHour", "Default"], policy)
    ])
    error_message = "Termination policies must be valid AWS Auto Scaling termination policies."
  }
}

# Advanced database configuration with compliance validation
variable "database_config" {
  description = "Database configuration with compliance validation"
  type = object({
    engine = string
    engine_version = string
    instance_class = string
    allocated_storage = number
    max_storage = number
    storage_type = string
    storage_encrypted = bool
    kms_key_id = string
    backup_retention_period = number
    backup_window = string
    maintenance_window = string
    multi_az = bool
    publicly_accessible = bool
    deletion_protection = bool
    performance_insights_enabled = bool
    monitoring_interval = number
    auto_minor_version_upgrade = bool
    parameter_group_family = string
    option_group_name = string
  })

  # Engine validation
  validation {
    condition = contains(["mysql", "postgres", "mariadb", "oracle-ee", "oracle-se2", "sqlserver-ee", "sqlserver-se", "sqlserver-ex", "sqlserver-web"], var.database_config.engine)
    error_message = "Database engine must be a supported RDS engine."
  }

  # Storage validation
  validation {
    condition = var.database_config.allocated_storage >= 20 && var.database_config.allocated_storage <= 65536
    error_message = "Allocated storage must be between 20 GB and 65,536 GB."
  }

  # Storage type validation
  validation {
    condition = contains(["gp2", "gp3", "io1", "io2", "magnetic"], var.database_config.storage_type)
    error_message = "Storage type must be gp2, gp3, io1, io2, or magnetic."
  }

  # Backup retention validation
  validation {
    condition = var.database_config.backup_retention_period >= 0 && var.database_config.backup_retention_period <= 35
    error_message = "Backup retention period must be between 0 and 35 days."
  }

  # Backup window format validation
  validation {
    condition = can(regex("^[0-2][0-9]:[0-5][0-9]-[0-2][0-9]:[0-5][0-9]$", var.database_config.backup_window))
    error_message = "Backup window must be in format HH:MM-HH:MM (e.g., 03:00-04:00)."
  }

  # Maintenance window format validation
  validation {
    condition = can(regex("^(sun|mon|tue|wed|thu|fri|sat):[0-2][0-9]:[0-5][0-9]-(sun|mon|tue|wed|thu|fri|sat):[0-2][0-9]:[0-5][0-9]$", var.database_config.maintenance_window))
    error_message = "Maintenance window must be in format ddd:HH:MM-ddd:HH:MM (e.g., sun:04:00-sun:05:00)."
  }

  # Compliance validation: encryption required for production
  validation {
    condition = var.database_config.storage_encrypted == true
    error_message = "Storage encryption is required for all database instances for compliance."
  }

  # Compliance validation: deletion protection for production
  validation {
    condition = var.database_config.deletion_protection == true
    error_message = "Deletion protection must be enabled for all database instances."
  }

  # Compliance validation: no public access
  validation {
    condition = var.database_config.publicly_accessible == false
    error_message = "Database instances must not be publicly accessible for security compliance."
  }

  # Monitoring interval validation
  validation {
    condition = contains([0, 1, 5, 10, 15, 30, 60], var.database_config.monitoring_interval)
    error_message = "Monitoring interval must be 0, 1, 5, 10, 15, 30, or 60 seconds."
  }
}

# Security configuration with advanced validations
variable "security_config" {
  description = "Security configuration with advanced validation"
  type = object({
    enable_flow_logs = bool
    enable_cloudtrail = bool
    enable_config = bool
    enable_guardduty = bool
    security_group_rules = list(object({
      type = string
      from_port = number
      to_port = number
      protocol = string
      cidr_blocks = list(string)
      description = string
    }))
    kms_key_rotation_enabled = bool
    password_policy = object({
      minimum_password_length = number
      require_lowercase_characters = bool
      require_uppercase_characters = bool
      require_numbers = bool
      require_symbols = bool
      max_password_age = number
    })
  })

  # Security group rules validation
  validation {
    condition = alltrue([
      for rule in var.security_config.security_group_rules :
      contains(["ingress", "egress"], rule.type) &&
      rule.from_port >= 0 && rule.from_port <= 65535 &&
      rule.to_port >= 0 && rule.to_port <= 65535 &&
      rule.from_port <= rule.to_port
    ])
    error_message = "Security group rules must have valid types, ports, and port ranges."
  }

  # CIDR blocks validation for security groups
  validation {
    condition = alltrue([
      for rule in var.security_config.security_group_rules :
      alltrue([
        for cidr in rule.cidr_blocks :
        can(cidrhost(cidr, 0))
      ])
    ])
    error_message = "All CIDR blocks in security group rules must be valid."
  }

  # Dangerous ports validation
  validation {
    condition = alltrue([
      for rule in var.security_config.security_group_rules :
      !(rule.type == "ingress" && contains(rule.cidr_blocks, "0.0.0.0/0") &&
        (rule.from_port <= 22 && rule.to_port >= 22 ||
         rule.from_port <= 3389 && rule.to_port >= 3389))
    ])
    error_message = "SSH (22) and RDP (3389) ports must not be open to 0.0.0.0/0 for security."
  }

  # Password policy validation
  validation {
    condition = var.security_config.password_policy.minimum_password_length >= 8 && var.security_config.password_policy.minimum_password_length <= 128
    error_message = "Password minimum length must be between 8 and 128 characters."
  }

  # Password complexity validation
  validation {
    condition = var.security_config.password_policy.require_lowercase_characters && var.security_config.password_policy.require_uppercase_characters && var.security_config.password_policy.require_numbers && var.security_config.password_policy.require_symbols
    error_message = "Password policy must require all character types for security compliance."
  }
}

Validation strategies:

Validation type Use Performance Complexity
Type validation Base data types Very high Low
Regex validation Format check High Medium
Range validation Numeric ranges Very high Low
List validation Allowed values High Low
Business logic Complex rules Medium High
Cross-field Dependencies Medium High

💡 Validation optimisation: Use simple validations first, then complex ones. That improves performance and makes error messages easier to understand.

Advanced validation functions:


# Conditional validation based on other variables
locals {
  is_production = var.environment == "prod"
  is_development = var.environment == "dev"
}

variable "monitoring_config" {
  description = "Monitoring configuration with conditional validation"
  type = object({
    enabled = bool
    retention_days = number
    detailed_monitoring = bool
    alert_endpoints = list(string)
  })

  # Conditional validation: monitoring required for production
  validation {
    condition = var.environment == "prod" ? var.monitoring_config.enabled : true
    error_message = "Monitoring must be enabled for production environments."
  }

  # Conditional validation: retention for production
  validation {
    condition = var.environment == "prod" ? var.monitoring_config.retention_days >= 90 : var.monitoring_config.retention_days >= 7
    error_message = "Production environments require minimum 90 days retention, others require minimum 7 days."
  }

  # Email format validation
  validation {
    condition = alltrue([
      for email in var.monitoring_config.alert_endpoints :
      can(regex("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$", email))
    ])
    error_message = "All alert endpoints must be valid email addresses."
  }
}

# Cross-variable validation
variable "scaling_config" {
  description = "Auto scaling configuration"
  type = object({
    min_size = number
    max_size = number
    desired_capacity = number
    target_cpu_utilization = number
  })

  # Cross-field validation
  validation {
    condition = var.scaling_config.min_size <= var.scaling_config.desired_capacity && var.scaling_config.desired_capacity <= var.scaling_config.max_size
    error_message = "Scaling configuration must follow: min_size <= desired_capacity <= max_size."
  }

  # Business-logic validation
  validation {
    condition = var.scaling_config.max_size <= (var.environment == "prod" ? 100 : 10)
    error_message = "Production environments can scale to 100 instances, others are limited to 10."
  }
}

Validation pitfalls:

Problem Symptom Solution
Slow validation Plan takes a long time Simple checks first
Circular references Validation error Check dependencies
Unclear error messages Confusion on errors Specific messages
Over-validation Hard to use Only necessary checks

Sensitive variables and credential handling

What are sensitive variables? Sensitive variables hold confidential information such as passwords, API keys or certificates. Terraform treats them specially: they are not shown in plan output, not written to logs, and obscured in the state file.

Why is correct credential handling critical? Badly handled credentials become security holes. Passwords in plan output, API keys in logs or unencrypted secrets in state files are common weaknesses.

What must you watch for with sensitive variables? Sensitive variables are only "sensitive" inside Terraform, not in the real world. State files must be encrypted, logs must be handled securely, and team access must be controlled.

What do you use sensitive variables for? Passwords, API keys, certificates, tokens and any other confidential information used in the infrastructure.


# Advanced sensitive variable configuration
variable "database_credentials" {
  description = "Database credentials - marked as sensitive"
  type = object({
    master_username = string
    master_password = string
    replica_username = string
    replica_password = string
  })
  sensitive = true

  validation {
    condition = length(var.database_credentials.master_password) >= 12
    error_message = "Master password must be at least 12 characters long."
  }

  validation {
    condition = can(regex("^[A-Za-z0-9!@#$%^&*()_+=-]+$", var.database_credentials.master_password))
    error_message = "Password must contain only allowed characters."
  }
}

# API keys and tokens
variable "external_api_keys" {
  description = "External API keys and tokens"
  type = object({
    github_token = string
    docker_registry_token = string
    monitoring_api_key = string
    backup_service_key = string
  })
  sensitive = true

  validation {
    condition = alltrue([
      for key in values(var.external_api_keys) :
      length(key) >= 8 && length(key) <= 256
    ])
    error_message = "API keys must be between 8 and 256 characters long."
  }
}

# SSL certificates
variable "ssl_certificates" {
  description = "SSL certificates and private keys"
  type = object({
    certificate_body = string
    private_key = string
    certificate_chain = string
  })
  sensitive = true

  validation {
    condition = can(regex("^-----BEGIN CERTIFICATE-----", var.ssl_certificates.certificate_body))
    error_message = "Certificate body must be a valid PEM-encoded certificate."
  }

  validation {
    condition = can(regex("^-----BEGIN (RSA )?PRIVATE KEY-----", var.ssl_certificates.private_key))
    error_message = "Private key must be a valid PEM-encoded private key."
  }
}

# Encryption keys
variable "encryption_keys" {
  description = "Encryption keys for various services"
  type = object({
    application_secret_key = string
    session_encryption_key = string
    database_encryption_key = string
  })
  sensitive = true

  validation {
    condition = alltrue([
      for key in values(var.encryption_keys) :
      length(key) >= 32
    ])
    error_message = "Encryption keys must be at least 32 characters long."
  }
}

Secure credential passing:


# Local sensitive values with calculations
locals {
  # Secure password generation
  generated_passwords = {
    db_master = random_password.db_master.result
    db_replica = random_password.db_replica.result
    app_secret = random_password.app_secret.result
  }

  # Secure combinations
  database_connection_strings = {
    master = "postgresql://${var.database_credentials.master_username}:${local.generated_passwords.db_master}@${aws_db_instance.master.endpoint}:5432/${aws_db_instance.master.name}"
    replica = "postgresql://${var.database_credentials.replica_username}:${local.generated_passwords.db_replica}@${aws_db_instance.replica.endpoint}:5432/${aws_db_instance.replica.name}"
  }
}

# Secure password generation
resource "random_password" "db_master" {
  length  = 16
  special = true
  numeric = true
  upper   = true
  lower   = true
}

resource "random_password" "db_replica" {
  length  = 16
  special = true
  numeric = true
  upper   = true
  lower   = true
}

resource "random_password" "app_secret" {
  length  = 32
  special = true
  numeric = true
  upper   = true
  lower   = true
}

# AWS Secrets Manager integration
resource "aws_secretsmanager_secret" "database_credentials" {
  name                    = "${var.project_name}-${var.environment}-db-credentials"
  description             = "Database credentials for ${var.project_name}"
  recovery_window_in_days = 7

  tags = {
    Name = "${var.project_name}-db-credentials"
    Environment = var.environment
  }
}

resource "aws_secretsmanager_secret_version" "database_credentials" {
  secret_id = aws_secretsmanager_secret.database_credentials.id
  secret_string = jsonencode({
    master_username = var.database_credentials.master_username
    master_password = random_password.db_master.result
    replica_username = var.database_credentials.replica_username
    replica_password = random_password.db_replica.result
    connection_strings = local.database_connection_strings
  })
}

# KMS key for encryption
resource "aws_kms_key" "secrets" {
  description             = "KMS key for ${var.project_name} secrets"
  deletion_window_in_days = 7
  enable_key_rotation     = true

  tags = {
    Name = "${var.project_name}-secrets-key"
    Environment = var.environment
  }
}

resource "aws_kms_alias" "secrets" {
  name          = "alias/${var.project_name}-${var.environment}-secrets"
  target_key_id = aws_kms_key.secrets.key_id
}

# RDS instance with Secrets Manager
resource "aws_db_instance" "master" {
  identifier = "${var.project_name}-${var.environment}-master"

  engine         = "postgres"
  engine_version = "13.7"
  instance_class = "db.t3.micro"

  allocated_storage     = 20
  max_allocated_storage = 100
  storage_type         = "gp2"
  storage_encrypted    = true
  kms_key_id          = aws_kms_key.secrets.arn

  # Credentials from Secrets Manager
  manage_master_user_password = true
  master_user_secret_kms_key_id = aws_kms_key.secrets.arn

  db_name  = replace("${var.project_name}_${var.environment}", "-", "_")
  username = var.database_credentials.master_username

  # Secure network configuration
  db_subnet_group_name   = aws_db_subnet_group.main.name
  vpc_security_group_ids = [aws_security_group.database.id]
  publicly_accessible    = false

  # Backup configuration
  backup_retention_period = 30
  backup_window          = "03:00-04:00"
  maintenance_window     = "sun:04:00-sun:05:00"

  # Monitoring
  monitoring_interval = 60
  monitoring_role_arn = aws_iam_role.rds_monitoring.arn

  # Protection against deletion
  deletion_protection = true
  skip_final_snapshot = false
  final_snapshot_identifier = "${var.project_name}-${var.environment}-final-snapshot"

  tags = {
    Name = "${var.project_name}-${var.environment}-master"
    Environment = var.environment
  }
}

Credential-handling strategies:


┌─────────────────────────────────────────────────────────────┐
│                CREDENTIAL HANDLING WORKFLOW                 │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  1. INPUT (Sensitive Variables)                             │
│     • Environment Variables & TF Cloud Workspaces           │
│     • HashiCorp Vault & AWS Secrets Manager                 │
│                                                             │
│  2. PROCESSING (Terraform Core)                             │
│     • Marked as sensitive (masked in plan output)           │
│     • Encrypted hand-off and exclusion from logs            │
│                                                             │
│  3. STORAGE (State Backend)                                 │
│     • Encryption at Rest & in Transit (TLS)                 │
│     • Strict access control and audit logging               │
│                                                             │
│  4. USAGE (target resources)                                │
│     • Pass to cloud resources via secrets                   │
│     • Automated secret rotation and least privilege         │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Best practices for credential handling:

Aspect Recommendation Reason
Input Environment variables or Vault Do not store in code
Transfer TLS-encrypted Protection against eavesdropping
Storage Encrypted state backend Protection of state files
Use Secrets Manager/Vault Central management
Rotation Automated Regular renewal
Access Least privilege Minimal permissions

🔧 Practical example - environment-variable setup:


# Secure environment-variable configuration
export TF_VAR_database_credentials='{
  "master_username": "dbadmin",
  "master_password": "super-secure-password-123!",
  "replica_username": "readonly",
  "replica_password": "another-secure-password-456!"
}'

export TF_VAR_external_api_keys='{
  "github_token": "ghp_abcdef1234567890",
  "docker_registry_token": "dckr_pat_abcdef1234567890",
  "monitoring_api_key": "mon_abcdef1234567890",
  "backup_service_key": "bck_abcdef1234567890"
}'

# Run Terraform with secure variables
terraform plan
terraform apply

⚠️ Security tip: Never use real credentials in examples or tests. Use placeholders or test credentials.

Sensitive-variable pitfalls:

Problem Risk Solution
Unencrypted state Credentials in plaintext Encrypted backend
Logs contain secrets Credential leakage Log filtering
Plan output shows secrets Accidental disclosure Sensitive marking
Hardcoded credentials Source-code leakage Environment variables

Output values with conditional logic

What are advanced output features? Outputs can do more than return values. They can contain conditional logic, transform values, be marked as sensitive and emit complex data structures. That makes them powerful tools for modules and automation.

Why do advanced outputs matter? They make Terraform modules more flexible and reusable. Conditional outputs let you return different information based on the configuration. That reduces the number of module variants you need.

What must you watch for with advanced outputs? Complex output logic can be hard to debug. Sensitive outputs must be marked correctly to prevent credential leakage. Performance can suffer with complex calculations.

What do you use advanced outputs for? Multi-environment modules, conditional resource information, complex data structures and integration with other tools.


# Advanced output configuration with conditional logic
output "network_configuration" {
  description = "Complete network configuration details"
  value = {
    # Base information
    vpc_id = aws_vpc.main.id
    vpc_cidr = aws_vpc.main.cidr_block

    # Conditional subnet information
    public_subnets = var.network_config.public_subnets != [] ? {
      ids = aws_subnet.public[*].id
      cidrs = aws_subnet.public[*].cidr_block
      availability_zones = aws_subnet.public[*].availability_zone
    } : null

    private_subnets = var.network_config.private_subnets != [] ? {
      ids = aws_subnet.private_app[*].id
      cidrs = aws_subnet.private_app[*].cidr_block
      availability_zones = aws_subnet.private_app[*].availability_zone
    } : null

    database_subnets = var.network_config.database_subnets != [] ? {
      ids = aws_subnet.private_db[*].id
      cidrs = aws_subnet.private_db[*].cidr_block
      availability_zones = aws_subnet.private_db[*].availability_zone
      subnet_group_name = aws_db_subnet_group.main.name
    } : null

    # Conditional gateway information
    internet_gateway = length(aws_subnet.public) > 0 ? {
      id = aws_internet_gateway.main.id
      route_table_id = aws_route_table.public.id
    } : null

    nat_gateways = var.network_config.enable_nat_gateway ? {
      ids = aws_nat_gateway.main[*].id
      eip_ids = aws_eip.nat[*].id
      public_ips = aws_eip.nat[*].public_ip
    } : null

    # VPN gateway if enabled
    vpn_gateway = var.network_config.enable_vpn_gateway ? {
      id = aws_vpn_gateway.main[0].id
      amazon_side_asn = aws_vpn_gateway.main[0].amazon_side_asn
    } : null

    # DNS configuration
    dns_configuration = {
      domain_name = var.network_config.dns_domain
      private_zone_id = aws_route53_zone.private.zone_id
      public_zone_id = var.network_config.dns_domain != "" ? aws_route53_zone.public[0].zone_id : null
    }

    # Environment-specific information
    environment_details = {
      environment = var.network_config.environment
      is_production = var.network_config.environment == "prod"
      monitoring_enabled = var.network_config.environment == "prod" ? true : false
      backup_enabled = var.network_config.environment == "prod" ? true : false
    }
  }
}

# Advanced compute outputs
output "compute_configuration" {
  description = "Complete compute configuration details"
  value = {
    # Launch template information
    launch_template = {
      id = aws_launch_template.app.id
      latest_version = aws_launch_template.app.latest_version
      name = aws_launch_template.app.name
    }

    # Auto Scaling group information
    autoscaling_group = var.compute_config.auto_scaling_enabled ? {
      name = aws_autoscaling_group.app[0].name
      arn = aws_autoscaling_group.app[0].arn
      min_size = aws_autoscaling_group.app[0].min_size
      max_size = aws_autoscaling_group.app[0].max_size
      desired_capacity = aws_autoscaling_group.app[0].desired_capacity
      availability_zones = aws_autoscaling_group.app[0].availability_zones
    } : null

    # Load balancer information (if present)
    load_balancer = var.compute_config.load_balancer_enabled ? {
      arn = aws_lb.app[0].arn
      dns_name = aws_lb.app[0].dns_name
      zone_id = aws_lb.app[0].zone_id
      target_group_arn = aws_lb_target_group.app[0].arn
    } : null

    # Spot instance information
    spot_configuration = var.compute_config.spot_instances_enabled ? {
      enabled = true
      max_price = var.compute_config.spot_max_price
      instance_types = var.compute_config.instance_types
    } : {
      enabled = false
      max_price = null
      instance_types = var.compute_config.instance_types
    }

    # Monitoring configuration
    monitoring = {
      cloudwatch_log_group = aws_cloudwatch_log_group.app.name
      metrics_namespace = "${var.project_name}/${var.environment}"
      dashboard_url = "https://console.aws.amazon.com/cloudwatch/home?region=${data.aws_region.current.name}#dashboards:name=${var.project_name}-${var.environment}"
    }
  }
}

# Sensitive database outputs
output "database_configuration" {
  description = "Database configuration details (some values are sensitive)"
  value = {
    # Public information
    instance_identifier = aws_db_instance.master.id
    endpoint = aws_db_instance.master.endpoint
    port = aws_db_instance.master.port
    engine = aws_db_instance.master.engine
    engine_version = aws_db_instance.master.engine_version

    # Security configuration
    security_configuration = {
      encrypted = aws_db_instance.master.storage_encrypted
      kms_key_id = aws_db_instance.master.kms_key_id
      vpc_security_group_ids = aws_db_instance.master.vpc_security_group_ids
      subnet_group_name = aws_db_instance.master.db_subnet_group_name
    }

    # Backup configuration
    backup_configuration = {
      retention_period = aws_db_instance.master.backup_retention_period
      backup_window = aws_db_instance.master.backup_window
      maintenance_window = aws_db_instance.master.maintenance_window
    }

    # Monitoring configuration
    monitoring_configuration = {
      monitoring_interval = aws_db_instance.master.monitoring_interval
      monitoring_role_arn = aws_db_instance.master.monitoring_role_arn
      performance_insights_enabled = aws_db_instance.master.performance_insights_enabled
    }

    # Replica information (if present)
    read_replica = var.database_config.read_replica_enabled ? {
      identifier = aws_db_instance.replica[0].id
      endpoint = aws_db_instance.replica[0].endpoint
      port = aws_db_instance.replica[0].port
    } : null
  }

  # Not sensitive, because no credentials are included
  sensitive = false
}

# Sensitive credential outputs
output "database_credentials" {
  description = "Database credentials (sensitive)"
  value = {
    secrets_manager_arn = aws_secretsmanager_secret.database_credentials.arn
    secrets_manager_name = aws_secretsmanager_secret.database_credentials.name
    kms_key_arn = aws_kms_key.secrets.arn
    kms_key_alias = aws_kms_alias.secrets.name
  }
  sensitive = true
}

# Advanced security outputs
output "security_configuration" {
  description = "Security configuration details"
  value = {
    # Security groups
    security_groups = {
      alb = {
        id = aws_security_group.alb.id
        name = aws_security_group.alb.name
        description = aws_security_group.alb.description
      }
      app_servers = {
        id = aws_security_group.app_servers.id
        name = aws_security_group.app_servers.name
        description = aws_security_group.app_servers.description
      }
      database = {
        id = aws_security_group.database.id
        name = aws_security_group.database.name
        description = aws_security_group.database.description
      }
    }

    # KMS keys
    kms_keys = {
      secrets = {
        id = aws_kms_key.secrets.id
        arn = aws_kms_key.secrets.arn
        alias = aws_kms_alias.secrets.name
      }
    }

    # Compliance status
    compliance_status = {
      encryption_enabled = aws_db_instance.master.storage_encrypted
      vpc_flow_logs_enabled = var.security_config.enable_flow_logs
      cloudtrail_enabled = var.security_config.enable_cloudtrail
      config_enabled = var.security_config.enable_config
      guardduty_enabled = var.security_config.enable_guardduty
    }

    # Audit information
    audit_configuration = {
      cloudtrail_arn = var.security_config.enable_cloudtrail ? aws_cloudtrail.main[0].arn : null
      config_recorder_name = var.security_config.enable_config ? aws_config_configuration_recorder.main[0].name : null
      guardduty_detector_id = var.security_config.enable_guardduty ? aws_guardduty_detector.main[0].id : null
    }
  }
}

# Conditional outputs for different environments
output "environment_specific_outputs" {
  description = "Environment-specific configuration outputs"
  value = var.environment == "prod" ? {
    # Production-specific outputs
    type = "production"
    high_availability = true
    backup_enabled = true
    monitoring_level = "detailed"

    # Production URLs
    application_url = "https://${var.network_config.dns_domain}"
    admin_url = "https://admin.${var.network_config.dns_domain}"
    api_url = "https://api.${var.network_config.dns_domain}"

    # Production metrics
    metrics_dashboard = "https://console.aws.amazon.com/cloudwatch/home?region=${data.aws_region.current.name}#dashboards:name=${var.project_name}-production"
    cost_dashboard = "https://console.aws.amazon.com/billing/home#/dashboard"

    # Production alerts
    alert_endpoints = var.monitoring_config.alert_endpoints
    escalation_policy = "immediate"

  } : {
    # Development-specific outputs
    type = "development"
    high_availability = false
    backup_enabled = false
    monitoring_level = "basic"

    # Development URLs
    application_url = "http://${aws_lb.app[0].dns_name}"
    admin_url = "http://${aws_lb.app[0].dns_name}/admin"
    api_url = "http://${aws_lb.app[0].dns_name}/api"

    # Development notes
    ssh_access = "Available from VPC"
    debug_mode = "enabled"
    auto_shutdown = "enabled"
  }
}

# Aggregated outputs for other tools
output "terraform_outputs_summary" {
  description = "Summary of all important outputs for external tools"
  value = {
    # Network summary
    network = {
      vpc_id = aws_vpc.main.id
      public_subnet_ids = aws_subnet.public[*].id
      private_subnet_ids = aws_subnet.private_app[*].id
      database_subnet_ids = aws_subnet.private_db[*].id
    }

    # Security summary
    security = {
      alb_security_group_id = aws_security_group.alb.id
      app_security_group_id = aws_security_group.app_servers.id
      db_security_group_id = aws_security_group.database.id
    }

    # Compute summary
    compute = {
      launch_template_id = aws_launch_template.app.id
      autoscaling_group_name = var.compute_config.auto_scaling_enabled ? aws_autoscaling_group.app[0].name : null
      load_balancer_dns = var.compute_config.load_balancer_enabled ? aws_lb.app[0].dns_name : null
    }

    # Database summary
    database = {
      endpoint = aws_db_instance.master.endpoint
      port = aws_db_instance.master.port
      secrets_manager_arn = aws_secretsmanager_secret.database_credentials.arn
    }

    # Monitoring summary
    monitoring = {
      log_group_name = aws_cloudwatch_log_group.app.name
      metrics_namespace = "${var.project_name}/${var.environment}"
    }
  }
}

Output debugging techniques:


# Show all outputs
terraform output

# Show a specific output
terraform output network_configuration

# Raw output without formatting
terraform output -raw database_credentials

# JSON format for machine processing
terraform output -json | jq .

# Debug sensitive outputs (use carefully)
terraform output -json | jq '.database_credentials.value'

Output performance optimisation:

Optimisation Description Use
Lazy evaluation Compute outputs only when retrieved Complex calculations
Caching Computed values in locals Reused values
Conditional logic Only necessary outputs Environment-dependent data
Structured data Grouped outputs Better organisation

Output best practices:

  • Use meaningful descriptions
  • Group related outputs in objects
  • Mark sensitive outputs correctly
  • Use conditional logic for flexibility
  • Document complex output structures

Local values (locals) for complex calculations

What are local values (locals)? Locals are computed values that can be defined and reused inside a Terraform configuration. They work like variables, but are calculated at runtime and can contain complex expressions, functions and conditional logic. Locals are the key to clean, maintainable Terraform configurations.

Why do locals matter so much? They eliminate code duplication, improve performance by caching calculations, and make complex logic readable. Without locals you would repeat the same complex expressions in several places, which leads to errors and poor maintainability.

What must you watch for with locals? Locals are recomputed on every plan and apply. Complex calculations can hurt performance. Circular dependencies between locals cause errors. Definition order matters.

What do you use locals for? Complex naming, conditional configurations, data transformations, tag generation, and anywhere you need to reuse complex logic.


# Advanced locals configuration for complex infrastructure
locals {
  # ============================================================================
  # BASE CALCULATIONS AND METADATA
  # ============================================================================

  # Current time and date for tagging
  current_timestamp = timestamp()
  current_date      = formatdate("YYYY-MM-DD", local.current_timestamp)
  current_time      = formatdate("hh:mm:ss", local.current_timestamp)

  # Git information (if available)
  git_commit = try(file("${path.module}/.git/refs/heads/main"), "unknown")
  git_branch = try(trimspace(file("${path.module}/.git/HEAD")), "unknown")

  # Environment-specific settings
  environment_config = {
    dev = {
      is_production = false
      backup_retention = 7
      monitoring_level = "basic"
      instance_count = 1
      storage_size = 20
      enable_logging = true
      enable_encryption = false
      cost_optimization = true
    }
    staging = {
      is_production = false
      backup_retention = 14
      monitoring_level = "standard"
      instance_count = 2
      storage_size = 50
      enable_logging = true
      enable_encryption = true
      cost_optimization = true
    }
    prod = {
      is_production = true
      backup_retention = 30
      monitoring_level = "detailed"
      instance_count = 3
      storage_size = 100
      enable_logging = true
      enable_encryption = true
      cost_optimization = false
    }
  }

  # Current environment configuration
  current_env = local.environment_config[var.environment]

  # ============================================================================
  # ADVANCED NAMING AND TAGGING
  # ============================================================================

  # Base naming conventions
  naming_convention = {
    # Base prefix for all resources
    prefix = "${var.project_name}-${var.environment}"

    # Specific names for different resource types
    vpc_name = "${var.project_name}-${var.environment}-vpc"
    subnet_prefix = "${var.project_name}-${var.environment}"
    sg_prefix = "${var.project_name}-${var.environment}"

    # Advanced names with region and AZ
    regional_prefix = "${var.project_name}-${var.environment}-${data.aws_region.current.name}"

    # Names with hash for uniqueness
    unique_suffix = substr(md5("${var.project_name}-${var.environment}-${local.current_timestamp}"), 0, 8)
    unique_prefix = "${var.project_name}-${var.environment}-${local.unique_suffix}"
  }

  # Advanced tag generation
  base_tags = {
    # Project information
    Project     = var.project_name
    Environment = var.environment
    Region      = data.aws_region.current.name

    # Deployment information
    ManagedBy     = "terraform"
    DeployedAt    = local.current_timestamp
    DeployedDate  = local.current_date
    TerraformWorkspace = terraform.workspace

    # Git information (if available)
    GitCommit = local.git_commit
    GitBranch = local.git_branch

    # Environment-specific tags
    IsProduction    = local.current_env.is_production
    BackupPolicy    = "${local.current_env.backup_retention}days"
    MonitoringLevel = local.current_env.monitoring_level
    CostOptimized   = local.current_env.cost_optimization
  }

  # Resource-specific tags
  compute_tags = merge(local.base_tags, {
    ResourceType = "compute"
    AutoShutdown = local.current_env.cost_optimization
    InstanceCount = local.current_env.instance_count
  })

  storage_tags = merge(local.base_tags, {
    ResourceType = "storage"
    Encrypted = local.current_env.enable_encryption
    BackupEnabled = true
    StorageSize = "${local.current_env.storage_size}GB"
  })

  network_tags = merge(local.base_tags, {
    ResourceType = "network"
    VPCFlowLogs = local.current_env.enable_logging
  })

  # ============================================================================
  # NETWORK CALCULATIONS
  # ============================================================================

  # Automatic CIDR calculation for subnets
  vpc_cidr = var.vpc_cidr

  # Compute subnet CIDRs automatically based on AZs
  availability_zones = data.aws_availability_zones.available.names
  az_count = length(local.availability_zones)

  # Public subnets (first X subnets)
  public_subnet_cidrs = [
    for i in range(local.az_count) :
    cidrsubnet(local.vpc_cidr, 8, i)
  ]

  # Private app subnets (next X subnets)
  private_app_subnet_cidrs = [
    for i in range(local.az_count) :
    cidrsubnet(local.vpc_cidr, 8, i + 10)
  ]

  # Database subnets (last X subnets)
  database_subnet_cidrs = [
    for i in range(local.az_count) :
    cidrsubnet(local.vpc_cidr, 8, i + 20)
  ]

  # Subnet configurations for different tiers
  subnet_configurations = {
    public = {
      name_prefix = "${local.naming_convention.subnet_prefix}-public"
      cidrs = local.public_subnet_cidrs
      map_public_ip = true
      route_table_type = "public"
      tags = merge(local.network_tags, {
        SubnetType = "public"
        InternetAccess = "direct"
      })
    }

    private_app = {
      name_prefix = "${local.naming_convention.subnet_prefix}-private-app"
      cidrs = local.private_app_subnet_cidrs
      map_public_ip = false
      route_table_type = "private"
      tags = merge(local.network_tags, {
        SubnetType = "private"
        InternetAccess = "nat"
        Tier = "application"
      })
    }

    database = {
      name_prefix = "${local.naming_convention.subnet_prefix}-private-db"
      cidrs = local.database_subnet_cidrs
      map_public_ip = false
      route_table_type = "database"
      tags = merge(local.network_tags, {
        SubnetType = "private"
        InternetAccess = "none"
        Tier = "data"
      })
    }
  }

  # ============================================================================
  # SECURITY CONFIGURATIONS
  # ============================================================================

  # Security group rules based on environment
  security_group_rules = {
    # ALB security group
    alb = {
      name = "${local.naming_convention.sg_prefix}-alb"
      description = "Security group for Application Load Balancer"
      ingress_rules = [
        {
          description = "HTTP from internet"
          from_port = 80
          to_port = 80
          protocol = "tcp"
          cidr_blocks = ["0.0.0.0/0"]
        },
        {
          description = "HTTPS from internet"
          from_port = 443
          to_port = 443
          protocol = "tcp"
          cidr_blocks = ["0.0.0.0/0"]
        }
      ]
      egress_rules = [
        {
          description = "All outbound traffic"
          from_port = 0
          to_port = 0
          protocol = "-1"
          cidr_blocks = ["0.0.0.0/0"]
        }
      ]
    }

    # Application servers security group
    app = {
      name = "${local.naming_convention.sg_prefix}-app"
      description = "Security group for application servers"
      ingress_rules = concat(
        [
          {
            description = "HTTP from ALB"
            from_port = 8080
            to_port = 8080
            protocol = "tcp"
            cidr_blocks = local.private_app_subnet_cidrs
          }
        ],
        # SSH access only in development
        local.current_env.is_production == false ? [
          {
            description = "SSH from VPC"
            from_port = 22
            to_port = 22
            protocol = "tcp"
            cidr_blocks = [local.vpc_cidr]
          }
        ] : []
      )
      egress_rules = [
        {
          description = "All outbound traffic"
          from_port = 0
          to_port = 0
          protocol = "-1"
          cidr_blocks = ["0.0.0.0/0"]
        }
      ]
    }

    # Database security group
    database = {
      name = "${local.naming_convention.sg_prefix}-db"
      description = "Security group for database servers"
      ingress_rules = [
        {
          description = "MySQL from app servers"
          from_port = 3306
          to_port = 3306
          protocol = "tcp"
          cidr_blocks = local.private_app_subnet_cidrs
        }
      ]
      egress_rules = []  # No outbound rules for databases
    }
  }

  # ============================================================================
  # COMPUTE CONFIGURATIONS
  # ============================================================================

  # Instance configuration based on environment
  instance_configurations = {
    # Launch template configuration
    launch_template = {
      name_prefix = "${local.naming_convention.prefix}-app"
      image_id = data.aws_ami.ubuntu.id
      instance_type = local.current_env.is_production ? "t3.large" : "t3.micro"

      # User data with complex template variables
      user_data_vars = {
        project_name = var.project_name
        environment = var.environment
        region = data.aws_region.current.name

        # Environment-specific variables
        debug_mode = !local.current_env.is_production
        log_level = local.current_env.is_production ? "INFO" : "DEBUG"
        monitoring_enabled = local.current_env.monitoring_level != "basic"

        # Database configuration
        database_host = "localhost"  # Replaced later by RDS
        database_port = 3306
        database_name = replace("${var.project_name}_${var.environment}", "-", "_")

        # Application configuration
        app_port = 8080
        health_check_path = "/health"
        metrics_enabled = local.current_env.monitoring_level == "detailed"

        # Computed values
        instance_role = "application"
        deployment_timestamp = local.current_timestamp
      }

      # Block device mappings
      block_device_mappings = [
        {
          device_name = "/dev/sda1"
          volume_size = local.current_env.storage_size
          volume_type = "gp3"
          encrypted = local.current_env.enable_encryption
          iops = local.current_env.is_production ? 3000 : 1000
        }
      ]

      # Instance tags
      tags = merge(local.compute_tags, {
        Name = "${local.naming_convention.prefix}-app"
        InstanceProfile = "application"
      })
    }

    # Auto Scaling group configuration
    autoscaling_group = {
      name = "${local.naming_convention.prefix}-app-asg"
      min_size = local.current_env.is_production ? 2 : 1
      max_size = local.current_env.is_production ? 10 : 3
      desired_capacity = local.current_env.instance_count

      # Health check configuration
      health_check_type = "ELB"
      health_check_grace_period = 300

      # Termination policies
      termination_policies = ["OldestInstance"]

      # Scaling policies
      target_group_arns = []  # Set later

      tags = merge(local.compute_tags, {
        Name = "${local.naming_convention.prefix}-app-asg"
        AutoScalingGroup = "application"
      })
    }
  }

  # ============================================================================
  # DATABASE CONFIGURATIONS
  # ============================================================================

  # RDS configuration based on environment
  database_configurations = {
    master = {
      identifier = "${local.naming_convention.prefix}-master"
      engine = "mysql"
      engine_version = "8.0"
      instance_class = local.current_env.is_production ? "db.t3.medium" : "db.t3.micro"

      # Storage configuration
      allocated_storage = local.current_env.storage_size
      max_allocated_storage = local.current_env.storage_size * 2
      storage_type = "gp3"
      storage_encrypted = local.current_env.enable_encryption

      # Backup configuration
      backup_retention_period = local.current_env.backup_retention
      backup_window = "03:00-04:00"
      maintenance_window = "sun:04:00-sun:05:00"

      # Monitoring configuration
      monitoring_interval = local.current_env.monitoring_level == "detailed" ? 15 : 60
      performance_insights_enabled = local.current_env.monitoring_level == "detailed"

      # Multi-AZ for production
      multi_az = local.current_env.is_production

      # Database name
      db_name = replace("${var.project_name}_${var.environment}", "-", "_")

      # Parameter group
      parameter_group_family = "mysql8.0"

      tags = merge(local.storage_tags, {
        Name = "${local.naming_convention.prefix}-master"
        DatabaseRole = "master"
        Engine = "mysql"
      })
    }

    # Read replica only for production
    replica = local.current_env.is_production ? {
      identifier = "${local.naming_convention.prefix}-replica"
      replicate_source_db = "${local.naming_convention.prefix}-master"
      instance_class = "db.t3.medium"

      # Monitoring for replica
      monitoring_interval = 60
      performance_insights_enabled = false

      tags = merge(local.storage_tags, {
        Name = "${local.naming_convention.prefix}-replica"
        DatabaseRole = "replica"
        Engine = "mysql"
      })
    } : null
  }

  # ============================================================================
  # MONITORING AND LOGGING CONFIGURATIONS
  # ============================================================================

  # CloudWatch configuration
  cloudwatch_configurations = {
    # Log groups
    log_groups = {
      application = {
        name = "/aws/ec2/${local.naming_convention.prefix}/application"
        retention_days = local.current_env.is_production ? 90 : 7
        tags = merge(local.base_tags, {
          LogType = "application"
        })
      }

      system = {
        name = "/aws/ec2/${local.naming_convention.prefix}/system"
        retention_days = local.current_env.is_production ? 30 : 7
        tags = merge(local.base_tags, {
          LogType = "system"
        })
      }

      access = {
        name = "/aws/elb/${local.naming_convention.prefix}/access"
        retention_days = local.current_env.is_production ? 180 : 14
        tags = merge(local.base_tags, {
          LogType = "access"
        })
      }
    }

    # CloudWatch alarms
    alarms = local.current_env.monitoring_level != "basic" ? {
      high_cpu = {
        alarm_name = "${local.naming_convention.prefix}-high-cpu"
        comparison_operator = "GreaterThanThreshold"
        evaluation_periods = "2"
        metric_name = "CPUUtilization"
        namespace = "AWS/EC2"
        period = "120"
        statistic = "Average"
        threshold = "80"
        alarm_description = "This metric monitors ec2 cpu utilization"
        alarm_actions = []  # SNS topics added later
      }

      high_memory = {
        alarm_name = "${local.naming_convention.prefix}-high-memory"
        comparison_operator = "GreaterThanThreshold"
        evaluation_periods = "2"
        metric_name = "MemoryUtilization"
        namespace = "CWAgent"
        period = "120"
        statistic = "Average"
        threshold = "85"
        alarm_description = "This metric monitors memory utilization"
        alarm_actions = []
      }
    } : {}
  }

  # ============================================================================
  # COMPUTED OUTPUT STRUCTURES
  # ============================================================================

  # Computed outputs for other modules
  computed_outputs = {
    # Network summary
    network_summary = {
      vpc_id = var.vpc_id  # Replaced by the real VPC ID
      vpc_cidr = local.vpc_cidr
      availability_zones = local.availability_zones
      subnet_count = local.az_count

      # Subnet information
      public_subnets = {
        count = local.az_count
        cidrs = local.public_subnet_cidrs
      }

      private_app_subnets = {
        count = local.az_count
        cidrs = local.private_app_subnet_cidrs
      }

      database_subnets = {
        count = local.az_count
        cidrs = local.database_subnet_cidrs
      }
    }

    # Compute summary
    compute_summary = {
      instance_type = local.instance_configurations.launch_template.instance_type
      min_instances = local.instance_configurations.autoscaling_group.min_size
      max_instances = local.instance_configurations.autoscaling_group.max_size
      desired_instances = local.instance_configurations.autoscaling_group.desired_capacity
      storage_size = "${local.current_env.storage_size}GB"
      encrypted = local.current_env.enable_encryption
    }

    # Database summary
    database_summary = {
      engine = local.database_configurations.master.engine
      engine_version = local.database_configurations.master.engine_version
      instance_class = local.database_configurations.master.instance_class
      storage_size = "${local.database_configurations.master.allocated_storage}GB"
      backup_retention = "${local.database_configurations.master.backup_retention_period}days"
      multi_az = local.database_configurations.master.multi_az
      has_replica = local.database_configurations.replica != null
    }

    # Environment summary
    environment_summary = {
      environment = var.environment
      is_production = local.current_env.is_production
      monitoring_level = local.current_env.monitoring_level
      cost_optimized = local.current_env.cost_optimization
      backup_retention = local.current_env.backup_retention
      encryption_enabled = local.current_env.enable_encryption
    }
  }
}

Performance optimisation for locals:


# Performance-optimised locals strategies
locals {
  # ✅ Good: cache simple calculations
  vpc_cidr_base = split("/", var.vpc_cidr)[0]
  vpc_cidr_prefix = split("/", var.vpc_cidr)[1]

  # ✅ Good: cache complex list operations
  filtered_availability_zones = [
    for az in data.aws_availability_zones.available.names :
    az if length(regexall("us-west-2[abc]", az)) > 0
  ]

  # ✅ Good: cache conditional logic
  production_settings = var.environment == "prod" ? {
    instance_count = 5
    backup_retention = 30
    monitoring = "detailed"
  } : {
    instance_count = 1
    backup_retention = 7
    monitoring = "basic"
  }

  # ⚠️ Caution: very complex calculations
  # Use these only when they are really needed
  complex_subnet_calculation = {
    for i, az in local.filtered_availability_zones :
    az => {
      public_cidr = cidrsubnet(var.vpc_cidr, 8, i)
      private_cidr = cidrsubnet(var.vpc_cidr, 8, i + 10)
      database_cidr = cidrsubnet(var.vpc_cidr, 8, i + 20)
      route_table_id = "rt-${md5("${az}-${i}")}"
    }
  }
}

Locals organisation strategy (layer model):


┌─────────────────────────────────────────────────────────────┐
│                Locals Organization Strategy                 │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ 1. BASE CONFIGURATION                                 │  │
│  │ ───────────────────────────────────────────────────── │  │
│  │ - Environment Settings        - Common Tags           │  │
│  │ - Naming Conventions          - Timestamps            │  │
│  └───────────────────────────┬───────────────────────────┘  │
│                              │                              │
│                              ▼                              │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ 2. COMPUTED VALUES                                    │  │
│  │ ───────────────────────────────────────────────────── │  │
│  │ - Network CIDRs               - Conditional Logic     │  │
│  │ - Resource Counts             - Transformations       │  │
│  └───────────────────────────┬───────────────────────────┘  │
│                              │                              │
│                              ▼                              │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ 3. CONFIGURATIONS                                     │  │
│  │ ───────────────────────────────────────────────────── │  │
│  │ - Security Groups             - Database Settings     │  │
│  │ - Launch Templates            - Monitoring & Alarms   │  │
│  └───────────────────────────┬───────────────────────────┘  │
│                              │                              │
│                              ▼                              │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ 4. OUTPUT STRUCTURES                                  │  │
│  │ ───────────────────────────────────────────────────── │  │
│  │ - Summaries                   - Exports               │  │
│  │ - Integration Data            - Module Payloads       │  │
│  └───────────────────────────────────────────────────────┘  │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Locals best practices:

Best practice Description Example
Logical grouping Group related calculations Network, Compute, Database
Meaningful names Self-explanatory local names current_env instead of env
Comments for complexity Document complex logic Why particular calculations
Watch performance Cache heavy calculations Compute once, use often
Manage dependencies Avoid circular references Clear hierarchy

🔧 Practical example - locals in real resources:


# Using locals in real resources
resource "aws_subnet" "public" {
  count = length(local.availability_zones)

  vpc_id                  = aws_vpc.main.id
  cidr_block              = local.subnet_configurations.public.cidrs[count.index]
  availability_zone       = local.availability_zones[count.index]
  map_public_ip_on_launch = local.subnet_configurations.public.map_public_ip

  tags = merge(
    local.subnet_configurations.public.tags,
    {
      Name = "${local.subnet_configurations.public.name_prefix}-${count.index + 1}"
      AvailabilityZone = local.availability_zones[count.index]
    }
  )
}

resource "aws_launch_template" "app" {
  name_prefix   = local.instance_configurations.launch_template.name_prefix
  image_id      = local.instance_configurations.launch_template.image_id
  instance_type = local.instance_configurations.launch_template.instance_type

  vpc_security_group_ids = [aws_security_group.app.id]

  # Use the computed user-data variables
  user_data = base64encode(templatefile("${path.module}/userdata.sh",
    local.instance_configurations.launch_template.user_data_vars
  ))

  # Use the computed block device mappings
  dynamic "block_device_mappings" {
    for_each = local.instance_configurations.launch_template.block_device_mappings
    content {
      device_name = block_device_mappings.value.device_name
      ebs {
        volume_size = block_device_mappings.value.volume_size
        volume_type = block_device_mappings.value.volume_type
        encrypted   = block_device_mappings.value.encrypted
        iops        = block_device_mappings.value.iops
      }
    }
  }

  tag_specifications {
    resource_type = "instance"
    tags = local.instance_configurations.launch_template.tags
  }
}

Locals performance tips:

  • Use locals for values that are used more than once
  • Cache complex for-expressions in locals
  • Avoid deep nesting in locals
  • Use separate locals blocks for better organisation

Locals pitfalls:

Problem Symptom Solution
Circular dependencies Error: Cycle in local values Restructure dependencies
Performance problems Slow plan times Optimise complex calculations
Unreadable complexity Hard-to-understand locals Split into smaller parts
Order problems Error: Reference to undeclared Order local definitions

Locals debugging techniques:


# Show locals values in terraform console
terraform console
> local.current_env
> local.naming_convention.prefix
> local.computed_outputs.network_summary

# Show locals values in the plan
terraform plan | grep "local\."

# Analyse locals dependencies
terraform graph | grep local

With these advanced variable and output features you write not only functional but intelligent Terraform code. Complex validations catch errors before they become expensive, sensitive variables protect your credentials, locals eliminate duplication and keep complex logic maintainable, and advanced outputs make modules flexible and reusable. These techniques are the difference between simple Terraform scripts and professional, production-ready infrastructure definitions.

Built-in functions and expressions

What are built-in functions in Terraform? Built-in functions are ready-made tools integrated into HCL that enable complex operations. They turn static configurations into dynamic, intelligent infrastructure definitions. Terraform offers more than 100 built-in functions for string manipulation, maths, collection processing and conditional logic.

Why are built-in functions decisive? They remove the need for external scripting and make Terraform configurations self-contained. Without them you would have to push complex logic into external tools, which leads to fragile, hard-to-maintain setups. Built-in functions keep everything in Terraform and make infrastructure predictable.

What must you watch for with built-in functions? Functions run on every plan and apply. Complex nesting can hurt performance and reduce readability. Not all functions are available in all Terraform versions. Missing input validation can cause runtime errors.

What do you use built-in functions for? Dynamic naming, data transformation, conditional logic, collection processing, JSON/YAML generation and complex calculations needed when creating infrastructure.

String manipulation and formatting

What does Terraform offer for string operations? Terraform has extensive string functions for formatting, manipulation, validation and transformation. Those functions let you create dynamic configurations that can adapt to different environments.

Why is string manipulation so important? In Terraform you constantly work with strings — resource names, tags, URLs, paths and configuration values. Professional string manipulation makes configurations flexible and maintainable. Without it you get static, hard-to-change definitions.

What should you watch for with string functions? String operations are case-sensitive. Empty strings and null values can produce unexpected results. Regex patterns must be escaped correctly. Performance can suffer on large string operations.

What do you use string functions for? Resource naming, tag generation, URL construction, path manipulation, JSON template creation and data validation.

🔧 Practical example - advanced string functions:


# Extensive string manipulation for infrastructure naming
locals {
  # Base variables for string operations
  project_name = "e-commerce-platform"
  environment  = "production"
  region       = "us-east-1"
  team_name    = "Platform Engineering"

  # ============================================================================
  # STRING FORMATTING AND TRANSFORMATION
  # ============================================================================

  # Base string operations
  string_operations = {
    # Case manipulation
    project_upper = upper(local.project_name)                    # "E-COMMERCE-PLATFORM"
    project_lower = lower(local.project_name)                    # "e-commerce-platform"
    project_title = title(replace(local.project_name, "-", " ")) # "E Commerce Platform"

    # String cleanup for different contexts
    dns_safe_name = replace(lower(local.project_name), "_", "-")              # DNS-safe names
    db_safe_name  = replace(replace(local.project_name, "-", "_"), ".", "_")  # Database-safe names
    s3_safe_name  = replace(lower(local.project_name), "_", "-")              # S3-safe names

    # Length validation and adjustment
    truncated_name = substr(local.project_name, 0, 10)           # "e-commerce" (first 10 characters)
    padded_name    = format("%-20s", local.project_name)         # Left-aligned to 20 characters

    # String combinations
    full_prefix = join("-", [local.project_name, local.environment, local.region])
    compact_prefix = join("", [substr(local.project_name, 0, 3), local.environment, substr(local.region, -1, 1)])
  }

  # Advanced naming conventions with string functions
  naming_patterns = {
    # Standard resource names
    vpc_name = format("%s-%s-vpc", local.string_operations.dns_safe_name, local.environment)

    # Subnet names with index formatting
    subnet_pattern = format("%s-%s-subnet-%%02d", local.string_operations.dns_safe_name, local.environment)

    # Security group names with role
    sg_pattern = format("%s-%s-%%s-sg", local.string_operations.dns_safe_name, local.environment)

    # Database names (underscores for SQL compatibility)
    database_name = format("%s_%s_db", local.string_operations.db_safe_name, local.environment)

    # S3 bucket names (globally unique)
    bucket_pattern = format("%s-%s-%%s-%s",
      local.string_operations.s3_safe_name,
      local.environment,
      formatdate("YYYY-MM", timestamp())
    )

    # CloudWatch log groups
    log_group_pattern = format("/aws/%%s/%s-%s", local.string_operations.dns_safe_name, local.environment)
  }

  # ============================================================================
  # ADVANCED STRING VALIDATION AND CLEANUP
  # ============================================================================

  # String validation with regex
  validation_results = {
    # DNS name validation
    is_valid_dns = can(regex("^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$", local.string_operations.dns_safe_name))

    # Email validation
    admin_email = "admin@company.com"
    is_valid_email = can(regex("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$", local.validation_results.admin_email))

    # IP address validation
    vpc_cidr = "10.0.0.0/16"
    is_valid_cidr = can(cidrhost(local.validation_results.vpc_cidr, 0))

    # Password complexity check
    temp_password = "TempPass123!"
    has_upper = can(regex("[A-Z]", local.validation_results.temp_password))
    has_lower = can(regex("[a-z]", local.validation_results.temp_password))
    has_digit = can(regex("[0-9]", local.validation_results.temp_password))
    has_special = can(regex("[!@#$%^&*()_+-=]", local.validation_results.temp_password))
    is_complex_password = (
      local.validation_results.has_upper &&
      local.validation_results.has_lower &&
      local.validation_results.has_digit &&
      local.validation_results.has_special &&
      length(local.validation_results.temp_password) >= 8
    )
  }

  # String template functions
  template_strings = {
    # Advanced formatting with multiple variables
    resource_description = format(
      "Managed by Terraform for %s project in %s environment, deployed to %s region by %s team",
      local.project_name,
      local.environment,
      local.region,
      local.team_name
    )

    # JSON template for policies
    policy_template = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = [
            "s3:GetObject",
            "s3:PutObject"
          ]
          Resource = format("arn:aws:s3:::%s/*", format(local.naming_patterns.bucket_pattern, "data"))
          Condition = {
            StringEquals = {
              "s3:x-amz-server-side-encryption" = "AES256"
            }
          }
        }
      ]
    })

    # YAML template for user data
    user_data_yaml = yamlencode({
      users = [
        {
          name = "ubuntu"
          sudo = "ALL=(ALL) NOPASSWD:ALL"
          shell = "/bin/bash"
          ssh_authorized_keys = [
            format("ssh-rsa %s %s@%s",
              "AAAAB3NzaC1yc2EAAAADAQABAAABAQC...",
              "admin",
              replace(local.project_name, "-", "")
            )
          ]
        }
      ]
      packages = [
        "curl",
        "wget",
        "unzip",
        "docker.io"
      ]
      runcmd = [
        format("echo 'Project: %s' >> /etc/motd", local.project_name),
        format("echo 'Environment: %s' >> /etc/motd", local.environment),
        "systemctl enable docker",
        "systemctl start docker"
      ]
    })
  }

  # ============================================================================
  # URL AND PATH MANIPULATION
  # ============================================================================

  # URL construction for various services
  service_urls = {
    # Application URLs
    app_base_url = format("https://%s.%s.example.com", local.environment, replace(local.project_name, "-", ""))
    api_url = format("%s/api/v1", local.service_urls.app_base_url)
    admin_url = format("%s/admin", local.service_urls.app_base_url)

    # Monitoring URLs
    grafana_url = format("https://grafana-%s.monitoring.example.com", local.environment)
    prometheus_url = format("https://prometheus-%s.monitoring.example.com", local.environment)

    # Dashboard URLs with URL encoding
    cloudwatch_dashboard = format(
      "https://console.aws.amazon.com/cloudwatch/home?region=%s#dashboards:name=%s",
      local.region,
      urlencode(format("%s-%s-dashboard", local.project_name, local.environment))
    )
  }

  # Path manipulation for different contexts
  file_paths = {
    # Linux paths for configuration
    app_config_dir = format("/etc/%s", replace(local.project_name, "-", "_"))
    app_log_dir = format("/var/log/%s", replace(local.project_name, "-", "_"))
    app_data_dir = format("/opt/%s/data", replace(local.project_name, "-", "_"))

    # Relative paths for modules
    module_path = format("./modules/%s", local.environment)
    template_path = format("${path.module}/templates/%s", local.environment)

    # S3 paths for backups
    backup_prefix = format("backups/%s/%s", local.environment, formatdate("YYYY/MM/DD", timestamp()))
    log_prefix = format("logs/%s/%s", local.environment, formatdate("YYYY/MM/DD", timestamp()))
  }
}

# Practical use of string functions in resources
resource "aws_s3_bucket" "data" {
  bucket = format(local.naming_patterns.bucket_pattern, "data")

  tags = {
    Name = format(local.naming_patterns.bucket_pattern, "data")
    Description = local.template_strings.resource_description
    Environment = local.environment
    Project = local.project_name
  }
}

resource "aws_s3_bucket_object" "config" {
  bucket = aws_s3_bucket.data.bucket
  key    = format("%s/config.yaml", local.file_paths.backup_prefix)
  content = local.template_strings.user_data_yaml

  tags = {
    ConfigType = "user-data"
    CreatedAt = formatdate("YYYY-MM-DD hh:mm:ss ZZZ", timestamp())
  }
}

# CloudWatch log group with string manipulation
resource "aws_cloudwatch_log_group" "application" {
  name              = format(local.naming_patterns.log_group_pattern, "application")
  retention_in_days = local.environment == "prod" ? 90 : 7

  tags = {
    Name = format("Logs for %s", title(replace(local.project_name, "-", " ")))
    LogType = "application"
  }
}

String function categories:

Category Functions Use case Performance
Case manipulation upper(), lower(), title() Naming conventions Very high
String operations substr(), replace(), trim() Text transformation High
Formatting format(), formatdate() Template generation High
Validation regex(), can() Input validation Medium
Encoding urlencode(), base64encode() Data transfer High

String performance tips:

  • Cache complex string operations in locals
  • Use format() instead of string interpolation for better readability
  • Validate inputs early with can() and regex()
  • Use trimspace() to clean user input

Collection functions

What are collection functions? Collection functions process lists, maps and sets. They let you filter, transform, combine and organise data. These functions are the heart of dynamic Terraform configurations.

Why are collection functions indispensable? They eliminate static, repetitive configurations. Instead of defining dozens of similar resources by hand, you create them dynamically based on data. That makes infrastructure more scalable and maintainable.

What must you watch for with collection functions? Performance can suffer with large collections. Nested collection operations can become hard to read. Type consistency matters — do not mix different data types. Null values can produce unexpected results.

What do you use collection functions for? Dynamic resource creation, data aggregation, configuration merging, filtering of inputs and transformation between data formats.


# Advanced collection functions for dynamic infrastructure
locals {
  # ============================================================================
  # BASE DATA FOR COLLECTION OPERATIONS
  # ============================================================================

  # Availability zone data
  all_availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e", "us-east-1f"]

  # Environment configurations
  environment_configs = {
    dev = {
      instance_count = 1
      instance_type = "t3.micro"
      storage_size = 20
      backup_enabled = false
      monitoring_level = "basic"
      allowed_cidrs = ["10.0.0.0/16", "172.16.0.0/12"]
    }
    staging = {
      instance_count = 2
      instance_type = "t3.small"
      storage_size = 50
      backup_enabled = true
      monitoring_level = "standard"
      allowed_cidrs = ["10.0.0.0/16", "172.16.0.0/12", "192.168.0.0/16"]
    }
    prod = {
      instance_count = 5
      instance_type = "t3.large"
      storage_size = 100
      backup_enabled = true
      monitoring_level = "detailed"
      allowed_cidrs = ["10.0.0.0/16"]
    }
  }

  # Application services with different requirements
  application_services = {
    frontend = {
      port = 80
      protocol = "HTTP"
      health_check_path = "/health"
      cpu_request = "100m"
      memory_request = "128Mi"
      replicas = 3
      environment_variables = {
        NODE_ENV = "production"
        API_URL = "https://api.example.com"
        CDN_URL = "https://cdn.example.com"
      }
    }
    backend = {
      port = 8080
      protocol = "HTTP"
      health_check_path = "/api/health"
      cpu_request = "500m"
      memory_request = "512Mi"
      replicas = 5
      environment_variables = {
        DATABASE_URL = "postgresql://..."
        REDIS_URL = "redis://..."
        LOG_LEVEL = "info"
      }
    }
    worker = {
      port = 9090
      protocol = "HTTP"
      health_check_path = "/metrics"
      cpu_request = "200m"
      memory_request = "256Mi"
      replicas = 2
      environment_variables = {
        QUEUE_URL = "redis://..."
        WORKER_CONCURRENCY = "10"
        LOG_LEVEL = "debug"
      }
    }
    database = {
      port = 5432
      protocol = "TCP"
      health_check_path = null
      cpu_request = "1000m"
      memory_request = "2Gi"
      replicas = 1
      environment_variables = {
        POSTGRES_DB = "application"
        POSTGRES_USER = "app_user"
        POSTGRES_MAX_CONNECTIONS = "100"
      }
    }
  }

  # ============================================================================
  # FOR-EXPRESSIONS FOR DATA TRANSFORMATION
  # ============================================================================

  # Advanced for-expressions for various use cases
  for_expression_examples = {
    # Simple list transformation
    subnet_cidrs = [
      for i, az in slice(local.all_availability_zones, 0, 3) :
      cidrsubnet("10.0.0.0/16", 8, i)
    ]

    # Map transformation with conditional logic
    environment_instance_types = {
      for env, config in local.environment_configs :
      env => config.instance_type
      if config.instance_count > 0
    }

    # Complex object transformation
    service_configurations = {
      for service_name, service_config in local.application_services :
      service_name => {
        name = service_name
        port = service_config.port
        replicas = service_config.replicas
        resource_requests = {
          cpu = service_config.cpu_request
          memory = service_config.memory_request
        }
        environment = [
          for key, value in service_config.environment_variables :
          {
            name = key
            value = value
          }
        ]
        health_check = service_config.health_check_path != null ? {
          path = service_config.health_check_path
          port = service_config.port
        } : null
      }
    }

    # Nested for-expressions for multi-dimensional data
    all_service_environment_combinations = flatten([
      for env_name, env_config in local.environment_configs : [
        for service_name, service_config in local.application_services : {
          key = "${env_name}-${service_name}"
          environment = env_name
          service = service_name
          instance_count = env_config.instance_count
          service_replicas = service_config.replicas
          total_instances = env_config.instance_count * service_config.replicas
          resource_suffix = "${env_name}-${service_name}"
        }
        if service_name != "database" || env_name == "prod"  # Database only in production
      ]
    ])

    # Conditional for-expressions
    production_services = {
      for service_name, service_config in local.application_services :
      service_name => merge(service_config, {
        replicas = service_config.replicas * 2  # Double replicas for production
        monitoring_enabled = true
      })
      if var.environment == "prod"
    }
  }

  # ============================================================================
  # FILTER FUNCTIONS FOR DATA SELECTION
  # ============================================================================

  # Advanced filtering operations
  filtered_data = {
    # Filter availability zones (only the first 3)
    selected_azs = slice(local.all_availability_zones, 0, 3)

    # Filter services by type
    web_services = {
      for name, config in local.application_services :
      name => config
      if config.protocol == "HTTP"
    }

    # Identify high-memory services
    memory_intensive_services = [
      for name, config in local.application_services :
      name
      if can(regex("Gi", config.memory_request))
    ]

    # Environment-specific allowed CIDRs
    current_allowed_cidrs = lookup(local.environment_configs, var.environment, {}).allowed_cidrs

    # Services with health checks
    services_with_health_checks = [
      for name, config in local.application_services :
      {
        name = name
        health_check_path = config.health_check_path
        port = config.port
      }
      if config.health_check_path != null
    ]

    # Conditional service configuration based on environment
    environment_specific_services = {
      for name, config in local.application_services :
      name => merge(config, {
        # Development: reduced replicas
        replicas = var.environment == "dev" ? 1 : config.replicas
        # Production: increased resources
        cpu_request = var.environment == "prod" ? "${tonumber(split("m", config.cpu_request)[0]) * 2}m" : config.cpu_request
      })
    }
  }

  # ============================================================================
  # MERGE FUNCTIONS FOR DATA COMBINATION
  # ============================================================================

  # Advanced merge operations
  merged_configurations = {
    # Merge base tags with environment-specific tags
    base_tags = {
      ManagedBy = "terraform"
      Project = var.project_name
      CreatedAt = timestamp()
    }

    environment_tags = {
      Environment = var.environment
      CostCenter = var.environment == "prod" ? "production" : "development"
      BackupPolicy = var.environment == "prod" ? "daily" : "weekly"
    }

    # Final tags for all resources
    final_tags = merge(
      local.merged_configurations.base_tags,
      local.merged_configurations.environment_tags,
      var.additional_tags  # External tags
    )

    # Merge service configurations with environment overrides
    final_service_configs = {
      for name, config in local.application_services :
      name => merge(
        config,
        lookup(local.environment_configs[var.environment], "service_overrides", {}),
        {
          # Environment-specific adjustments
          replicas = var.environment == "dev" ? 1 : (
            var.environment == "staging" ? max(1, config.replicas - 1) : config.replicas
          )
          monitoring_enabled = var.environment == "prod"
          log_level = var.environment == "prod" ? "warn" : "debug"
        }
      )
    }

    # Merge network configuration
    network_config = merge(
      {
        # Base network settings
        vpc_cidr = "10.0.0.0/16"
        enable_nat_gateway = true
        enable_vpn_gateway = false
      },
      # Environment-specific network settings
      var.environment == "prod" ? {
        enable_nat_gateway = true
        enable_vpn_gateway = true
        enable_flow_logs = true
        enable_dns_hostnames = true
      } : {},
      # Additional custom settings
      var.custom_network_config
    )
  }

  # ============================================================================
  # FLATTEN FUNCTIONS FOR HIERARCHICAL DATA
  # ============================================================================

  # Complex flatten operations
  flattened_data = {
    # Flatten all service environment-variable combinations
    all_environment_variables = flatten([
      for service_name, service_config in local.application_services : [
        for var_name, var_value in service_config.environment_variables : {
          service = service_name
          variable_name = var_name
          variable_value = var_value
          full_key = "${service_name}_${var_name}"
        }
      ]
    ])

    # Flatten security group rules for all services
    all_security_group_rules = flatten([
      for service_name, service_config in local.application_services : [
        {
          service = service_name
          type = "ingress"
          from_port = service_config.port
          to_port = service_config.port
          protocol = "tcp"
          description = "Allow ${service_config.protocol} traffic to ${service_name}"
        },
        # Health-check port (if different)
        service_config.health_check_path != null && service_config.port != 80 ? {
          service = service_name
          type = "ingress"
          from_port = 80
          to_port = 80
          protocol = "tcp"
          description = "Allow health check traffic to ${service_name}"
        } : null
      ]
      # Filter out null values
      if service_config.port != null
    ])

    # Flatten subnet configurations for all AZs and tiers
    all_subnet_configurations = flatten([
      for tier in ["public", "private", "database"] : [
        for i, az in local.filtered_data.selected_azs : {
          name = "${var.project_name}-${var.environment}-${tier}-${i + 1}"
          tier = tier
          availability_zone = az
          cidr_block = cidrsubnet("10.0.0.0/16", 8,
            tier == "public" ? i : (
              tier == "private" ? i + 10 : i + 20
            )
          )
          map_public_ip = tier == "public"
          route_table_association = tier == "public" ? "public" : "private"
        }
      ]
    ])

    # Flatten load balancer target configurations
    load_balancer_targets = flatten([
      for service_name, service_config in local.filtered_data.web_services : [
        for i in range(service_config.replicas) : {
          service = service_name
          instance_index = i
          target_id = "${service_name}-${i}"
          port = service_config.port
          health_check_path = service_config.health_check_path
        }
      ]
    ])
  }

  # ============================================================================
  # ADVANCED COLLECTION COMBINATIONS
  # ============================================================================

  # Combine complex collection operations
  advanced_operations = {
    # Generate service-discovery configuration
    service_discovery_config = {
      for service_name, service_config in local.application_services :
      service_name => {
        name = service_name
        port = service_config.port
        instances = [
          for i in range(service_config.replicas) : {
            id = "${service_name}-${i}"
            address = "10.0.${i + 1}.${index(keys(local.application_services), service_name) + 10}"
            port = service_config.port
            health_check = service_config.health_check_path != null ? {
              http = "http://10.0.${i + 1}.${index(keys(local.application_services), service_name) + 10}:${service_config.port}${service_config.health_check_path}"
            } : null
          }
        ]
        load_balancer = {
          algorithm = "round_robin"
          health_check = service_config.health_check_path != null
        }
      }
    }

    # Monitoring configuration for all services
    monitoring_targets = merge([
      for service_name, service_config in local.application_services : {
        for i in range(service_config.replicas) :
        "${service_name}-${i}" => {
          job_name = service_name
          instance = "${service_name}-${i}"
          targets = ["10.0.${i + 1}.${index(keys(local.application_services), service_name) + 10}:${service_config.port}"]
          labels = {
            service = service_name
            environment = var.environment
            replica = tostring(i)
          }
          scrape_interval = "30s"
          metrics_path = "/metrics"
        }
      }
      if service_config.health_check_path != null  # Monitor only services with health checks
    ]...)

    # Calculate resource quotas based on service requirements
    total_resource_requirements = {
      total_cpu = sum([
        for service_name, service_config in local.application_services :
        tonumber(split("m", service_config.cpu_request)[0]) * service_config.replicas
      ])

      total_memory_mb = sum([
        for service_name, service_config in local.application_services :
        tonumber(split("Mi", service_config.memory_request)[0]) * service_config.replicas
      ])

      total_instances = sum([
        for service_name, service_config in local.application_services :
        service_config.replicas
      ])

      services_count = length(keys(local.application_services))
    }
  }
}

Collection functions in detail:

Function Purpose Input type Output type Performance
for Transformation List/Map List/Map High
filter Selection List List High
merge Combination Maps Map Very high
flatten Hierarchy flattening List of lists List Medium
slice Subset List List Very high
concat Concatenation Lists List High

🔧 Practical example - applying to resources:


# Dynamic subnet creation with collection functions
resource "aws_subnet" "main" {
  for_each = {
    for subnet in local.flattened_data.all_subnet_configurations :
    subnet.name => subnet
  }

  vpc_id                  = aws_vpc.main.id
  cidr_block              = each.value.cidr_block
  availability_zone       = each.value.availability_zone
  map_public_ip_on_launch = each.value.map_public_ip

  tags = merge(
    local.merged_configurations.final_tags,
    {
      Name = each.value.name
      Tier = each.value.tier
      AvailabilityZone = each.value.availability_zone
    }
  )
}

# Security group with dynamic rules
resource "aws_security_group" "services" {
  name_prefix = "${var.project_name}-${var.environment}-services-"
  vpc_id      = aws_vpc.main.id

  dynamic "ingress" {
    for_each = {
      for rule in local.flattened_data.all_security_group_rules :
      "${rule.service}-${rule.from_port}" => rule
      if rule.type == "ingress" && rule != null
    }

    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ["10.0.0.0/16"]
      description = ingress.value.description
    }
  }

  tags = local.merged_configurations.final_tags
}

Collection performance optimisations:

  • Use for_each instead of count for better state management
  • Cache complex collection operations in locals
  • Avoid deep nesting of for expressions
  • Use slice() for large lists instead of iterating everything

Collection pitfalls:

Problem Symptom Solution
Type inconsistency Error: Invalid value type Explicit type conversion
Null values Error: Invalid function argument Null checks with != null
Performance Slow plan times Limit collection size
Complexity Unreadable for-expressions Split into smaller operations

Conditional expressions and dynamic blocks

What are conditional expressions? Conditional expressions enable if-else logic in HCL. They use ternary operator syntax (condition ? true_value : false_value) and make configurations environment-dependent and intelligent.

Why are conditional expressions essential? They remove the need for separate configuration files per environment. A single Terraform configuration can adapt dynamically to different scenarios. That reduces code duplication and makes infrastructure more maintainable.

What are dynamic blocks? Dynamic blocks generate repeating blocks based on data. They are like loops for Terraform resource configurations and make static, repetitive definitions unnecessary.

What must you watch for with conditional logic? Complex nested conditions can become unreadable. Performance can suffer with many dynamic blocks. Type consistency between true and false values matters. Debugging gets harder with complex conditional logic.

What do you use conditional expressions and dynamic blocks for? Environment-specific configurations, feature flags, resource counts, security policies, and anywhere you need flexible, data-driven infrastructure.


# Advanced conditional expressions and dynamic blocks
locals {
  # ============================================================================
  # BASE CONDITIONAL LOGIC
  # ============================================================================

  # Environment detection and classification
  environment_classification = {
    is_production = var.environment == "prod"
    is_staging = var.environment == "staging"
    is_development = var.environment == "dev"
    is_non_production = var.environment != "prod"

    # Advanced environment checks
    requires_high_availability = contains(["prod", "staging"], var.environment)
    allows_experimental_features = contains(["dev", "sandbox"], var.environment)
    requires_compliance = var.environment == "prod"
    supports_cost_optimization = var.environment != "prod"
  }

  # Time-based conditionals
  time_based_conditions = {
    current_hour = tonumber(formatdate("hh", timestamp()))
    is_business_hours = local.time_based_conditions.current_hour >= 9 && local.time_based_conditions.current_hour <= 17
    is_weekend = contains(["saturday", "sunday"], lower(formatdate("EEEE", timestamp())))

    # Auto-shutdown logic for development
    should_auto_shutdown = (
      local.environment_classification.is_development &&
      !local.time_based_conditions.is_business_hours
    )
  }

  # ============================================================================
  # ADVANCED CONDITIONAL CONFIGURATIONS
  # ============================================================================

  # Instance configuration with complex conditional logic
  instance_configuration = {
    # Instance type based on environment and workload
    instance_type = (
      local.environment_classification.is_production ? "t3.large" :
      local.environment_classification.is_staging ? "t3.medium" :
      "t3.micro"
    )

    # Instance count with business logic
    instance_count = (
      local.environment_classification.is_production ? 5 :
      local.environment_classification.requires_high_availability ? 3 :
      1
    )

    # Storage configuration
    storage_configuration = {
      volume_type = local.environment_classification.is_production ? "gp3" : "gp2"
      volume_size = (
        local.environment_classification.is_production ? 100 :
        local.environment_classification.is_staging ? 50 :
        20
      )
      iops = local.environment_classification.is_production ? 3000 : null
      throughput = local.environment_classification.is_production ? 125 : null
      encrypted = local.environment_classification.requires_compliance
    }

    # Monitoring configuration
    monitoring = {
      enabled = local.environment_classification.requires_high_availability
      detailed_monitoring = local.environment_classification.is_production
      log_level = (
        local.environment_classification.is_production ? "ERROR" :
        local.environment_classification.is_staging ? "WARN" :
        "DEBUG"
      )
      retention_days = (
        local.environment_classification.is_production ? 90 :
        local.environment_classification.is_staging ? 30 :
        7
      )
    }
  }

  # Database configuration with conditional logic
  database_configuration = {
    # Engine-specific configuration
    engine_config = {
      engine = "mysql"
      version = local.environment_classification.is_production ? "8.0.35" : "8.0.28"
      instance_class = (
        local.environment_classification.is_production ? "db.r5.xlarge" :
        local.environment_classification.is_staging ? "db.t3.medium" :
        "db.t3.micro"
      )
    }

    # Backup strategy
    backup_strategy = {
      backup_retention_period = (
        local.environment_classification.is_production ? 30 :
        local.environment_classification.is_staging ? 7 :
        0
      )
      backup_window = local.environment_classification.requires_high_availability ? "03:00-04:00" : "06:00-07:00"
      maintenance_window = local.environment_classification.requires_high_availability ? "sun:04:00-sun:05:00" : "sun:07:00-sun:08:00"

      # Cross-region backup only for production
      copy_tags_to_snapshot = local.environment_classification.is_production
      delete_automated_backups = !local.environment_classification.requires_compliance
    }

    # Performance configuration
    performance_config = {
      multi_az = local.environment_classification.requires_high_availability
      performance_insights_enabled = local.environment_classification.is_production
      monitoring_interval = (
        local.environment_classification.is_production ? 15 :
        local.environment_classification.is_staging ? 60 :
        0
      )
    }
  }

  # ============================================================================
  # SECURITY CONFIGURATION WITH CONDITIONAL LOGIC
  # ============================================================================

  # Security policies based on environment
  security_configuration = {
    # Encryption requirements
    encryption_requirements = {
      storage_encrypted = local.environment_classification.requires_compliance
      kms_key_rotation = local.environment_classification.is_production
      backup_encryption = local.environment_classification.requires_compliance
      log_encryption = local.environment_classification.is_production
    }

    # Network security
    network_security = {
      # SSH access only in development
      allow_ssh_from_internet = local.environment_classification.is_development

      # VPC Flow Logs for compliance
      enable_vpc_flow_logs = local.environment_classification.requires_compliance

      # Network ACLs for extra security
      enable_network_acls = local.environment_classification.is_production

      # Bastion host only when needed
      deploy_bastion_host = (
        local.environment_classification.requires_high_availability &&
        !local.environment_classification.is_development
      )
    }

    # Compliance features
    compliance_features = {
      enable_cloudtrail = local.environment_classification.requires_compliance
      enable_config = local.environment_classification.requires_compliance
      enable_guardduty = local.environment_classification.is_production
      enable_security_hub = local.environment_classification.is_production

      # Audit logging
      audit_log_retention = (
        local.environment_classification.requires_compliance ? 2555 : # 7 years
        local.environment_classification.is_staging ? 365 : # 1 year
        30 # 30 days
      )
    }
  }

  # ============================================================================
  # DYNAMIC BLOCK DATA STRUCTURES
  # ============================================================================

  # Security group rules for different scenarios
  security_group_rules = {
    # Base rules for all environments
    base_rules = [
      {
        type = "egress"
        from_port = 0
        to_port = 0
        protocol = "-1"
        cidr_blocks = ["0.0.0.0/0"]
        description = "All outbound traffic"
      }
    ]

    # Web traffic rules
    web_rules = [
      {
        type = "ingress"
        from_port = 80
        to_port = 80
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
        description = "HTTP from internet"
      },
      {
        type = "ingress"
        from_port = 443
        to_port = 443
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
        description = "HTTPS from internet"
      }
    ]

    # Development-specific rules
    development_rules = local.environment_classification.is_development ? [
      {
        type = "ingress"
        from_port = 22
        to_port = 22
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
        description = "SSH for development"
      },
      {
        type = "ingress"
        from_port = 8080
        to_port = 8080
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
        description = "Development server"
      }
    ] : []

    # Monitoring rules for production
    monitoring_rules = local.environment_classification.is_production ? [
      {
        type = "ingress"
        from_port = 9090
        to_port = 9090
        protocol = "tcp"
        cidr_blocks = ["10.0.0.0/16"]
        description = "Prometheus metrics"
      },
      {
        type = "ingress"
        from_port = 3000
        to_port = 3000
        protocol = "tcp"
        cidr_blocks = ["10.0.0.0/16"]
        description = "Grafana dashboard"
      }
    ] : []

    # Combined rules
    all_rules = concat(
      local.security_group_rules.base_rules,
      local.security_group_rules.web_rules,
      local.security_group_rules.development_rules,
      local.security_group_rules.monitoring_rules
    )
  }

  # Auto Scaling policies for different scenarios
  autoscaling_policies = {
    # CPU-based scaling
    cpu_policies = local.environment_classification.requires_high_availability ? [
      {
        name = "scale-up-cpu"
        scaling_adjustment = 2
        adjustment_type = "ChangeInCapacity"
        cooldown = 300
        metric_name = "CPUUtilization"
        threshold = 70
        comparison_operator = "GreaterThanThreshold"
        evaluation_periods = 2
        period = 120
      },
      {
        name = "scale-down-cpu"
        scaling_adjustment = -1
        adjustment_type = "ChangeInCapacity"
        cooldown = 300
        metric_name = "CPUUtilization"
        threshold = 30
        comparison_operator = "LessThanThreshold"
        evaluation_periods = 2
        period = 120
      }
    ] : []

    # Memory-based scaling for production
    memory_policies = local.environment_classification.is_production ? [
      {
        name = "scale-up-memory"
        scaling_adjustment = 1
        adjustment_type = "ChangeInCapacity"
        cooldown = 600
        metric_name = "MemoryUtilization"
        threshold = 80
        comparison_operator = "GreaterThanThreshold"
        evaluation_periods = 3
        period = 300
      }
    ] : []

    # Schedule-based scaling for cost optimisation
    schedule_policies = local.environment_classification.supports_cost_optimization ? [
      {
        name = "scale-down-night"
        min_size = 0
        max_size = 1
        desired_capacity = 0
        recurrence = "0 22 * * MON-FRI"  # 22:00 on weekdays
      },
      {
        name = "scale-up-morning"
        min_size = 1
        max_size = 3
        desired_capacity = 1
        recurrence = "0 8 * * MON-FRI"   # 08:00 on weekdays
      }
    ] : []

    # All policies combined
    all_policies = concat(
      local.autoscaling_policies.cpu_policies,
      local.autoscaling_policies.memory_policies,
      local.autoscaling_policies.schedule_policies
    )
  }
}

# Advanced resources with dynamic blocks and conditional logic
resource "aws_instance" "application" {
  count = local.instance_configuration.instance_count

  ami           = data.aws_ami.ubuntu.id
  instance_type = local.instance_configuration.instance_type

  # Conditional subnet placement
  subnet_id = var.subnet_ids[count.index % length(var.subnet_ids)]

  # Conditional storage configuration
  root_block_device {
    volume_type = local.instance_configuration.storage_configuration.volume_type
    volume_size = local.instance_configuration.storage_configuration.volume_size
    encrypted   = local.instance_configuration.storage_configuration.encrypted

    # Conditional IOPS and throughput for gp3
    iops       = local.instance_configuration.storage_configuration.volume_type == "gp3" ? local.instance_configuration.storage_configuration.iops : null
    throughput = local.instance_configuration.storage_configuration.volume_type == "gp3" ? local.instance_configuration.storage_configuration.throughput : null
  }

  # Conditional monitoring
  monitoring = local.instance_configuration.monitoring.detailed_monitoring

  # Dynamic user data based on environment
  user_data = base64encode(templatefile("${path.module}/userdata.sh", {
    environment = var.environment
    instance_index = count.index
    log_level = local.instance_configuration.monitoring.log_level
    monitoring_enabled = local.instance_configuration.monitoring.enabled
    is_production = local.environment_classification.is_production
  }))

  # Conditional auto-shutdown for development
  dynamic "credit_specification" {
    for_each = local.environment_classification.supports_cost_optimization && startswith(local.instance_configuration.instance_type, "t") ? [1] : []
    content {
      cpu_credits = "unlimited"
    }
  }

  tags = merge(
    var.base_tags,
    {
      Name = "${var.project_name}-${var.environment}-app-${count.index + 1}"
      Environment = var.environment
      InstanceIndex = count.index
      AutoShutdown = local.time_based_conditions.should_auto_shutdown
    }
  )

  lifecycle {
    # Conditional lifecycle rules
    create_before_destroy = local.environment_classification.requires_high_availability
    ignore_changes = local.environment_classification.is_development ? ["ami"] : []
  }
}

# Security group with dynamic blocks for flexible rules
resource "aws_security_group" "application" {
  name_prefix = "${var.project_name}-${var.environment}-app-"
  vpc_id      = var.vpc_id

  # Dynamic ingress rules
  dynamic "ingress" {
    for_each = {
      for rule in local.security_group_rules.all_rules :
      "${rule.type}-${rule.from_port}-${rule.to_port}" => rule
      if rule.type == "ingress"
    }

    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
      description = ingress.value.description
    }
  }

  # Dynamic egress rules
  dynamic "egress" {
    for_each = {
      for rule in local.security_group_rules.all_rules :
      "${rule.type}-${rule.from_port}-${rule.to_port}" => rule
      if rule.type == "egress"
    }

    content {
      from_port   = egress.value.from_port
      to_port     = egress.value.to_port
      protocol    = egress.value.protocol
      cidr_blocks = egress.value.cidr_blocks
      description = egress.value.description
    }
  }

  tags = {
    Name = "${var.project_name}-${var.environment}-app-sg"
    Environment = var.environment
    RuleCount = length(local.security_group_rules.all_rules)
  }
}

# Auto Scaling group with dynamic scaling policies
resource "aws_autoscaling_group" "application" {
  count = local.environment_classification.requires_high_availability ? 1 : 0

  name                = "${var.project_name}-${var.environment}-asg"
  vpc_zone_identifier = var.subnet_ids

  min_size         = local.instance_configuration.instance_count
  max_size         = local.instance_configuration.instance_count * 3
  desired_capacity = local.instance_configuration.instance_count

  launch_template {
    id      = aws_launch_template.application.id
    version = "$Latest"
  }

  # Conditional health check
  health_check_type         = local.environment_classification.is_production ? "ELB" : "EC2"
  health_check_grace_period = local.environment_classification.is_production ? 300 : 60

  # Dynamic tags
  dynamic "tag" {
    for_each = merge(
      var.base_tags,
      {
        Name = "${var.project_name}-${var.environment}-asg"
        Environment = var.environment
        AutoScaling = "enabled"
      }
    )

    content {
      key                 = tag.key
      value               = tag.value
      propagate_at_launch = true
    }
  }
}

# CloudWatch alarms with dynamic blocks
resource "aws_cloudwatch_metric_alarm" "application_alarms" {
  for_each = {
    for policy in local.autoscaling_policies.all_policies :
    policy.name => policy
    if policy.metric_name != null
  }

  alarm_name          = "${var.project_name}-${var.environment}-${each.value.name}"
  comparison_operator = each.value.comparison_operator
  evaluation_periods  = each.value.evaluation_periods
  metric_name         = each.value.metric_name
  namespace           = "AWS/EC2"
  period              = each.value.period
  statistic           = "Average"
  threshold           = each.value.threshold
  alarm_description   = "Auto scaling alarm for ${each.value.name}"

  # Conditional alarm actions
  alarm_actions = local.environment_classification.requires_high_availability ? [
    aws_autoscaling_policy.application_policies[each.key].arn
  ] : []

  # Dynamic dimensions
  dynamic "dimensions" {
    for_each = local.environment_classification.requires_high_availability ? [1] : []
    content {
      AutoScalingGroupName = aws_autoscaling_group.application[0].name
    }
  }

  tags = {
    Name = "${var.project_name}-${var.environment}-${each.value.name}-alarm"
    Environment = var.environment
    MetricName = each.value.metric_name
  }
}

Conditional logic flow diagram:


┌─────────────────────────────────────────────────────────────┐
│                   Conditional Logic Flow                    │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│                      Environment Input                      │
│                      (var.environment)                      │
│                              │                              │
│                              ▼                              │
│               ┌─────────────────────────────┐               │
│               │ Environment Classification  │               │
│               │ ─────────────────────────── │               │
│               │ - is_production             │               │
│               │ - requires_ha               │               │
│               │ - allows_experimental       │               │
│               └──────────────┬──────────────┘               │
│                              │                              │
│              ┌───────────────┴───────────────┐              │
│              ▼                               ▼              │
│  ┌───────────────────────┐       ┌───────────────────────┐  │
│  │ Instance Config       │       │ Security Config       │  │
│  │ ───────────────────── │       │ ───────────────────── │  │
│  │ - type: conditional   │       │ - enc: conditional    │  │
│  │ - count: conditional  │       │ - mon: conditional    │  │
│  │ - disk: conditional   │       │ - comp: conditional   │  │
│  └───────────┬───────────┘       └───────────┬───────────┘  │
│              │                               │              │
│              ▼                               │              │
│  ┌───────────────────────┐                   │              │
│  │ Dynamic Blocks        │                   │              │
│  │ ───────────────────── │                   │              │
│  │ - security_rules      │                   │              │
│  │ - scaling_policies    │                   │              │
│  │ - monitoring_targets  │                   │              │
│  └───────────┬───────────┘                   │              │
│              │                               │              │
│              └───────────────┬───────────────┘              │
│                              │                              │
│                              ▼                              │
│               ┌─────────────────────────────┐               │
│               │ Resource Configuration      │               │
│               │ (Evaluated Final State)     │               │
│               └─────────────────────────────┘               │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Conditional logic best practices:

Pattern Use Performance Maintainability
Ternary operator Simple if-else Very high High
Complex conditions Multi-criteria High Medium
Dynamic blocks Repeating structure Medium High
for_each with conditions Conditional resources Medium Very high

Conditional logic optimisations:

  • Use for_each instead of count for better state management
  • Cache complex collection operations in locals
  • Avoid deep nesting of for-expressions
  • Use slice() for large lists instead of iterating everything

Conditional logic pitfalls:

Problem Symptom Solution
Type mismatch Error: Inconsistent conditional result types Same types in both branches
Null conditions Error: Invalid value for conditional Add null checks
Performance Slow evaluation Cache conditions in locals
Debugging Hard-to-follow logic Use simpler conditions

🔧 Practical example - debugging conditional logic:


# Test conditional logic in terraform console
terraform console
> local.environment_classification.is_production
> local.instance_configuration.instance_type
> local.security_group_rules.all_rules

# Inspect dynamic block contents
terraform plan | grep -A 10 "dynamic"

# Validate conditional resources
terraform state list | grep conditional_resource

For-expressions for complex transformations

What are for-expressions? For-expressions are one of the most powerful HCL features for data transformation. They combine the flexibility of loops with the expressiveness of conditional logic and let you build complex data structures in a single, elegant expression. For-expressions are like SQL queries for your Terraform configuration.

Why do for-expressions matter so much? They turn static Terraform configurations into dynamic, data-driven infrastructure definitions. Without them you would have to define repetitive resources by hand or use external scripts. With them you create flexible, scalable configurations that adapt automatically to changing data structures.

What must you watch for with for-expressions? Complex nested for-expressions can be hard to read and debug. Performance can suffer on large datasets. Type safety matters — make sure every iteration returns consistent data types. Missing null checks can cause runtime errors.

What do you use for-expressions for? Dynamic resource generation, data aggregation, complex transformations between data formats, filtering with business logic, and anywhere you must build one data structure from another.

Basic for-expression syntax


# Advanced for-expressions for complex data transformations
locals {
  # ============================================================================
  # BASE DATA FOR FOR-EXPRESSION EXAMPLES
  # ============================================================================

  # Complex input data
  application_services = {
    frontend = {
      name = "frontend"
      port = 3000
      replicas = 3
      cpu_limit = "500m"
      memory_limit = "512Mi"
      environment = "production"
      health_check = "/health"
      dependencies = ["backend", "cache"]
      labels = {
        tier = "presentation"
        version = "v1.2.3"
        team = "frontend-team"
      }
      volumes = [
        {
          name = "config"
          path = "/app/config"
          size = "1Gi"
        },
        {
          name = "logs"
          path = "/app/logs"
          size = "5Gi"
        }
      ]
    }
    backend = {
      name = "backend"
      port = 8080
      replicas = 5
      cpu_limit = "1000m"
      memory_limit = "1Gi"
      environment = "production"
      health_check = "/api/health"
      dependencies = ["database", "cache"]
      labels = {
        tier = "application"
        version = "v2.1.0"
        team = "backend-team"
      }
      volumes = [
        {
          name = "data"
          path = "/app/data"
          size = "10Gi"
        }
      ]
    }
    worker = {
      name = "worker"
      port = 9000
      replicas = 2
      cpu_limit = "750m"
      memory_limit = "768Mi"
      environment = "production"
      health_check = "/worker/health"
      dependencies = ["database", "queue"]
      labels = {
        tier = "worker"
        version = "v1.0.5"
        team = "backend-team"
      }
      volumes = []
    }
    cache = {
      name = "cache"
      port = 6379
      replicas = 1
      cpu_limit = "250m"
      memory_limit = "256Mi"
      environment = "production"
      health_check = "/ping"
      dependencies = []
      labels = {
        tier = "cache"
        version = "v6.2.0"
        team = "platform-team"
      }
      volumes = [
        {
          name = "cache-data"
          path = "/data"
          size = "2Gi"
        }
      ]
    }
  }

  # Availability zones and regions
  regions = {
    "us-east-1" = {
      azs = ["us-east-1a", "us-east-1b", "us-east-1c"]
      vpc_cidr = "10.0.0.0/16"
      instance_types = ["t3.micro", "t3.small", "t3.medium"]
    }
    "us-west-2" = {
      azs = ["us-west-2a", "us-west-2b", "us-west-2c"]
      vpc_cidr = "10.1.0.0/16"
      instance_types = ["t3.small", "t3.medium", "t3.large"]
    }
    "eu-west-1" = {
      azs = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
      vpc_cidr = "10.2.0.0/16"
      instance_types = ["t3.medium", "t3.large", "t3.xlarge"]
    }
  }

  # Environment configurations
  environments = {
    dev = {
      replica_multiplier = 0.5
      cpu_multiplier = 0.5
      memory_multiplier = 0.5
      storage_multiplier = 0.5
      monitoring_enabled = false
      backup_enabled = false
    }
    staging = {
      replica_multiplier = 0.8
      cpu_multiplier = 0.8
      memory_multiplier = 0.8
      storage_multiplier = 0.8
      monitoring_enabled = true
      backup_enabled = true
    }
    prod = {
      replica_multiplier = 1.0
      cpu_multiplier = 1.0
      memory_multiplier = 1.0
      storage_multiplier = 1.0
      monitoring_enabled = true
      backup_enabled = true
    }
  }

  # ============================================================================
  # BASIC FOR-EXPRESSION PATTERNS
  # ============================================================================

  # Simple list comprehension
  basic_list_transformations = {
    # Extract service names
    service_names = [
      for service_name, config in local.application_services :
      service_name
    ]

    # Extract ports
    service_ports = [
      for service_name, config in local.application_services :
      config.port
    ]

    # Service names in uppercase
    service_names_upper = [
      for service_name, config in local.application_services :
      upper(service_name)
    ]

    # Combined values
    service_endpoints = [
      for service_name, config in local.application_services :
      "${service_name}:${config.port}"
    ]
  }

  # Simple object comprehension
  basic_object_transformations = {
    # Port mapping
    port_mapping = {
      for service_name, config in local.application_services :
      service_name => config.port
    }

    # Health-check mapping
    health_check_mapping = {
      for service_name, config in local.application_services :
      service_name => config.health_check
    }

    # Replica mapping
    replica_mapping = {
      for service_name, config in local.application_services :
      service_name => config.replicas
    }

    # Label-tier mapping
    tier_mapping = {
      for service_name, config in local.application_services :
      service_name => config.labels.tier
    }
  }

  # ============================================================================
  # CONDITIONAL FOR-EXPRESSIONS
  # ============================================================================

  # For-expressions with conditional logic
  conditional_transformations = {
    # Web services only (frontend/backend)
    web_services = {
      for service_name, config in local.application_services :
      service_name => config
      if contains(["frontend", "backend"], service_name)
    }

    # Services with high memory demand
    memory_intensive_services = [
      for service_name, config in local.application_services :
      service_name
      if tonumber(split("Mi", config.memory_limit)[0]) > 500
    ]

    # Services with dependencies
    services_with_dependencies = {
      for service_name, config in local.application_services :
      service_name => config.dependencies
      if length(config.dependencies) > 0
    }

    # Services with volumes
    services_with_storage = {
      for service_name, config in local.application_services :
      service_name => {
        volumes = config.volumes
        total_storage = sum([
          for volume in config.volumes :
          tonumber(split("Gi", volume.size)[0])
        ])
      }
      if length(config.volumes) > 0
    }

    # Team-based grouping
    backend_team_services = [
      for service_name, config in local.application_services :
      service_name
      if config.labels.team == "backend-team"
    ]

    # Environment-specific configuration
    production_ready_services = {
      for service_name, config in local.application_services :
      service_name => merge(config, {
        # Production-specific adjustments
        replicas = max(2, config.replicas)  # Minimum 2 replicas
        monitoring_enabled = true
        backup_enabled = true
      })
      if config.environment == "production"
    }
  }

  # ============================================================================
  # NESTED FOR-EXPRESSIONS
  # ============================================================================

  # Complex nested transformations
  nested_transformations = {
    # All service-volume combinations
    all_service_volumes = flatten([
      for service_name, config in local.application_services : [
        for volume in config.volumes : {
          service = service_name
          volume_name = volume.name
          volume_path = volume.path
          volume_size = volume.size
          full_name = "${service_name}-${volume.name}"
        }
      ]
    ])

    # Service dependency graph
    service_dependency_graph = {
      for service_name, config in local.application_services :
      service_name => {
        depends_on = config.dependencies
        dependents = [
          for other_service, other_config in local.application_services :
          other_service
          if contains(other_config.dependencies, service_name)
        ]
        total_connections = length(config.dependencies) + length([
          for other_service, other_config in local.application_services :
          other_service
          if contains(other_config.dependencies, service_name)
        ])
      }
    }

    # Multi-region service deployment
    multi_region_deployments = {
      for region_name, region_config in local.regions :
      region_name => {
        region = region_name
        vpc_cidr = region_config.vpc_cidr
        services = {
          for service_name, service_config in local.application_services :
          service_name => {
            service = service_name
            region = region_name
            subnets = [
              for i, az in region_config.azs :
              {
                name = "${service_name}-${region_name}-${i + 1}"
                az = az
                cidr = cidrsubnet(region_config.vpc_cidr, 8, i)
                instance_type = region_config.instance_types[i % length(region_config.instance_types)]
              }
            ]
            load_balancer_targets = [
              for i, az in region_config.azs :
              {
                target_id = "${service_name}-${region_name}-${i + 1}"
                az = az
                port = service_config.port
                health_check = service_config.health_check
              }
            ]
          }
        }
      }
    }

    # Environment-service matrix
    environment_service_matrix = {
      for env_name, env_config in local.environments :
      env_name => {
        environment = env_name
        services = {
          for service_name, service_config in local.application_services :
          service_name => {
            name = service_name
            environment = env_name
            replicas = max(1, floor(service_config.replicas * env_config.replica_multiplier))
            cpu_limit = "${floor(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier)}m"
            memory_limit = "${floor(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier)}Mi"
            monitoring_enabled = env_config.monitoring_enabled
            backup_enabled = env_config.backup_enabled
            volumes = [
              for volume in service_config.volumes :
              {
                name = volume.name
                path = volume.path
                size = "${floor(tonumber(split("Gi", volume.size)[0]) * env_config.storage_multiplier)}Gi"
              }
            ]
            full_name = "${service_name}-${env_name}"
          }
        }
        total_replicas = sum([
          for service_name, service_config in local.application_services :
          max(1, floor(service_config.replicas * env_config.replica_multiplier))
        ])
        total_cpu = sum([
          for service_name, service_config in local.application_services :
          floor(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier)
        ])
        total_memory = sum([
          for service_name, service_config in local.application_services :
          floor(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier)
        ])
      }
    }
  }

  # ============================================================================
  # ADVANCED FOR-EXPRESSION PATTERNS
  # ============================================================================

  # Complex data aggregation
  advanced_aggregations = {
    # Team-based statistics
    team_statistics = {
      for team in distinct([
        for service_name, config in local.application_services :
        config.labels.team
      ]) :
      team => {
        team_name = team
        services = [
          for service_name, config in local.application_services :
          service_name
          if config.labels.team == team
        ]
        total_replicas = sum([
          for service_name, config in local.application_services :
          config.replicas
          if config.labels.team == team
        ])
        total_cpu = sum([
          for service_name, config in local.application_services :
          tonumber(split("m", config.cpu_limit)[0])
          if config.labels.team == team
        ])
        total_memory = sum([
          for service_name, config in local.application_services :
          tonumber(split("Mi", config.memory_limit)[0])
          if config.labels.team == team
        ])
        service_count = length([
          for service_name, config in local.application_services :
          service_name
          if config.labels.team == team
        ])
      }
    }

    # Tier-based configuration
    tier_configurations = {
      for tier in distinct([
        for service_name, config in local.application_services :
        config.labels.tier
      ]) :
      tier => {
        tier_name = tier
        services = {
          for service_name, config in local.application_services :
          service_name => {
            port = config.port
            replicas = config.replicas
            health_check = config.health_check
            dependencies = config.dependencies
            resource_requirements = {
              cpu = config.cpu_limit
              memory = config.memory_limit
            }
          }
          if config.labels.tier == tier
        }
        load_balancer_config = {
          enabled = contains(["presentation", "application"], tier)
          port = tier == "presentation" ? 80 : 8080
          health_check_path = "/health"
          targets = [
            for service_name, config in local.application_services :
            {
              service = service_name
              port = config.port
              health_check = config.health_check
            }
            if config.labels.tier == tier
          ]
        }
        security_group_rules = [
          for service_name, config in local.application_services : {
            service = service_name
            port = config.port
            tier = tier
            rule = {
              type = "ingress"
              from_port = config.port
              to_port = config.port
              protocol = "tcp"
              description = "Allow traffic to ${service_name} in ${tier} tier"
              source_tier = tier == "presentation" ? "internet" : "application"
            }
          }
          if config.labels.tier == tier
        ]
      }
    }

    # Dependency-chain analysis
    dependency_chains = {
      for service_name, config in local.application_services :
      service_name => {
        service = service_name
        direct_dependencies = config.dependencies
        indirect_dependencies = flatten([
          for dep in config.dependencies : [
            for indirect_dep in lookup(local.application_services, dep, {}).dependencies :
            indirect_dep
            if !contains(config.dependencies, indirect_dep)
          ]
        ])
        all_dependencies = distinct(concat(
          config.dependencies,
          flatten([
            for dep in config.dependencies : [
              for indirect_dep in lookup(local.application_services, dep, {}).dependencies :
              indirect_dep
              if !contains(config.dependencies, indirect_dep)
            ]
          ])
        ))
        dependency_depth = length(distinct(concat(
          config.dependencies,
          flatten([
            for dep in config.dependencies : [
              for indirect_dep in lookup(local.application_services, dep, {}).dependencies :
              indirect_dep
              if !contains(config.dependencies, indirect_dep)
            ]
          ])
        )))
        is_leaf_service = length(config.dependencies) == 0
        is_root_service = length([
          for other_service, other_config in local.application_services :
          other_service
          if contains(other_config.dependencies, service_name)
        ]) == 0
      }
    }
  }

  # ============================================================================
  # PERFORMANCE-OPTIMISED FOR-EXPRESSIONS
  # ============================================================================

  # Optimised transformations for large datasets
  optimized_transformations = {
    # Service lookup table (fast access)
    service_lookup = {
      for service_name, config in local.application_services :
      service_name => {
        port = config.port
        health_check = config.health_check
        tier = config.labels.tier
        team = config.labels.team
      }
    }

    # Port index for fast search
    port_index = {
      for service_name, config in local.application_services :
      config.port => service_name
    }

    # Team index for grouped operations
    team_index = {
      for team in distinct([
        for service_name, config in local.application_services :
        config.labels.team
      ]) :
      team => [
        for service_name, config in local.application_services :
        service_name
        if config.labels.team == team
      ]
    }

    # Tier index for load balancer configuration
    tier_index = {
      for tier in distinct([
        for service_name, config in local.application_services :
        config.labels.tier
      ]) :
      tier => {
        services = [
          for service_name, config in local.application_services :
          service_name
          if config.labels.tier == tier
        ]
        total_replicas = sum([
          for service_name, config in local.application_services :
          config.replicas
          if config.labels.tier == tier
        ])
        needs_load_balancer = contains(["presentation", "application"], tier)
      }
    }

    # Cached computed values
    resource_summary = {
      total_services = length(keys(local.application_services))
      total_replicas = sum([
        for service_name, config in local.application_services :
        config.replicas
      ])
      total_cpu_millicores = sum([
        for service_name, config in local.application_services :
        tonumber(split("m", config.cpu_limit)[0]) * config.replicas
      ])
      total_memory_mi = sum([
        for service_name, config in local.application_services :
        tonumber(split("Mi", config.memory_limit)[0]) * config.replicas
      ])
      total_storage_gi = sum([
        for service_name, config in local.application_services :
        sum([
          for volume in config.volumes :
          tonumber(split("Gi", volume.size)[0])
        ])
      ])
      unique_teams = length(distinct([
        for service_name, config in local.application_services :
        config.labels.team
      ]))
      unique_tiers = length(distinct([
        for service_name, config in local.application_services :
        config.labels.tier
      ]))
    }
  }
}

For-expression syntax variants

Syntax Purpose Example Output type
[for item in list : expression] List comprehension [for s in services : s.name] List
{for key, value in map : key => expression} Object comprehension {for k, v in map : k => v.port} Map
[for item in list : expression if condition] Conditional list [for s in services : s if s.enabled] List
{for key, value in map : key => expression if condition} Conditional object {for k, v in map : k => v if v.active} Map

Practical use in real resources


# Dynamic Kubernetes deployments with for-expressions
resource "kubernetes_deployment" "services" {
  for_each = local.conditional_transformations.web_services

  metadata {
    name = each.key
    labels = merge(
      each.value.labels,
      {
        managed-by = "terraform"
        environment = var.environment
      }
    )
  }

  spec {
    replicas = each.value.replicas

    selector {
      match_labels = {
        app = each.key
      }
    }

    template {
      metadata {
        labels = merge(
          each.value.labels,
          {
            app = each.key
          }
        )
      }

      spec {
        container {
          name  = each.key
          image = "${each.key}:${each.value.labels.version}"

          port {
            container_port = each.value.port
          }

          # Dynamic environment variables
          dynamic "env" {
            for_each = {
              for key, value in merge(
                {
                  SERVICE_NAME = each.key
                  SERVICE_PORT = tostring(each.value.port)
                  ENVIRONMENT = var.environment
                },
                # Service-specific environment variables
                lookup(local.service_environment_variables, each.key, {})
              ) :
              key => value
            }

            content {
              name  = env.key
              value = env.value
            }
          }

          # Dynamic volume mounts
          dynamic "volume_mount" {
            for_each = {
              for volume in each.value.volumes :
              volume.name => volume
            }

            content {
              name       = volume_mount.key
              mount_path = volume_mount.value.path
            }
          }

          # Health check
          liveness_probe {
            http_get {
              path = each.value.health_check
              port = each.value.port
            }
            initial_delay_seconds = 30
            period_seconds = 10
          }

          readiness_probe {
            http_get {
              path = each.value.health_check
              port = each.value.port
            }
            initial_delay_seconds = 5
            period_seconds = 5
          }

          resources {
            limits = {
              cpu    = each.value.cpu_limit
              memory = each.value.memory_limit
            }
            requests = {
              cpu    = "${floor(tonumber(split("m", each.value.cpu_limit)[0]) * 0.5)}m"
              memory = "${floor(tonumber(split("Mi", each.value.memory_limit)[0]) * 0.5)}Mi"
            }
          }
        }

        # Dynamic volumes
        dynamic "volume" {
          for_each = {
            for volume in each.value.volumes :
            volume.name => volume
          }

          content {
            name = volume.key
            persistent_volume_claim {
              claim_name = "${each.key}-${volume.key}-pvc"
            }
          }
        }
      }
    }
  }
}

# Load balancer services with for-expressions
resource "kubernetes_service" "load_balancers" {
  for_each = local.nested_transformations.multi_region_deployments["us-east-1"].services

  metadata {
    name = "${each.key}-lb"
    labels = {
      service = each.key
      type = "load-balancer"
    }
  }

  spec {
    selector = {
      app = each.key
    }

    # Dynamic ports
    dynamic "port" {
      for_each = {
        for target in each.value.load_balancer_targets :
        target.target_id => target
      }

      content {
        name        = "http"
        port        = 80
        target_port = port.value.port
        protocol    = "TCP"
      }
    }

    type = "LoadBalancer"
  }
}

# Persistent volume claims with for-expressions
resource "kubernetes_persistent_volume_claim" "service_storage" {
  for_each = {
    for volume_config in local.nested_transformations.all_service_volumes :
    volume_config.full_name => volume_config
  }

  metadata {
    name = "${each.key}-pvc"
    labels = {
      service = each.value.service
      volume = each.value.volume_name
    }
  }

  spec {
    access_modes = ["ReadWriteOnce"]
    resources {
      requests = {
        storage = each.value.volume_size
      }
    }
    storage_class_name = "gp2"
  }
}

For-expression performance optimisation:

Data preparation

  • Use small datasets
  • Remove unnecessary fields
  • Indexing for lookups
  • Caching in locals

Expression optimisation

  • Prefer simple expressions
  • Minimise nesting
  • Apply conditional logic early
  • Flatten only when needed

Iteration efficiency

  • for_each > count
  • Distinct() for duplicates
  • Slice() for subsets
  • Lookup() for access

Memory management

  • Avoid large lists
  • Use streaming patterns
  • Watch garbage collection
  • Minimise state size

For-expression debugging strategies:

Debugging technique Use Example
Step-by-step build Split complex expressions One transformation per step
Terraform console Interactive tests terraform console
Print debugging Outputs for intermediate results output "debug" { value = local.test }
Type validation Check consistency can() for type checks

For-expression best practices:

  • Use meaningful variable names in iterations
  • Keep expressions as simple as possible
  • Cache complex calculations in locals
  • Use conditional logic for performance optimisation
  • Document complex transformations in detail

For-expression pitfalls:

Problem Symptom Solution
Circular dependencies Error: Cycle in values Restructure dependencies
Type inconsistency Error: Inconsistent types Explicit type conversion
Performance problems Slow plan times Optimise expressions
Memory exhaustion Error: Out of memory Shrink datasets
Complex nested loops Unreadable expressions Split into several steps

For-expression debugging techniques


# Test for-expression results in terraform console
terraform console
> [for s in local.application_services : s.name]
> {for k, v in local.application_services : k => v.port}
> local.nested_transformations.all_service_volumes

# Step-by-step debugging
terraform console
> local.application_services
> [for s in local.application_services : s]
> [for s in local.application_services : s.name]
> [for s in local.application_services : s.name if s.replicas > 2]

# Performance analysis
terraform plan -detailed-exitcode
time terraform plan

Advanced for-expression patterns


# Advanced patterns for production code
locals {
  # Pattern 1: flatten hierarchical data
  flattened_configuration = flatten([
    for region_name, region in local.regions : [
      for service_name, service in local.application_services : {
        key = "${region_name}-${service_name}"
        region = region_name
        service = service_name
        az_count = length(region.azs)
        vpc_cidr = region.vpc_cidr
        service_port = service.port
        service_replicas = service.replicas
        full_config = merge(service, {
          region = region_name
          azs = region.azs
        })
      }
    ]
  ])

  # Pattern 2: lookup tables for performance
  service_by_port = {
    for service_name, config in local.application_services :
    config.port => service_name
  }

  # Pattern 3: conditional aggregation
  resource_allocation = {
    for env_name, env_config in local.environments :
    env_name => {
      services = {
        for service_name, service_config in local.application_services :
        service_name => {
          replicas = max(1, ceil(service_config.replicas * env_config.replica_multiplier))
          cpu_total = ceil(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
          memory_total = ceil(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
        }
      }
      totals = {
        cpu = sum([
          for service_name, service_config in local.application_services :
          ceil(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
        ])
        memory = sum([
          for service_name, service_config in local.application_services :
          ceil(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
        ])
        replicas = sum([
          for service_name, service_config in local.application_services :
          max(1, ceil(service_config.replicas * env_config.replica_multiplier))
        ])
      }
    }
  }

  # Pattern 4: error handling with can()
  safe_transformations = {
    for service_name, config in local.application_services :
    service_name => {
      cpu_millicores = can(tonumber(split("m", config.cpu_limit)[0])) ? tonumber(split("m", config.cpu_limit)[0]) : 0
      memory_mi = can(tonumber(split("Mi", config.memory_limit)[0])) ? tonumber(split("Mi", config.memory_limit)[0]) : 0
      has_health_check = can(config.health_check) && config.health_check != null
      volume_count = can(length(config.volumes)) ? length(config.volumes) : 0
      has_dependencies = can(length(config.dependencies)) ? length(config.dependencies) > 0 : false
    }
  }
}

For-expressions are the most powerful tool for data transformation in Terraform. They let you turn static configurations into dynamic, data-driven infrastructure. With list comprehensions, object comprehensions, conditional logic and nested loops you implement complex transformations in elegant, readable expressions. That command of for-expressions is what makes HCL configurations adapt automatically to changing requirements.

Command Reference (Cheatsheet)

For quick access while developing complex Terraform configurations, the following table summarises the most important HCL built-in functions, data transformations and expression syntax patterns:

Function / syntax Category Description and practical use
lookup(map, key, default) Map function Reads a key from a map and returns a default if it is missing.
merge(map1, map2, ...) Collection Merges several maps; later values overwrite earlier keys.
flatten(list_of_lists) Collection Reduces multi-dimensional nested lists to a flat one-dimensional list.
keys(map) / values(map) Map function Returns all keys or all values of a map as a list.
element(list, index) List function Returns the element at an index with automatic wrap-around (modulo).
slice(list, start, end) List function Extracts a sublist from the start index to the end index.
concat(list1, list2, ...) List function Joins two or more lists into a single list.
distinct(list) List function Removes all duplicates from a list and keeps the original order.
compact(list_of_strings) List function Removes all empty strings "" from a list of strings.
contains(list, value) Collection Checks whether a given value exists in a list or set (true/false).
try(expr1, expr2, ...) Error handling Evaluates expressions in order and returns the first error-free value.
can(expression) Error handling Checks whether an expression can be evaluated without error (ideal for validations).
coalesce(v1, v2, ...) Conditional Returns the first value that is neither null nor an empty string "".
templatefile(path, vars) String function Renders an external template file and substitutes HCL variables.
jsonencode(value) Encoding Serialises arbitrary HCL data structures (objects, lists) into valid JSON.
jsondecode(string) Encoding Parses a JSON string into native HCL data structures.
yamldecode(string) Encoding Parses YAML documents directly into HCL maps or lists.
sensitive(value) Security Marks a value as sensitive so it is masked in the plan and in CLI logs.
nonsensitive(value) Security Removes the sensitive flag for non-critical follow-on operations.
[for x in list : f(x)] For-expression List comprehension: transforms every element of a list into a new format.
{for k, v in map : k => f(v)} For-expression Map comprehension: builds a new map with modified keys or values.
[for x in list : x if condition] For-expression Filters elements based on a boolean condition.
dynamic "block" { for_each = ... } Dynamic block Generates repeated resource blocks dynamically from collections.

Further Resources

The following official documentation, registry directories and best-practice guides go deeper into the advanced HCL concepts covered above:

Resource Description
Terraform Configuration Language Official language specification for HCL syntax, expressions and the type system.
Terraform Built-in Functions Reference Complete reference of all mathematical, string and collection functions.
Terraform Developer Hub Central place for HashiCorp documentation, tutorials and best practices.
Terraform CLI Documentation Command reference for the Terraform CLI and debugging options.
Terraform Registry Public directory for official and verified providers and modules.
AWS Provider Documentation Official documentation of all AWS resources, attributes and IAM roles.
Azure Provider Documentation Specification of the Microsoft Azure Resource Manager (azurerm) provider.
Google Cloud Terraform Documentation Guide for Google Cloud Platform infrastructure with HashiCorp Terraform.

💡 Tip: Start with the official documentation and the free resources. Once you have practical experience, deeper books and architecture blueprints are a worthwhile investment for architectural understanding.

Conclusion

With advanced HCL syntax in place you have the foundation for professional Terraform development. Complex data structures, intelligent validations, secure credential management and powerful built-in functions turn infrastructure definitions from static scripts into dynamic, self-adapting systems.

That language expertise is the prerequisite for every further Terraform challenge. You now write maintainable, reusable code that adapts cleanly to different environments and stays readable.

💡 Coming next: Part three of the series covers Terraform state management — the core of every professional Terraform implementation. Remote state, locking mechanisms, team workflows and the practices that make infrastructure scalable and team-ready.

👉 Overview: All DevOps articles and guides

Share & export

Export as Markdown

Related posts