You have Terraform fundamentals in place and a first project running. The next step is HashiCorp Configuration Language (HCL) in full: advanced HCL features and professional syntax techniques that make Terraform configurations maintainable, reusable and robust.
Building on Terraform fundamentals, the language features that separate simple Terraform scripts from professional infrastructure definitions come next: advanced data structures, complex validation, sensitive data handling and Terraform's built-in functions. Each section pairs theory with examples you can apply directly.
⚠️ Important notes and prerequisites: The material is aimed at experienced Linux administrators, junior DevOps engineers and IT professionals with solid Linux and networking knowledge. If you are comfortable on the command line and already understand cloud concepts such as virtual machines, networks and IAM, you are in the right place.
Terraform newcomers are welcome, but Linux beginners should complete basic tutorials first so you can focus on the essentials.
📋 Additional prerequisites:
- Basic understanding of Terraform concepts (providers, resources, state)
- Experience with the Terraform workflow (
init,plan,apply) - Practical knowledge of basic HCL syntax
- First experience with variables and outputs
🎯 Goals: After working through the material you command the advanced HCL features that professional Terraform configurations need. You understand complex data structures, can implement robust validation, and know how to manage sensitive data safely. Built-in functions become tools for elegant, reusable infrastructure definitions.
The outcome is HCL you can use to write maintainable, secure and efficient Terraform configurations — and a base for more complex topics such as state management, AWS infrastructures and team workflows in later parts of the series.
From advanced syntax features through built-in functions, the path is toward Terraform code that is not only functional but elegant and professional.
Advanced HCL syntax
The fundamentals already covered first HCL syntax in practice. HCL is more than syntax — it is the language in which you express the entire infrastructure. Advanced HCL features let you implement complex logic, build dynamic configurations and write maintainable infrastructure definitions.
Complex data structures (objects, sets, maps)
What makes HCL a powerful configuration language? HCL combines the simplicity of a configuration language with the flexibility of a programming language. Unlike plain JSON or YAML, HCL offers functions, variables and conditional logic. That lets you reuse configurations written once across different scenarios.
Why is understanding the advanced syntax decisive? Without advanced HCL features you write repetitive code that is hard to maintain. With the advanced functions you create dynamic, reusable configurations that can adapt to different environments. That lowers the error rate and makes the infrastructure more scalable.
What should you watch for with advanced HCL features? The power of the advanced syntax can also produce complex, hard-to-read configuration. Balance between functionality and readability is decisive. Every piece of complexity must have a clear benefit.
What do you use advanced HCL syntax for? Mainly for multi-environment configurations, dynamic resource creation, complex validation and automating infrastructure patterns that repeat.
What are complex data structures in HCL? Besides the simple data types (string, number, bool), HCL offers complex structures such as objects, sets and maps. They let you organise hierarchical data and keep complex configurations readable.
Why do complex data structures matter? They solve variable explosion. Instead of defining dozens of individual variables, you organise related data in logical structures. That makes the configuration more maintainable and reduces sources of error.
What must you watch for with complex data structures? Type consistency is decisive. HCL is type-safe, which means you cannot simply mix different data types. Nested structures can also raise debugging complexity.
What do you use complex data structures for? For configurations that belong together logically: database configurations, network settings, application configs and multi-environment definitions.
Object data type in detail:
Objects are structured data types with named attributes. Each attribute can have a different data type. They are ideal for configurations that have several related properties.
# Advanced object definition for database configuration
variable "database_config" {
description = "Complete database configuration for different environments"
type = object({
# Base configuration
engine = string
engine_version = string
instance_class = string
# Storage configuration
storage_type = string
allocated_storage = number
max_storage = number
storage_encrypted = bool
# Backup configuration
backup_enabled = bool
backup_retention_days = number
backup_window = string
maintenance_window = string
# Monitoring configuration
monitoring_enabled = bool
monitoring_interval = number
performance_insights = bool
# Network configuration
subnet_group_name = string
security_group_ids = list(string)
publicly_accessible = bool
# Tags and metadata
tags = map(string)
# Advanced options
parameters = map(string)
})
validation {
condition = contains(["mysql", "postgres", "mariadb"], var.database_config.engine)
error_message = "Database engine must be mysql, postgres, or mariadb."
}
validation {
condition = var.database_config.allocated_storage >= 20 && var.database_config.allocated_storage <= 1000
error_message = "Allocated storage must be between 20 and 1000 GB."
}
validation {
condition = var.database_config.backup_retention_days >= 0 && var.database_config.backup_retention_days <= 35
error_message = "Backup retention must be between 0 and 35 days."
}
}
# Environment-specific database configurations
locals {
database_configs = {
dev = {
engine = "mysql"
engine_version = "8.0"
instance_class = "db.t3.micro"
storage_type = "gp2"
allocated_storage = 20
max_storage = 50
storage_encrypted = false
backup_enabled = false
backup_retention_days = 0
backup_window = "03:00-04:00"
maintenance_window = "sun:04:00-sun:05:00"
monitoring_enabled = false
monitoring_interval = 0
performance_insights = false
subnet_group_name = "dev-db-subnet-group"
security_group_ids = ["sg-dev-db"]
publicly_accessible = false
tags = {
Environment = "development"
BackupPolicy = "none"
CostCenter = "engineering"
}
parameters = {
"innodb_buffer_pool_size" = "128M"
"max_connections" = "100"
}
}
staging = {
engine = "mysql"
engine_version = "8.0"
instance_class = "db.t3.small"
storage_type = "gp3"
allocated_storage = 50
max_storage = 200
storage_encrypted = true
backup_enabled = true
backup_retention_days = 7
backup_window = "03:00-04:00"
maintenance_window = "sun:04:00-sun:05:00"
monitoring_enabled = true
monitoring_interval = 60
performance_insights = true
subnet_group_name = "staging-db-subnet-group"
security_group_ids = ["sg-staging-db"]
publicly_accessible = false
tags = {
Environment = "staging"
BackupPolicy = "weekly"
CostCenter = "engineering"
}
parameters = {
"innodb_buffer_pool_size" = "256M"
"max_connections" = "200"
}
}
prod = {
engine = "mysql"
engine_version = "8.0"
instance_class = "db.r5.large"
storage_type = "gp3"
allocated_storage = 200
max_storage = 1000
storage_encrypted = true
backup_enabled = true
backup_retention_days = 30
backup_window = "03:00-04:00"
maintenance_window = "sun:04:00-sun:05:00"
monitoring_enabled = true
monitoring_interval = 15
performance_insights = true
subnet_group_name = "prod-db-subnet-group"
security_group_ids = ["sg-prod-db-primary", "sg-prod-db-backup"]
publicly_accessible = false
tags = {
Environment = "production"
BackupPolicy = "daily"
CostCenter = "operations"
Compliance = "required"
}
parameters = {
"innodb_buffer_pool_size" = "1024M"
"max_connections" = "500"
"slow_query_log" = "1"
"long_query_time" = "2"
}
}
}
# Current environment configuration
current_db_config = local.database_configs[var.environment]
}
# RDS instance with object configuration
resource "aws_db_instance" "main" {
identifier = "${var.project_name}-${var.environment}-db"
# Engine configuration
engine = local.current_db_config.engine
engine_version = local.current_db_config.engine_version
instance_class = local.current_db_config.instance_class
# Storage configuration
storage_type = local.current_db_config.storage_type
allocated_storage = local.current_db_config.allocated_storage
max_allocated_storage = local.current_db_config.max_storage
storage_encrypted = local.current_db_config.storage_encrypted
# Backup configuration
backup_retention_period = local.current_db_config.backup_retention_days
backup_window = local.current_db_config.backup_window
maintenance_window = local.current_db_config.maintenance_window
# Monitoring configuration
monitoring_interval = local.current_db_config.monitoring_interval
monitoring_role_arn = local.current_db_config.monitoring_enabled ? aws_iam_role.rds_monitoring[0].arn : null
performance_insights_enabled = local.current_db_config.performance_insights
performance_insights_retention_period = local.current_db_config.performance_insights ? 7 : null
# Network configuration
db_subnet_group_name = local.current_db_config.subnet_group_name
vpc_security_group_ids = local.current_db_config.security_group_ids
publicly_accessible = local.current_db_config.publicly_accessible
# Database configuration
db_name = "${var.project_name}_${var.environment}"
username = var.db_username
password = var.db_password
# Tags
tags = merge(
local.current_db_config.tags,
{
Name = "${var.project_name}-${var.environment}-db"
}
)
# Lifecycle management
lifecycle {
prevent_destroy = true
}
}
# Parameter group for database optimisation
resource "aws_db_parameter_group" "main" {
family = "${local.current_db_config.engine}${substr(local.current_db_config.engine_version, 0, 3)}"
name = "${var.project_name}-${var.environment}-params"
dynamic "parameter" {
for_each = local.current_db_config.parameters
content {
name = parameter.key
value = parameter.value
}
}
tags = local.current_db_config.tags
}
💡 Object optimisation: Use objects for configurations that belong together logically. That makes your Terraform configuration not only clearer but also more type-safe.
Set data type extended:
Sets are collections of unique values with no particular order. They are ideal for configurations where order is irrelevant but uniqueness matters.
# Advanced set configuration for security groups
variable "security_rules" {
description = "Security group rules configuration"
type = set(object({
type = string
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
description = string
}))
default = [
{
type = "ingress"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["10.0.0.0/16"]
description = "SSH access from VPC"
},
{
type = "ingress"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTP access from internet"
},
{
type = "ingress"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTPS access from internet"
},
{
type = "egress"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "All outbound traffic"
}
]
}
# Environment-specific security rules
locals {
# Base rules for all environments
base_security_rules = [
{
type = "egress"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "All outbound traffic"
}
]
# Development environment - less restrictive
dev_security_rules = [
{
type = "ingress"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "SSH access for development"
},
{
type = "ingress"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTP access"
},
{
type = "ingress"
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Development server"
}
]
# Production environment - restrictive
prod_security_rules = [
{
type = "ingress"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["10.0.0.0/16"]
description = "SSH access from VPC only"
},
{
type = "ingress"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTP access"
},
{
type = "ingress"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTPS access"
}
]
# Merge environment-specific rules
environment_rules = var.environment == "prod" ? local.prod_security_rules : local.dev_security_rules
# Final rule set
final_security_rules = toset(concat(local.base_security_rules, local.environment_rules))
}
# Security group with dynamic rules
resource "aws_security_group" "app" {
name_prefix = "${var.project_name}-${var.environment}-app-"
description = "Security group for ${var.project_name} application"
vpc_id = var.vpc_id
# Dynamic rule creation
dynamic "ingress" {
for_each = { for rule in local.final_security_rules : "${rule.type}-${rule.from_port}-${rule.to_port}" => rule if rule.type == "ingress" }
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ingress.value.cidr_blocks
description = ingress.value.description
}
}
dynamic "egress" {
for_each = { for rule in local.final_security_rules : "${rule.type}-${rule.from_port}-${rule.to_port}" => rule if rule.type == "egress" }
content {
from_port = egress.value.from_port
to_port = egress.value.to_port
protocol = egress.value.protocol
cidr_blocks = egress.value.cidr_blocks
description = egress.value.description
}
}
tags = {
Name = "${var.project_name}-${var.environment}-app-sg"
Environment = var.environment
}
lifecycle {
create_before_destroy = true
}
}
⚠️ Set pitfall: Sets have no order. If you need the order of the elements, use a list instead.
Map data type for complex configurations:
Maps are key-value pairs that can contain more complex structures as values. They are ideal for configurations that must be addressed by keys.
# Advanced map configuration for multi-region deployment
variable "region_configurations" {
description = "Configuration for different AWS regions"
type = map(object({
# Region-specific settings
availability_zones = list(string)
cidr_block = string
# Compute configuration
instance_types = object({
web = string
app = string
db = string
})
# Auto Scaling configuration
scaling_config = object({
min_size = number
max_size = number
desired_capacity = number
scale_up_threshold = number
scale_down_threshold = number
})
# Backup configuration
backup_config = object({
enabled = bool
retention_days = number
cross_region_copy = bool
destination_region = string
})
# Monitoring configuration
monitoring_config = object({
enabled = bool
detailed_monitoring = bool
log_retention_days = number
alert_endpoints = list(string)
})
# Compliance settings
compliance_config = object({
encryption_required = bool
audit_logging = bool
vpc_flow_logs = bool
cloudtrail_enabled = bool
})
# Cost optimisation
cost_config = object({
spot_instances_enabled = bool
reserved_instances = bool
auto_shutdown = bool
shutdown_schedule = string
})
}))
default = {
us-east-1 = {
availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
cidr_block = "10.0.0.0/16"
instance_types = {
web = "t3.small"
app = "t3.medium"
db = "db.t3.small"
}
scaling_config = {
min_size = 2
max_size = 10
desired_capacity = 3
scale_up_threshold = 70
scale_down_threshold = 30
}
backup_config = {
enabled = true
retention_days = 30
cross_region_copy = true
destination_region = "us-west-2"
}
monitoring_config = {
enabled = true
detailed_monitoring = true
log_retention_days = 90
alert_endpoints = ["alerts-east@company.com"]
}
compliance_config = {
encryption_required = true
audit_logging = true
vpc_flow_logs = true
cloudtrail_enabled = true
}
cost_config = {
spot_instances_enabled = false
reserved_instances = true
auto_shutdown = false
shutdown_schedule = "never"
}
}
us-west-2 = {
availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
cidr_block = "10.1.0.0/16"
instance_types = {
web = "t3.micro"
app = "t3.small"
db = "db.t3.micro"
}
scaling_config = {
min_size = 1
max_size = 5
desired_capacity = 2
scale_up_threshold = 80
scale_down_threshold = 20
}
backup_config = {
enabled = true
retention_days = 7
cross_region_copy = false
destination_region = ""
}
monitoring_config = {
enabled = true
detailed_monitoring = false
log_retention_days = 30
alert_endpoints = ["alerts-west@company.com"]
}
compliance_config = {
encryption_required = false
audit_logging = false
vpc_flow_logs = false
cloudtrail_enabled = false
}
cost_config = {
spot_instances_enabled = true
reserved_instances = false
auto_shutdown = true
shutdown_schedule = "0 22 * * MON-FRI"
}
}
}
}
# Apply regional configuration
locals {
current_region_config = var.region_configurations[var.aws_region]
}
# VPC with regional configuration
resource "aws_vpc" "main" {
cidr_block = local.current_region_config.cidr_block
enable_dns_hostnames = true
enable_dns_support = true
tags = {
Name = "${var.project_name}-${var.environment}-vpc"
Region = var.aws_region
}
}
# Subnets based on regional configuration
resource "aws_subnet" "public" {
count = length(local.current_region_config.availability_zones)
vpc_id = aws_vpc.main.id
cidr_block = cidrsubnet(local.current_region_config.cidr_block, 8, count.index)
availability_zone = local.current_region_config.availability_zones[count.index]
map_public_ip_on_launch = true
tags = {
Name = "${var.project_name}-${var.environment}-public-${count.index + 1}"
Type = "public"
}
}
# Launch template with regional configuration
resource "aws_launch_template" "app" {
name_prefix = "${var.project_name}-${var.environment}-app-"
image_id = data.aws_ami.ubuntu.id
instance_type = local.current_region_config.instance_types.app
# Spot instances when configured
instance_market_options {
market_type = local.current_region_config.cost_config.spot_instances_enabled ? "spot" : null
dynamic "spot_options" {
for_each = local.current_region_config.cost_config.spot_instances_enabled ? [1] : []
content {
spot_instance_type = "one-time"
}
}
}
# Monitoring configuration
monitoring {
enabled = local.current_region_config.monitoring_config.detailed_monitoring
}
# Advanced EBS configuration
block_device_mappings {
device_name = "/dev/sda1"
ebs {
volume_size = 20
volume_type = "gp3"
encrypted = local.current_region_config.compliance_config.encryption_required
iops = 3000
}
}
user_data = base64encode(templatefile("${path.module}/userdata.sh", {
region = var.aws_region
environment = var.environment
monitoring_enabled = local.current_region_config.monitoring_config.enabled
log_retention = local.current_region_config.monitoring_config.log_retention_days
}))
tag_specifications {
resource_type = "instance"
tags = {
Name = "${var.project_name}-${var.environment}-app"
Region = var.aws_region
CostOptimized = local.current_region_config.cost_config.spot_instances_enabled
}
}
}
# Auto Scaling group with regional configuration
resource "aws_autoscaling_group" "app" {
name = "${var.project_name}-${var.environment}-app-asg"
vpc_zone_identifier = aws_subnet.public[*].id
min_size = local.current_region_config.scaling_config.min_size
max_size = local.current_region_config.scaling_config.max_size
desired_capacity = local.current_region_config.scaling_config.desired_capacity
launch_template {
id = aws_launch_template.app.id
version = "$Latest"
}
# Auto shutdown for cost optimisation
dynamic "tag" {
for_each = local.current_region_config.cost_config.auto_shutdown ? [1] : []
content {
key = "AutoShutdown"
value = "true"
propagate_at_launch = true
}
}
tag {
key = "Name"
value = "${var.project_name}-${var.environment}-app"
propagate_at_launch = true
}
}
Data structure comparison and use cases:
| Data type | Characteristics | Performance | Use case | Debugging difficulty |
|---|---|---|---|---|
| Object | Structure with named attributes | High | Configuration groups | Low |
| Set | Unique values, no order | Medium | Security groups, tags | Medium |
| Map | Key-value pairs | High | Environment configs | Medium |
| List | Ordered collection | Low | Sequential data | Low |
🔧 Practical example - choosing a data structure:
# Choose the right data structure
locals {
# ✅ Object for related configuration
database_config = {
engine = "mysql"
version = "8.0"
storage = 100
}
# ✅ Set for unique values without order
allowed_cidrs = toset([
"10.0.0.0/16",
"172.16.0.0/12",
"192.168.0.0/16"
])
# ✅ Map for key-value mappings
environment_settings = {
dev = { instance_type = "t3.micro", count = 1 }
prod = { instance_type = "t3.large", count = 3 }
}
# ✅ List for ordered data
deployment_stages = [
"build",
"test",
"deploy",
"verify"
]
}
💡 Performance tip: Objects and maps have O(1) access via keys, while lists need O(n) for search.
Escape sequences and string interpolation
What are escape sequences and string interpolation? Escape sequences let you use special characters in strings. String interpolation makes configurations dynamic by embedding variables, functions and expressions directly in strings.
Why do they matter so much in Terraform? Terraform works heavily with strings — from resource names through template files to user-data scripts. Without correct string handling you get syntax errors and security holes.
What must you watch for with string operations? String interpolation can lead to injection vulnerabilities if it is not escaped properly. Complex string operations can also hurt Terraform performance.
What do you use advanced string features for? Dynamic configuration, template generation, JSON/YAML creation and complex naming.
# Advanced string operations
locals {
# Base variables
project_name = "ecommerce-platform"
environment = "production"
region = "us-east-1"
# Advanced escape sequences
json_config = jsonencode({
# Strings with quotation marks
message = "Welcome to \"${local.project_name}\" platform"
# Linux paths for logs and configuration
log_path = "/var/log/${local.project_name}/application.log"
config_path = "/etc/${local.project_name}/config"
data_path = "/opt/${local.project_name}/data"
# Unix socket paths
socket_path = "/var/run/${local.project_name}/${local.project_name}.sock"
# Unicode characters
company_name = "Acme Corp™"
# Newlines and formatting
welcome_text = <<-EOT
Welcome to ${local.project_name}!
Environment: ${local.environment}
Region: ${local.region}
For support, contact: support@company.com
EOT
})
# Complex string interpolation
resource_naming = {
# Base naming
prefix = "${local.project_name}-${local.environment}"
# Conditional naming
db_name = "${local.project_name}-${local.environment == "prod" ? "production" : "development"}-db"
# Timestamp-based names
backup_name = "${local.project_name}-backup-${formatdate("YYYY-MM-DD-hhmm", timestamp())}"
# Hash-based unique names
unique_bucket = "${local.project_name}-${local.environment}-${substr(md5("${local.project_name}-${local.environment}-${timestamp()}"), 0, 8)}"
}
# Template variables for different formats
template_vars = {
# Base information
project_name = local.project_name
environment = local.environment
region = local.region
# Conditional values
debug_mode = local.environment != "prod"
ssl_enabled = local.environment == "prod"
# Computed values
instance_count = local.environment == "prod" ? 3 : 1
storage_size = local.environment == "prod" ? 100 : 20
# Formatted strings
formatted_date = formatdate("YYYY-MM-DD", timestamp())
formatted_time = formatdate("hh:mm:ss", timestamp())
}
}
# Advanced heredoc syntax for complex templates
resource "aws_instance" "app" {
count = local.template_vars.instance_count
ami = data.aws_ami.ubuntu.id
instance_type = local.environment == "prod" ? "t3.large" : "t3.micro"
# Complex user data with advanced string features
user_data = base64encode(templatefile("${path.module}/templates/userdata.sh", merge(local.template_vars, {
instance_index = count.index
instance_id = count.index + 1
total_instances = local.template_vars.instance_count
})))
tags = {
Name = "${local.resource_naming.prefix}-app-${count.index + 1}"
Role = "application"
}
}
# Advanced JSON configuration for CloudWatch
resource "aws_cloudwatch_log_group" "app" {
name = "/aws/ec2/${local.resource_naming.prefix}/application"
retention_in_days = local.environment == "prod" ? 90 : 7
tags = {
Name = "${local.resource_naming.prefix}-logs"
}
}
# S3 bucket policy with complex string interpolation
resource "aws_s3_bucket_policy" "app_data" {
bucket = aws_s3_bucket.app_data.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "DenyInsecureConnections"
Effect = "Deny"
Principal = "*"
Action = "s3:*"
Resource = [
"${aws_s3_bucket.app_data.arn}",
"${aws_s3_bucket.app_data.arn}/*"
]
Condition = {
Bool = {
"aws:SecureTransport" = "false"
}
}
},
{
Sid = "AllowApplicationAccess"
Effect = "Allow"
Principal = {
AWS = aws_iam_role.app_role.arn
}
Action = [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
]
Resource = "${aws_s3_bucket.app_data.arn}/*"
}
]
})
}
userdata.sh template file:
#!/bin/bash
# Generated by Terraform on ${formatted_date} at ${formatted_time}
# Project: ${project_name}
# Environment: ${environment}
# Instance: ${instance_id}/${total_instances}
# Set environment variables
export PROJECT_NAME="${project_name}"
export ENVIRONMENT="${environment}"
export AWS_REGION="${region}"
export INSTANCE_ID="${instance_id}"
export DEBUG_MODE="${debug_mode}"
export SSL_ENABLED="${ssl_enabled}"
# Log everything
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
echo "Starting initialization for ${project_name} instance ${instance_id}/${total_instances}"
# Update system
apt-get update -y
apt-get upgrade -y
# Install required packages
apt-get install -y \
curl \
wget \
unzip \
jq \
awscli \
docker.io \
docker-compose
# Configure Docker
systemctl enable docker
systemctl start docker
usermod -a -G docker ubuntu
# Create application directory
mkdir -p /opt/${project_name}
cd /opt/${project_name}
# Create application configuration
cat > config.json << 'EOF'
{
"project_name": "${project_name}",
"environment": "${environment}",
"region": "${region}",
"instance_id": "${instance_id}",
"debug_mode": ${debug_mode},
"ssl_enabled": ${ssl_enabled},
"storage_size": ${storage_size},
"database_config": {
"host": "localhost",
"port": 3306,
"name": "${project_name}_${environment}"
},
"monitoring": {
"enabled": true,
"log_level": "${debug_mode ? "DEBUG" : "INFO"}",
"metrics_interval": 60
}
}
EOF
# Environment-specific configuration
%{ if environment == "prod" }
# Production-specific setup
echo "Setting up production environment..."
# SSL certificates
mkdir -p /etc/ssl/certs/${project_name}
# SSL setup would go here
# Performance tuning
echo "net.core.rmem_max = 134217728" >> /etc/sysctl.conf
echo "net.core.wmem_max = 134217728" >> /etc/sysctl.conf
sysctl -p
%{ else }
# Development/staging setup
echo "Setting up development environment..."
# Development tools
apt-get install -y \
vim \
htop \
tree \
git
# Relaxed security for development
echo "Development mode - relaxed security settings"
%{ endif }
# Install CloudWatch agent
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
dpkg -i amazon-cloudwatch-agent.deb
# Configure CloudWatch agent
cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json << 'EOF'
{
"agent": {
"metrics_collection_interval": 60,
"run_as_user": "cwagent"
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/user-data.log",
"log_group_name": "/aws/ec2/${project_name}-${environment}/user-data",
"log_stream_name": "{instance_id}/user-data"
},
{
"file_path": "/opt/${project_name}/logs/application.log",
"log_group_name": "/aws/ec2/${project_name}-${environment}/application",
"log_stream_name": "{instance_id}/application"
}
]
}
}
},
"metrics": {
"namespace": "${project_name}/${environment}",
"metrics_collected": {
"cpu": {
"measurement": [
"cpu_usage_idle",
"cpu_usage_iowait",
"cpu_usage_user",
"cpu_usage_system"
],
"metrics_collection_interval": 60
},
"disk": {
"measurement": [
"used_percent"
],
"metrics_collection_interval": 60,
"resources": [
"*"
]
},
"mem": {
"measurement": [
"mem_used_percent"
],
"metrics_collection_interval": 60
}
}
}
}
EOF
# Start CloudWatch agent
systemctl enable amazon-cloudwatch-agent
systemctl start amazon-cloudwatch-agent
# Final status
echo "Initialization completed for ${project_name} instance ${instance_id}/${total_instances}"
echo "Environment: ${environment}"
echo "Debug mode: ${debug_mode}"
echo "SSL enabled: ${ssl_enabled}"
echo "Timestamp: $(date)"
Common string interpolation errors:
| Problem | Symptom | Solution |
|---|---|---|
| Unescaped quotation marks | Error: Invalid JSON |
Use \" |
| Wrong template syntax | Error: Invalid template |
${} for variables |
| Circular reference | Error: Cycle detected |
Check dependencies |
| Type mismatch | Error: Invalid value |
Explicit conversion |
Comments and code formatting
Why are comments decisive in infrastructure code? Terraform configurations are often managed by teams and maintained for years. Good comments explain not only what is being done but also why particular decisions were made. That matters especially for complex network configurations or security settings.
What should you watch for with comments? Comments can go stale and then mislead. Keep them current and focus on the "why" rather than the "what". The "what" is visible from the code.
What do you use different comment styles for? Single-line comments for short explanations, multi-line for complex architecture decisions, and documentation blocks for modules and important resources.
# ============================================================================
# MULTI-TIER ARCHITECTURE CONFIGURATION
# ============================================================================
# This configuration implements a three-tier architecture pattern:
# 1. Presentation Tier (Public Subnets) - Load Balancers, Bastion Hosts
# 2. Application Tier (Private Subnets) - Application Servers, APIs
# 3. Data Tier (Database Subnets) - RDS, ElastiCache, Data Storage
#
# Architecture Decision: We use separate subnets for each tier to implement
# defense-in-depth security and enable granular network access controls.
#
# Last Updated: 2024-01-15
# Author: DevOps Team
# Review Date: 2024-04-15
# ============================================================================
# VPC Configuration
# Why /16 CIDR: Provides 65,536 IP addresses, sufficient for enterprise growth
# Why enable_dns_hostnames: Required for RDS endpoint resolution
resource "aws_vpc" "main" {
cidr_block = var.vpc_cidr # Default: 10.0.0.0/16
enable_dns_hostnames = true
enable_dns_support = true
tags = {
Name = "${var.project_name}-vpc"
# Important: This tag is used by AWS Load Balancer Controller
"kubernetes.io/cluster/${var.project_name}" = "shared"
}
}
# ============================================================================
# PRESENTATION TIER (PUBLIC SUBNETS)
# ============================================================================
# Public subnets host internet-facing resources like:
# - Application Load Balancers
# - NAT Gateways
# - Bastion Hosts (if needed)
#
# Security Consideration: Only resources that need direct internet access
# should be placed here. Application servers go in private subnets.
# ============================================================================
# Public Subnets - One per Availability Zone
# Why count.index: Ensures subnets are distributed across AZs for high availability
# Why cidrsubnet: Automatically calculates subnet CIDR blocks
resource "aws_subnet" "public" {
count = length(var.availability_zones)
vpc_id = aws_vpc.main.id
cidr_block = cidrsubnet(var.vpc_cidr, 8, count.index)
availability_zone = var.availability_zones[count.index]
map_public_ip_on_launch = true # Required for NAT Gateway EIPs
tags = {
Name = "${var.project_name}-public-${count.index + 1}"
Type = "public"
Tier = "presentation"
# Kubernetes requires this tag for subnet discovery
"kubernetes.io/role/elb" = "1"
}
}
# Internet Gateway
# Why dependency: VPC must exist before IGW attachment
resource "aws_internet_gateway" "main" {
vpc_id = aws_vpc.main.id
tags = {
Name = "${var.project_name}-igw"
}
}
# Public Route Table
# Why single route table: All public subnets need identical routing
resource "aws_route_table" "public" {
vpc_id = aws_vpc.main.id
# Route all traffic to Internet Gateway
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.main.id
}
tags = {
Name = "${var.project_name}-public-rt"
}
}
# Associate public subnets with route table
resource "aws_route_table_association" "public" {
count = length(aws_subnet.public)
subnet_id = aws_subnet.public[count.index].id
route_table_id = aws_route_table.public.id
}
# ============================================================================
# APPLICATION TIER (PRIVATE SUBNETS)
# ============================================================================
# Private subnets host application servers and internal services:
# - Web servers (behind ALB)
# - API servers
# - Background workers
# - Internal services
#
# Security Consideration: These resources cannot be directly accessed from
# the internet. They receive traffic through the load balancer and can
# initiate outbound connections through NAT Gateway.
# ============================================================================
# Private Subnets - Application Tier
# Why separate from public: Defense-in-depth security architecture
# Why +10 offset: Avoids IP conflicts with public subnets
resource "aws_subnet" "private_app" {
count = length(var.availability_zones)
vpc_id = aws_vpc.main.id
cidr_block = cidrsubnet(var.vpc_cidr, 8, count.index + 10)
availability_zone = var.availability_zones[count.index]
tags = {
Name = "${var.project_name}-private-app-${count.index + 1}"
Type = "private"
Tier = "application"
# Kubernetes requires this tag for internal load balancers
"kubernetes.io/role/internal-elb" = "1"
}
}
# ============================================================================
# DATA TIER (DATABASE SUBNETS)
# ============================================================================
# Database subnets are the most restrictive tier:
# - RDS instances
# - ElastiCache clusters
# - Data storage services
#
# Security Consideration: These subnets have no internet access whatsoever.
# They only communicate with application tier through specific ports.
# ============================================================================
# Database Subnets - Most restrictive tier
# Why +20 offset: Clear separation from other tiers
# Why no internet access: Databases should never communicate with internet
resource "aws_subnet" "private_db" {
count = length(var.availability_zones)
vpc_id = aws_vpc.main.id
cidr_block = cidrsubnet(var.vpc_cidr, 8, count.index + 20)
availability_zone = var.availability_zones[count.index]
tags = {
Name = "${var.project_name}-private-db-${count.index + 1}"
Type = "private"
Tier = "data"
}
}
# ============================================================================
# NAT GATEWAY CONFIGURATION
# ============================================================================
# NAT Gateways enable outbound internet access for private subnets
#
# Architecture Decision: One NAT Gateway per AZ for high availability
# Cost Consideration: NAT Gateways are expensive. For cost optimization,
# consider using a single NAT Gateway (less resilient but cheaper).
# ============================================================================
# Elastic IPs for NAT Gateways
# Why depends_on: EIPs require IGW to be attached first
resource "aws_eip" "nat" {
count = var.multi_az_nat ? length(var.availability_zones) : 1
domain = "vpc"
depends_on = [aws_internet_gateway.main]
tags = {
Name = "${var.project_name}-nat-eip-${count.index + 1}"
}
}
# NAT Gateways
# Why public subnet: NAT Gateway needs internet access to route traffic
# Why conditional count: Supports both single and multi-AZ configurations
resource "aws_nat_gateway" "main" {
count = var.multi_az_nat ? length(var.availability_zones) : 1
allocation_id = aws_eip.nat[count.index].id
subnet_id = aws_subnet.public[count.index].id
depends_on = [aws_internet_gateway.main]
tags = {
Name = "${var.project_name}-nat-${count.index + 1}"
}
}
# ============================================================================
# PRIVATE SUBNET ROUTING
# ============================================================================
# Private subnets need different routing tables because:
# 1. They route internet traffic through NAT Gateway
# 2. Each AZ may have its own NAT Gateway for high availability
# 3. Database subnets may have different routing rules
# ============================================================================
# Route Tables for Private Application Subnets
# Why per-AZ: Each AZ routes through its own NAT Gateway
resource "aws_route_table" "private_app" {
count = length(var.availability_zones)
vpc_id = aws_vpc.main.id
# Route internet traffic through NAT Gateway
# Why conditional: Supports both single and multi-AZ NAT configurations
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.main[var.multi_az_nat ? count.index : 0].id
}
tags = {
Name = "${var.project_name}-private-app-rt-${count.index + 1}"
}
}
# Associate private app subnets with route tables
resource "aws_route_table_association" "private_app" {
count = length(aws_subnet.private_app)
subnet_id = aws_subnet.private_app[count.index].id
route_table_id = aws_route_table.private_app[count.index].id
}
# Route Tables for Database Subnets
# Why separate: Database subnets may need different routing rules
# Why no internet route: Databases should never access internet directly
resource "aws_route_table" "private_db" {
count = length(var.availability_zones)
vpc_id = aws_vpc.main.id
# No internet route - databases stay internal
# Future: Add routes for VPC endpoints, peering connections
tags = {
Name = "${var.project_name}-private-db-rt-${count.index + 1}"
}
}
# Associate database subnets with route tables
resource "aws_route_table_association" "private_db" {
count = length(aws_subnet.private_db)
subnet_id = aws_subnet.private_db[count.index].id
route_table_id = aws_route_table.private_db[count.index].id
}
# ============================================================================
# SECURITY GROUPS
# ============================================================================
# Security groups implement stateful firewall rules at the instance level
# Following principle of least privilege - only allow required traffic
# ============================================================================
# Application Load Balancer Security Group
# Why port 80/443: Standard web traffic
# Why 0.0.0.0/0: Internet-facing load balancer needs public access
resource "aws_security_group" "alb" {
name_prefix = "${var.project_name}-alb-"
description = "Security group for Application Load Balancer"
vpc_id = aws_vpc.main.id
# HTTP ingress from internet
ingress {
description = "HTTP from internet"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# HTTPS ingress from internet
ingress {
description = "HTTPS from internet"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# All outbound traffic (for health checks)
egress {
description = "All outbound traffic"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "${var.project_name}-alb-sg"
}
# Prevent accidental deletion
lifecycle {
create_before_destroy = true
}
}
# Application Servers Security Group
# Why reference ALB SG: Only allow traffic from load balancer
# Why port 8080: Common application port (adjust as needed)
resource "aws_security_group" "app_servers" {
name_prefix = "${var.project_name}-app-"
description = "Security group for application servers"
vpc_id = aws_vpc.main.id
# HTTP from ALB only
ingress {
description = "HTTP from ALB"
from_port = 8080
to_port = 8080
protocol = "tcp"
security_groups = [aws_security_group.alb.id]
}
# SSH from bastion host (if exists)
# TODO: Implement bastion host security group
dynamic "ingress" {
for_each = var.enable_bastion ? [1] : []
content {
description = "SSH from bastion"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [var.vpc_cidr] # Only from VPC
}
}
# All outbound traffic (for package updates, API calls)
egress {
description = "All outbound traffic"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "${var.project_name}-app-sg"
}
lifecycle {
create_before_destroy = true
}
}
# Database Security Group
# Why port 3306: MySQL/MariaDB default port
# Why app servers only: Databases should only accept app connections
resource "aws_security_group" "database" {
name_prefix = "${var.project_name}-db-"
description = "Security group for database servers"
vpc_id = aws_vpc.main.id
# MySQL/MariaDB from app servers only
ingress {
description = "MySQL from app servers"
from_port = 3306
to_port = 3306
protocol = "tcp"
security_groups = [aws_security_group.app_servers.id]
}
# PostgreSQL support (if needed)
dynamic "ingress" {
for_each = var.database_engine == "postgres" ? [1] : []
content {
description = "PostgreSQL from app servers"
from_port = 5432
to_port = 5432
protocol = "tcp"
security_groups = [aws_security_group.app_servers.id]
}
}
# No outbound rules needed for databases
# RDS manages its own outbound communication
tags = {
Name = "${var.project_name}-db-sg"
}
lifecycle {
create_before_destroy = true
}
}
/*
* ARCHITECTURE SUMMARY
* ====================
* This configuration creates a secure, scalable three-tier architecture:
*
* 1. Public Subnets (Presentation Tier)
* - Hosts internet-facing load balancers
* - Has direct internet access via Internet Gateway
* - Routes: 0.0.0.0/0 -> Internet Gateway
*
* 2. Private App Subnets (Application Tier)
* - Hosts application servers and APIs
* - Internet access via NAT Gateway for updates
* - Routes: 0.0.0.0/0 -> NAT Gateway
*
* 3. Private DB Subnets (Data Tier)
* - Hosts databases and data storage
* - No internet access at all
* - Routes: Local traffic only
*
* Security Groups implement defense-in-depth:
* - ALB: Accepts internet traffic on 80/443
* - App Servers: Only accept traffic from ALB
* - Database: Only accepts traffic from app servers
*
* High Availability:
* - Resources deployed across multiple AZs
* - Optional multi-AZ NAT Gateways
* - Database subnets ready for Multi-AZ RDS
*/
🔧 Practical example - code formatting:
# Before terraform fmt (poorly formatted)
resource "aws_security_group" "web" {
name="web-sg"
vpc_id=aws_vpc.main.id
ingress{
from_port=80
to_port=80
protocol="tcp"
cidr_blocks=["0.0.0.0/0"]
}
}
# After terraform fmt (well formatted)
resource "aws_security_group" "web" {
name = "web-sg"
vpc_id = aws_vpc.main.id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
Comment best practices:
| Comment type | Use | Example |
|---|---|---|
| Single-line (#) | Short explanations | # SSH access from bastion |
| Multi-line (/\ \/) | Architecture documentation | Block comments |
| Header blocks | Section separation | # ====== SECTION ====== |
| Inline | Specific values | port = 8080 # Application port |
| TODO/FIXME | Future changes | # TODO: Implement bastion host |
Formatting tips:
- Run
terraform fmtregularly for consistent formatting - Use meaningful indentation for better readability
- Group related resources with comment blocks
- Document architecture decisions in detail
Comment pitfalls:
- Stale comments are worse than no comments
- Too many comments make the code unreadable
- Comments should explain the "why", not the "what"
- Never put secrets or passwords in comments
❗ Debugging techniques for complex syntax:
# Validate Terraform configuration
terraform validate
# Check formatting
terraform fmt -check -diff
# Syntax highlighting in vim
vim -c "syntax on" -c "set ft=terraform" main.tf
# Debug local values
terraform console
> local.current_config
> local.resource_naming.prefix
# Test template functions
terraform console
> templatefile("userdata.sh", {var = "test"})
Advanced HCL syntax lets you create complex, maintainable Terraform configurations. With objects, sets and maps you structure data logically. String interpolation and templates make configurations dynamic. Good comments and formatting keep the work maintainable in a team.
Advanced variables and outputs
What makes advanced variables and outputs? The fundamentals covered simple variable definitions. Next come advanced features such as complex validation, sensitive data handling and conditional outputs. Those functions turn Terraform configuration from static definitions into intelligent, self-validating infrastructure templates.
Why are advanced variable features decisive? Without robust validation and structured outputs you get runtime errors that can be catastrophic in production. Advanced features catch errors at plan time and make infrastructure more predictable and safer.
What must you watch for with advanced variables? Over-complex validation can hurt performance and make the configuration hard to understand. Sensitive variables need extra care with state files and logs.
What do you use advanced variable features for? Enterprise environments where compliance requires strict validation, multi-team projects with complex dependencies, and infrastructures that must manage sensitive data.
Input variables with complex validation
What are complex validations? Validations go beyond simple type checks. They verify business logic, compliance requirements and technical constraints. You can validate regex patterns, ranges, list contents and even external API responses.
Why do validations matter so much? They prevent expensive production mistakes. A wrong CIDR block configuration can cause network outages. Invalid instance types can cause performance problems. Validations catch these issues before apply.
What should you watch for with validations? Validations run on every plan and apply. Complex validations can hurt performance. Error messages should be understandable and action-oriented.
What do you use complex validations for? Compliance checks, security validations, cost control and technical constraints.
🔧 Practical example - advanced variable validation:
# Advanced network configuration with complex validations
variable "network_config" {
description = "Complete network configuration with validation"
type = object({
vpc_cidr = string
environment = string
availability_zones = list(string)
public_subnets = list(string)
private_subnets = list(string)
database_subnets = list(string)
enable_nat_gateway = bool
enable_vpn_gateway = bool
dns_domain = string
tags = map(string)
})
# CIDR block validation
validation {
condition = can(cidrhost(var.network_config.vpc_cidr, 0))
error_message = "VPC CIDR must be a valid IPv4 CIDR block (e.g., 10.0.0.0/16)."
}
# CIDR size validation
validation {
condition = tonumber(split("/", var.network_config.vpc_cidr)[1]) >= 16 && tonumber(split("/", var.network_config.vpc_cidr)[1]) <= 24
error_message = "VPC CIDR netmask must be between /16 and /24 for proper subnet allocation."
}
# Private IP range validation (RFC 1918)
validation {
condition = can(regex("^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)", var.network_config.vpc_cidr))
error_message = "VPC CIDR must use private IP ranges (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16)."
}
# Environment validation
validation {
condition = contains(["dev", "staging", "prod", "sandbox"], var.network_config.environment)
error_message = "Environment must be one of: dev, staging, prod, sandbox."
}
# Availability zone validation
validation {
condition = length(var.network_config.availability_zones) >= 2 && length(var.network_config.availability_zones) <= 6
error_message = "Must specify between 2 and 6 availability zones for high availability."
}
# Subnet count validation
validation {
condition = length(var.network_config.public_subnets) == length(var.network_config.availability_zones)
error_message = "Number of public subnets must match number of availability zones."
}
# Subnet CIDR validation
validation {
condition = alltrue([
for subnet in var.network_config.public_subnets :
can(cidrsubnet(var.network_config.vpc_cidr, 4, 0)) &&
can(cidrhost(subnet, 0))
])
error_message = "All public subnet CIDRs must be valid and within VPC CIDR range."
}
# DNS domain validation
validation {
condition = can(regex("^[a-z0-9.-]+\\.[a-z]{2,}$", var.network_config.dns_domain))
error_message = "DNS domain must be a valid domain name (e.g., example.com)."
}
# Tags validation
validation {
condition = alltrue([
for key, value in var.network_config.tags :
can(regex("^[A-Za-z0-9._-]{1,128}$", key)) &&
can(regex("^[A-Za-z0-9._\\s-]{1,256}$", value))
])
error_message = "Tag keys and values must follow AWS naming conventions."
}
}
# Advanced compute configuration with business-logic validation
variable "compute_config" {
description = "Compute configuration with business logic validation"
type = object({
instance_types = map(string)
min_instances = number
max_instances = number
desired_instances = number
auto_scaling_enabled = bool
spot_instances_enabled = bool
spot_max_price = string
health_check_type = string
health_check_grace_period = number
termination_policies = list(string)
})
# Instance type validation
validation {
condition = alltrue([
for type in values(var.compute_config.instance_types) :
can(regex("^[a-z][0-9]+[a-z]*\\.(nano|micro|small|medium|large|xlarge|[0-9]+xlarge)$", type))
])
error_message = "Instance types must be valid AWS instance types (e.g., t3.micro, m5.large)."
}
# Scaling limits validation
validation {
condition = var.compute_config.min_instances <= var.compute_config.desired_instances && var.compute_config.desired_instances <= var.compute_config.max_instances
error_message = "Scaling configuration must follow: min_instances <= desired_instances <= max_instances."
}
# Spot price validation
validation {
condition = var.compute_config.spot_instances_enabled ? can(tonumber(var.compute_config.spot_max_price)) && tonumber(var.compute_config.spot_max_price) > 0 : true
error_message = "Spot max price must be a positive number when spot instances are enabled."
}
# Health check validation
validation {
condition = contains(["EC2", "ELB"], var.compute_config.health_check_type)
error_message = "Health check type must be 'EC2' or 'ELB'."
}
# Termination policy validation
validation {
condition = alltrue([
for policy in var.compute_config.termination_policies :
contains(["OldestInstance", "NewestInstance", "OldestLaunchConfiguration", "ClosestToNextInstanceHour", "Default"], policy)
])
error_message = "Termination policies must be valid AWS Auto Scaling termination policies."
}
}
# Advanced database configuration with compliance validation
variable "database_config" {
description = "Database configuration with compliance validation"
type = object({
engine = string
engine_version = string
instance_class = string
allocated_storage = number
max_storage = number
storage_type = string
storage_encrypted = bool
kms_key_id = string
backup_retention_period = number
backup_window = string
maintenance_window = string
multi_az = bool
publicly_accessible = bool
deletion_protection = bool
performance_insights_enabled = bool
monitoring_interval = number
auto_minor_version_upgrade = bool
parameter_group_family = string
option_group_name = string
})
# Engine validation
validation {
condition = contains(["mysql", "postgres", "mariadb", "oracle-ee", "oracle-se2", "sqlserver-ee", "sqlserver-se", "sqlserver-ex", "sqlserver-web"], var.database_config.engine)
error_message = "Database engine must be a supported RDS engine."
}
# Storage validation
validation {
condition = var.database_config.allocated_storage >= 20 && var.database_config.allocated_storage <= 65536
error_message = "Allocated storage must be between 20 GB and 65,536 GB."
}
# Storage type validation
validation {
condition = contains(["gp2", "gp3", "io1", "io2", "magnetic"], var.database_config.storage_type)
error_message = "Storage type must be gp2, gp3, io1, io2, or magnetic."
}
# Backup retention validation
validation {
condition = var.database_config.backup_retention_period >= 0 && var.database_config.backup_retention_period <= 35
error_message = "Backup retention period must be between 0 and 35 days."
}
# Backup window format validation
validation {
condition = can(regex("^[0-2][0-9]:[0-5][0-9]-[0-2][0-9]:[0-5][0-9]$", var.database_config.backup_window))
error_message = "Backup window must be in format HH:MM-HH:MM (e.g., 03:00-04:00)."
}
# Maintenance window format validation
validation {
condition = can(regex("^(sun|mon|tue|wed|thu|fri|sat):[0-2][0-9]:[0-5][0-9]-(sun|mon|tue|wed|thu|fri|sat):[0-2][0-9]:[0-5][0-9]$", var.database_config.maintenance_window))
error_message = "Maintenance window must be in format ddd:HH:MM-ddd:HH:MM (e.g., sun:04:00-sun:05:00)."
}
# Compliance validation: encryption required for production
validation {
condition = var.database_config.storage_encrypted == true
error_message = "Storage encryption is required for all database instances for compliance."
}
# Compliance validation: deletion protection for production
validation {
condition = var.database_config.deletion_protection == true
error_message = "Deletion protection must be enabled for all database instances."
}
# Compliance validation: no public access
validation {
condition = var.database_config.publicly_accessible == false
error_message = "Database instances must not be publicly accessible for security compliance."
}
# Monitoring interval validation
validation {
condition = contains([0, 1, 5, 10, 15, 30, 60], var.database_config.monitoring_interval)
error_message = "Monitoring interval must be 0, 1, 5, 10, 15, 30, or 60 seconds."
}
}
# Security configuration with advanced validations
variable "security_config" {
description = "Security configuration with advanced validation"
type = object({
enable_flow_logs = bool
enable_cloudtrail = bool
enable_config = bool
enable_guardduty = bool
security_group_rules = list(object({
type = string
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
description = string
}))
kms_key_rotation_enabled = bool
password_policy = object({
minimum_password_length = number
require_lowercase_characters = bool
require_uppercase_characters = bool
require_numbers = bool
require_symbols = bool
max_password_age = number
})
})
# Security group rules validation
validation {
condition = alltrue([
for rule in var.security_config.security_group_rules :
contains(["ingress", "egress"], rule.type) &&
rule.from_port >= 0 && rule.from_port <= 65535 &&
rule.to_port >= 0 && rule.to_port <= 65535 &&
rule.from_port <= rule.to_port
])
error_message = "Security group rules must have valid types, ports, and port ranges."
}
# CIDR blocks validation for security groups
validation {
condition = alltrue([
for rule in var.security_config.security_group_rules :
alltrue([
for cidr in rule.cidr_blocks :
can(cidrhost(cidr, 0))
])
])
error_message = "All CIDR blocks in security group rules must be valid."
}
# Dangerous ports validation
validation {
condition = alltrue([
for rule in var.security_config.security_group_rules :
!(rule.type == "ingress" && contains(rule.cidr_blocks, "0.0.0.0/0") &&
(rule.from_port <= 22 && rule.to_port >= 22 ||
rule.from_port <= 3389 && rule.to_port >= 3389))
])
error_message = "SSH (22) and RDP (3389) ports must not be open to 0.0.0.0/0 for security."
}
# Password policy validation
validation {
condition = var.security_config.password_policy.minimum_password_length >= 8 && var.security_config.password_policy.minimum_password_length <= 128
error_message = "Password minimum length must be between 8 and 128 characters."
}
# Password complexity validation
validation {
condition = var.security_config.password_policy.require_lowercase_characters && var.security_config.password_policy.require_uppercase_characters && var.security_config.password_policy.require_numbers && var.security_config.password_policy.require_symbols
error_message = "Password policy must require all character types for security compliance."
}
}
Validation strategies:
| Validation type | Use | Performance | Complexity |
|---|---|---|---|
| Type validation | Base data types | Very high | Low |
| Regex validation | Format check | High | Medium |
| Range validation | Numeric ranges | Very high | Low |
| List validation | Allowed values | High | Low |
| Business logic | Complex rules | Medium | High |
| Cross-field | Dependencies | Medium | High |
💡 Validation optimisation: Use simple validations first, then complex ones. That improves performance and makes error messages easier to understand.
Advanced validation functions:
# Conditional validation based on other variables
locals {
is_production = var.environment == "prod"
is_development = var.environment == "dev"
}
variable "monitoring_config" {
description = "Monitoring configuration with conditional validation"
type = object({
enabled = bool
retention_days = number
detailed_monitoring = bool
alert_endpoints = list(string)
})
# Conditional validation: monitoring required for production
validation {
condition = var.environment == "prod" ? var.monitoring_config.enabled : true
error_message = "Monitoring must be enabled for production environments."
}
# Conditional validation: retention for production
validation {
condition = var.environment == "prod" ? var.monitoring_config.retention_days >= 90 : var.monitoring_config.retention_days >= 7
error_message = "Production environments require minimum 90 days retention, others require minimum 7 days."
}
# Email format validation
validation {
condition = alltrue([
for email in var.monitoring_config.alert_endpoints :
can(regex("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$", email))
])
error_message = "All alert endpoints must be valid email addresses."
}
}
# Cross-variable validation
variable "scaling_config" {
description = "Auto scaling configuration"
type = object({
min_size = number
max_size = number
desired_capacity = number
target_cpu_utilization = number
})
# Cross-field validation
validation {
condition = var.scaling_config.min_size <= var.scaling_config.desired_capacity && var.scaling_config.desired_capacity <= var.scaling_config.max_size
error_message = "Scaling configuration must follow: min_size <= desired_capacity <= max_size."
}
# Business-logic validation
validation {
condition = var.scaling_config.max_size <= (var.environment == "prod" ? 100 : 10)
error_message = "Production environments can scale to 100 instances, others are limited to 10."
}
}
Validation pitfalls:
| Problem | Symptom | Solution |
|---|---|---|
| Slow validation | Plan takes a long time | Simple checks first |
| Circular references | Validation error | Check dependencies |
| Unclear error messages | Confusion on errors | Specific messages |
| Over-validation | Hard to use | Only necessary checks |
Sensitive variables and credential handling
What are sensitive variables? Sensitive variables hold confidential information such as passwords, API keys or certificates. Terraform treats them specially: they are not shown in plan output, not written to logs, and obscured in the state file.
Why is correct credential handling critical? Badly handled credentials become security holes. Passwords in plan output, API keys in logs or unencrypted secrets in state files are common weaknesses.
What must you watch for with sensitive variables? Sensitive variables are only "sensitive" inside Terraform, not in the real world. State files must be encrypted, logs must be handled securely, and team access must be controlled.
What do you use sensitive variables for? Passwords, API keys, certificates, tokens and any other confidential information used in the infrastructure.
# Advanced sensitive variable configuration
variable "database_credentials" {
description = "Database credentials - marked as sensitive"
type = object({
master_username = string
master_password = string
replica_username = string
replica_password = string
})
sensitive = true
validation {
condition = length(var.database_credentials.master_password) >= 12
error_message = "Master password must be at least 12 characters long."
}
validation {
condition = can(regex("^[A-Za-z0-9!@#$%^&*()_+=-]+$", var.database_credentials.master_password))
error_message = "Password must contain only allowed characters."
}
}
# API keys and tokens
variable "external_api_keys" {
description = "External API keys and tokens"
type = object({
github_token = string
docker_registry_token = string
monitoring_api_key = string
backup_service_key = string
})
sensitive = true
validation {
condition = alltrue([
for key in values(var.external_api_keys) :
length(key) >= 8 && length(key) <= 256
])
error_message = "API keys must be between 8 and 256 characters long."
}
}
# SSL certificates
variable "ssl_certificates" {
description = "SSL certificates and private keys"
type = object({
certificate_body = string
private_key = string
certificate_chain = string
})
sensitive = true
validation {
condition = can(regex("^-----BEGIN CERTIFICATE-----", var.ssl_certificates.certificate_body))
error_message = "Certificate body must be a valid PEM-encoded certificate."
}
validation {
condition = can(regex("^-----BEGIN (RSA )?PRIVATE KEY-----", var.ssl_certificates.private_key))
error_message = "Private key must be a valid PEM-encoded private key."
}
}
# Encryption keys
variable "encryption_keys" {
description = "Encryption keys for various services"
type = object({
application_secret_key = string
session_encryption_key = string
database_encryption_key = string
})
sensitive = true
validation {
condition = alltrue([
for key in values(var.encryption_keys) :
length(key) >= 32
])
error_message = "Encryption keys must be at least 32 characters long."
}
}
Secure credential passing:
# Local sensitive values with calculations
locals {
# Secure password generation
generated_passwords = {
db_master = random_password.db_master.result
db_replica = random_password.db_replica.result
app_secret = random_password.app_secret.result
}
# Secure combinations
database_connection_strings = {
master = "postgresql://${var.database_credentials.master_username}:${local.generated_passwords.db_master}@${aws_db_instance.master.endpoint}:5432/${aws_db_instance.master.name}"
replica = "postgresql://${var.database_credentials.replica_username}:${local.generated_passwords.db_replica}@${aws_db_instance.replica.endpoint}:5432/${aws_db_instance.replica.name}"
}
}
# Secure password generation
resource "random_password" "db_master" {
length = 16
special = true
numeric = true
upper = true
lower = true
}
resource "random_password" "db_replica" {
length = 16
special = true
numeric = true
upper = true
lower = true
}
resource "random_password" "app_secret" {
length = 32
special = true
numeric = true
upper = true
lower = true
}
# AWS Secrets Manager integration
resource "aws_secretsmanager_secret" "database_credentials" {
name = "${var.project_name}-${var.environment}-db-credentials"
description = "Database credentials for ${var.project_name}"
recovery_window_in_days = 7
tags = {
Name = "${var.project_name}-db-credentials"
Environment = var.environment
}
}
resource "aws_secretsmanager_secret_version" "database_credentials" {
secret_id = aws_secretsmanager_secret.database_credentials.id
secret_string = jsonencode({
master_username = var.database_credentials.master_username
master_password = random_password.db_master.result
replica_username = var.database_credentials.replica_username
replica_password = random_password.db_replica.result
connection_strings = local.database_connection_strings
})
}
# KMS key for encryption
resource "aws_kms_key" "secrets" {
description = "KMS key for ${var.project_name} secrets"
deletion_window_in_days = 7
enable_key_rotation = true
tags = {
Name = "${var.project_name}-secrets-key"
Environment = var.environment
}
}
resource "aws_kms_alias" "secrets" {
name = "alias/${var.project_name}-${var.environment}-secrets"
target_key_id = aws_kms_key.secrets.key_id
}
# RDS instance with Secrets Manager
resource "aws_db_instance" "master" {
identifier = "${var.project_name}-${var.environment}-master"
engine = "postgres"
engine_version = "13.7"
instance_class = "db.t3.micro"
allocated_storage = 20
max_allocated_storage = 100
storage_type = "gp2"
storage_encrypted = true
kms_key_id = aws_kms_key.secrets.arn
# Credentials from Secrets Manager
manage_master_user_password = true
master_user_secret_kms_key_id = aws_kms_key.secrets.arn
db_name = replace("${var.project_name}_${var.environment}", "-", "_")
username = var.database_credentials.master_username
# Secure network configuration
db_subnet_group_name = aws_db_subnet_group.main.name
vpc_security_group_ids = [aws_security_group.database.id]
publicly_accessible = false
# Backup configuration
backup_retention_period = 30
backup_window = "03:00-04:00"
maintenance_window = "sun:04:00-sun:05:00"
# Monitoring
monitoring_interval = 60
monitoring_role_arn = aws_iam_role.rds_monitoring.arn
# Protection against deletion
deletion_protection = true
skip_final_snapshot = false
final_snapshot_identifier = "${var.project_name}-${var.environment}-final-snapshot"
tags = {
Name = "${var.project_name}-${var.environment}-master"
Environment = var.environment
}
}
Credential-handling strategies:
┌─────────────────────────────────────────────────────────────┐
│ CREDENTIAL HANDLING WORKFLOW │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. INPUT (Sensitive Variables) │
│ • Environment Variables & TF Cloud Workspaces │
│ • HashiCorp Vault & AWS Secrets Manager │
│ │
│ 2. PROCESSING (Terraform Core) │
│ • Marked as sensitive (masked in plan output) │
│ • Encrypted hand-off and exclusion from logs │
│ │
│ 3. STORAGE (State Backend) │
│ • Encryption at Rest & in Transit (TLS) │
│ • Strict access control and audit logging │
│ │
│ 4. USAGE (target resources) │
│ • Pass to cloud resources via secrets │
│ • Automated secret rotation and least privilege │
│ │
└─────────────────────────────────────────────────────────────┘
Best practices for credential handling:
| Aspect | Recommendation | Reason |
|---|---|---|
| Input | Environment variables or Vault | Do not store in code |
| Transfer | TLS-encrypted | Protection against eavesdropping |
| Storage | Encrypted state backend | Protection of state files |
| Use | Secrets Manager/Vault | Central management |
| Rotation | Automated | Regular renewal |
| Access | Least privilege | Minimal permissions |
🔧 Practical example - environment-variable setup:
# Secure environment-variable configuration
export TF_VAR_database_credentials='{
"master_username": "dbadmin",
"master_password": "super-secure-password-123!",
"replica_username": "readonly",
"replica_password": "another-secure-password-456!"
}'
export TF_VAR_external_api_keys='{
"github_token": "ghp_abcdef1234567890",
"docker_registry_token": "dckr_pat_abcdef1234567890",
"monitoring_api_key": "mon_abcdef1234567890",
"backup_service_key": "bck_abcdef1234567890"
}'
# Run Terraform with secure variables
terraform plan
terraform apply
⚠️ Security tip: Never use real credentials in examples or tests. Use placeholders or test credentials.
Sensitive-variable pitfalls:
| Problem | Risk | Solution |
|---|---|---|
| Unencrypted state | Credentials in plaintext | Encrypted backend |
| Logs contain secrets | Credential leakage | Log filtering |
| Plan output shows secrets | Accidental disclosure | Sensitive marking |
| Hardcoded credentials | Source-code leakage | Environment variables |
Output values with conditional logic
What are advanced output features? Outputs can do more than return values. They can contain conditional logic, transform values, be marked as sensitive and emit complex data structures. That makes them powerful tools for modules and automation.
Why do advanced outputs matter? They make Terraform modules more flexible and reusable. Conditional outputs let you return different information based on the configuration. That reduces the number of module variants you need.
What must you watch for with advanced outputs? Complex output logic can be hard to debug. Sensitive outputs must be marked correctly to prevent credential leakage. Performance can suffer with complex calculations.
What do you use advanced outputs for? Multi-environment modules, conditional resource information, complex data structures and integration with other tools.
# Advanced output configuration with conditional logic
output "network_configuration" {
description = "Complete network configuration details"
value = {
# Base information
vpc_id = aws_vpc.main.id
vpc_cidr = aws_vpc.main.cidr_block
# Conditional subnet information
public_subnets = var.network_config.public_subnets != [] ? {
ids = aws_subnet.public[*].id
cidrs = aws_subnet.public[*].cidr_block
availability_zones = aws_subnet.public[*].availability_zone
} : null
private_subnets = var.network_config.private_subnets != [] ? {
ids = aws_subnet.private_app[*].id
cidrs = aws_subnet.private_app[*].cidr_block
availability_zones = aws_subnet.private_app[*].availability_zone
} : null
database_subnets = var.network_config.database_subnets != [] ? {
ids = aws_subnet.private_db[*].id
cidrs = aws_subnet.private_db[*].cidr_block
availability_zones = aws_subnet.private_db[*].availability_zone
subnet_group_name = aws_db_subnet_group.main.name
} : null
# Conditional gateway information
internet_gateway = length(aws_subnet.public) > 0 ? {
id = aws_internet_gateway.main.id
route_table_id = aws_route_table.public.id
} : null
nat_gateways = var.network_config.enable_nat_gateway ? {
ids = aws_nat_gateway.main[*].id
eip_ids = aws_eip.nat[*].id
public_ips = aws_eip.nat[*].public_ip
} : null
# VPN gateway if enabled
vpn_gateway = var.network_config.enable_vpn_gateway ? {
id = aws_vpn_gateway.main[0].id
amazon_side_asn = aws_vpn_gateway.main[0].amazon_side_asn
} : null
# DNS configuration
dns_configuration = {
domain_name = var.network_config.dns_domain
private_zone_id = aws_route53_zone.private.zone_id
public_zone_id = var.network_config.dns_domain != "" ? aws_route53_zone.public[0].zone_id : null
}
# Environment-specific information
environment_details = {
environment = var.network_config.environment
is_production = var.network_config.environment == "prod"
monitoring_enabled = var.network_config.environment == "prod" ? true : false
backup_enabled = var.network_config.environment == "prod" ? true : false
}
}
}
# Advanced compute outputs
output "compute_configuration" {
description = "Complete compute configuration details"
value = {
# Launch template information
launch_template = {
id = aws_launch_template.app.id
latest_version = aws_launch_template.app.latest_version
name = aws_launch_template.app.name
}
# Auto Scaling group information
autoscaling_group = var.compute_config.auto_scaling_enabled ? {
name = aws_autoscaling_group.app[0].name
arn = aws_autoscaling_group.app[0].arn
min_size = aws_autoscaling_group.app[0].min_size
max_size = aws_autoscaling_group.app[0].max_size
desired_capacity = aws_autoscaling_group.app[0].desired_capacity
availability_zones = aws_autoscaling_group.app[0].availability_zones
} : null
# Load balancer information (if present)
load_balancer = var.compute_config.load_balancer_enabled ? {
arn = aws_lb.app[0].arn
dns_name = aws_lb.app[0].dns_name
zone_id = aws_lb.app[0].zone_id
target_group_arn = aws_lb_target_group.app[0].arn
} : null
# Spot instance information
spot_configuration = var.compute_config.spot_instances_enabled ? {
enabled = true
max_price = var.compute_config.spot_max_price
instance_types = var.compute_config.instance_types
} : {
enabled = false
max_price = null
instance_types = var.compute_config.instance_types
}
# Monitoring configuration
monitoring = {
cloudwatch_log_group = aws_cloudwatch_log_group.app.name
metrics_namespace = "${var.project_name}/${var.environment}"
dashboard_url = "https://console.aws.amazon.com/cloudwatch/home?region=${data.aws_region.current.name}#dashboards:name=${var.project_name}-${var.environment}"
}
}
}
# Sensitive database outputs
output "database_configuration" {
description = "Database configuration details (some values are sensitive)"
value = {
# Public information
instance_identifier = aws_db_instance.master.id
endpoint = aws_db_instance.master.endpoint
port = aws_db_instance.master.port
engine = aws_db_instance.master.engine
engine_version = aws_db_instance.master.engine_version
# Security configuration
security_configuration = {
encrypted = aws_db_instance.master.storage_encrypted
kms_key_id = aws_db_instance.master.kms_key_id
vpc_security_group_ids = aws_db_instance.master.vpc_security_group_ids
subnet_group_name = aws_db_instance.master.db_subnet_group_name
}
# Backup configuration
backup_configuration = {
retention_period = aws_db_instance.master.backup_retention_period
backup_window = aws_db_instance.master.backup_window
maintenance_window = aws_db_instance.master.maintenance_window
}
# Monitoring configuration
monitoring_configuration = {
monitoring_interval = aws_db_instance.master.monitoring_interval
monitoring_role_arn = aws_db_instance.master.monitoring_role_arn
performance_insights_enabled = aws_db_instance.master.performance_insights_enabled
}
# Replica information (if present)
read_replica = var.database_config.read_replica_enabled ? {
identifier = aws_db_instance.replica[0].id
endpoint = aws_db_instance.replica[0].endpoint
port = aws_db_instance.replica[0].port
} : null
}
# Not sensitive, because no credentials are included
sensitive = false
}
# Sensitive credential outputs
output "database_credentials" {
description = "Database credentials (sensitive)"
value = {
secrets_manager_arn = aws_secretsmanager_secret.database_credentials.arn
secrets_manager_name = aws_secretsmanager_secret.database_credentials.name
kms_key_arn = aws_kms_key.secrets.arn
kms_key_alias = aws_kms_alias.secrets.name
}
sensitive = true
}
# Advanced security outputs
output "security_configuration" {
description = "Security configuration details"
value = {
# Security groups
security_groups = {
alb = {
id = aws_security_group.alb.id
name = aws_security_group.alb.name
description = aws_security_group.alb.description
}
app_servers = {
id = aws_security_group.app_servers.id
name = aws_security_group.app_servers.name
description = aws_security_group.app_servers.description
}
database = {
id = aws_security_group.database.id
name = aws_security_group.database.name
description = aws_security_group.database.description
}
}
# KMS keys
kms_keys = {
secrets = {
id = aws_kms_key.secrets.id
arn = aws_kms_key.secrets.arn
alias = aws_kms_alias.secrets.name
}
}
# Compliance status
compliance_status = {
encryption_enabled = aws_db_instance.master.storage_encrypted
vpc_flow_logs_enabled = var.security_config.enable_flow_logs
cloudtrail_enabled = var.security_config.enable_cloudtrail
config_enabled = var.security_config.enable_config
guardduty_enabled = var.security_config.enable_guardduty
}
# Audit information
audit_configuration = {
cloudtrail_arn = var.security_config.enable_cloudtrail ? aws_cloudtrail.main[0].arn : null
config_recorder_name = var.security_config.enable_config ? aws_config_configuration_recorder.main[0].name : null
guardduty_detector_id = var.security_config.enable_guardduty ? aws_guardduty_detector.main[0].id : null
}
}
}
# Conditional outputs for different environments
output "environment_specific_outputs" {
description = "Environment-specific configuration outputs"
value = var.environment == "prod" ? {
# Production-specific outputs
type = "production"
high_availability = true
backup_enabled = true
monitoring_level = "detailed"
# Production URLs
application_url = "https://${var.network_config.dns_domain}"
admin_url = "https://admin.${var.network_config.dns_domain}"
api_url = "https://api.${var.network_config.dns_domain}"
# Production metrics
metrics_dashboard = "https://console.aws.amazon.com/cloudwatch/home?region=${data.aws_region.current.name}#dashboards:name=${var.project_name}-production"
cost_dashboard = "https://console.aws.amazon.com/billing/home#/dashboard"
# Production alerts
alert_endpoints = var.monitoring_config.alert_endpoints
escalation_policy = "immediate"
} : {
# Development-specific outputs
type = "development"
high_availability = false
backup_enabled = false
monitoring_level = "basic"
# Development URLs
application_url = "http://${aws_lb.app[0].dns_name}"
admin_url = "http://${aws_lb.app[0].dns_name}/admin"
api_url = "http://${aws_lb.app[0].dns_name}/api"
# Development notes
ssh_access = "Available from VPC"
debug_mode = "enabled"
auto_shutdown = "enabled"
}
}
# Aggregated outputs for other tools
output "terraform_outputs_summary" {
description = "Summary of all important outputs for external tools"
value = {
# Network summary
network = {
vpc_id = aws_vpc.main.id
public_subnet_ids = aws_subnet.public[*].id
private_subnet_ids = aws_subnet.private_app[*].id
database_subnet_ids = aws_subnet.private_db[*].id
}
# Security summary
security = {
alb_security_group_id = aws_security_group.alb.id
app_security_group_id = aws_security_group.app_servers.id
db_security_group_id = aws_security_group.database.id
}
# Compute summary
compute = {
launch_template_id = aws_launch_template.app.id
autoscaling_group_name = var.compute_config.auto_scaling_enabled ? aws_autoscaling_group.app[0].name : null
load_balancer_dns = var.compute_config.load_balancer_enabled ? aws_lb.app[0].dns_name : null
}
# Database summary
database = {
endpoint = aws_db_instance.master.endpoint
port = aws_db_instance.master.port
secrets_manager_arn = aws_secretsmanager_secret.database_credentials.arn
}
# Monitoring summary
monitoring = {
log_group_name = aws_cloudwatch_log_group.app.name
metrics_namespace = "${var.project_name}/${var.environment}"
}
}
}
❗ Output debugging techniques:
# Show all outputs
terraform output
# Show a specific output
terraform output network_configuration
# Raw output without formatting
terraform output -raw database_credentials
# JSON format for machine processing
terraform output -json | jq .
# Debug sensitive outputs (use carefully)
terraform output -json | jq '.database_credentials.value'
Output performance optimisation:
| Optimisation | Description | Use |
|---|---|---|
| Lazy evaluation | Compute outputs only when retrieved | Complex calculations |
| Caching | Computed values in locals | Reused values |
| Conditional logic | Only necessary outputs | Environment-dependent data |
| Structured data | Grouped outputs | Better organisation |
Output best practices:
- Use meaningful descriptions
- Group related outputs in objects
- Mark sensitive outputs correctly
- Use conditional logic for flexibility
- Document complex output structures
Local values (locals) for complex calculations
What are local values (locals)? Locals are computed values that can be defined and reused inside a Terraform configuration. They work like variables, but are calculated at runtime and can contain complex expressions, functions and conditional logic. Locals are the key to clean, maintainable Terraform configurations.
Why do locals matter so much? They eliminate code duplication, improve performance by caching calculations, and make complex logic readable. Without locals you would repeat the same complex expressions in several places, which leads to errors and poor maintainability.
What must you watch for with locals? Locals are recomputed on every plan and apply. Complex calculations can hurt performance. Circular dependencies between locals cause errors. Definition order matters.
What do you use locals for? Complex naming, conditional configurations, data transformations, tag generation, and anywhere you need to reuse complex logic.
# Advanced locals configuration for complex infrastructure
locals {
# ============================================================================
# BASE CALCULATIONS AND METADATA
# ============================================================================
# Current time and date for tagging
current_timestamp = timestamp()
current_date = formatdate("YYYY-MM-DD", local.current_timestamp)
current_time = formatdate("hh:mm:ss", local.current_timestamp)
# Git information (if available)
git_commit = try(file("${path.module}/.git/refs/heads/main"), "unknown")
git_branch = try(trimspace(file("${path.module}/.git/HEAD")), "unknown")
# Environment-specific settings
environment_config = {
dev = {
is_production = false
backup_retention = 7
monitoring_level = "basic"
instance_count = 1
storage_size = 20
enable_logging = true
enable_encryption = false
cost_optimization = true
}
staging = {
is_production = false
backup_retention = 14
monitoring_level = "standard"
instance_count = 2
storage_size = 50
enable_logging = true
enable_encryption = true
cost_optimization = true
}
prod = {
is_production = true
backup_retention = 30
monitoring_level = "detailed"
instance_count = 3
storage_size = 100
enable_logging = true
enable_encryption = true
cost_optimization = false
}
}
# Current environment configuration
current_env = local.environment_config[var.environment]
# ============================================================================
# ADVANCED NAMING AND TAGGING
# ============================================================================
# Base naming conventions
naming_convention = {
# Base prefix for all resources
prefix = "${var.project_name}-${var.environment}"
# Specific names for different resource types
vpc_name = "${var.project_name}-${var.environment}-vpc"
subnet_prefix = "${var.project_name}-${var.environment}"
sg_prefix = "${var.project_name}-${var.environment}"
# Advanced names with region and AZ
regional_prefix = "${var.project_name}-${var.environment}-${data.aws_region.current.name}"
# Names with hash for uniqueness
unique_suffix = substr(md5("${var.project_name}-${var.environment}-${local.current_timestamp}"), 0, 8)
unique_prefix = "${var.project_name}-${var.environment}-${local.unique_suffix}"
}
# Advanced tag generation
base_tags = {
# Project information
Project = var.project_name
Environment = var.environment
Region = data.aws_region.current.name
# Deployment information
ManagedBy = "terraform"
DeployedAt = local.current_timestamp
DeployedDate = local.current_date
TerraformWorkspace = terraform.workspace
# Git information (if available)
GitCommit = local.git_commit
GitBranch = local.git_branch
# Environment-specific tags
IsProduction = local.current_env.is_production
BackupPolicy = "${local.current_env.backup_retention}days"
MonitoringLevel = local.current_env.monitoring_level
CostOptimized = local.current_env.cost_optimization
}
# Resource-specific tags
compute_tags = merge(local.base_tags, {
ResourceType = "compute"
AutoShutdown = local.current_env.cost_optimization
InstanceCount = local.current_env.instance_count
})
storage_tags = merge(local.base_tags, {
ResourceType = "storage"
Encrypted = local.current_env.enable_encryption
BackupEnabled = true
StorageSize = "${local.current_env.storage_size}GB"
})
network_tags = merge(local.base_tags, {
ResourceType = "network"
VPCFlowLogs = local.current_env.enable_logging
})
# ============================================================================
# NETWORK CALCULATIONS
# ============================================================================
# Automatic CIDR calculation for subnets
vpc_cidr = var.vpc_cidr
# Compute subnet CIDRs automatically based on AZs
availability_zones = data.aws_availability_zones.available.names
az_count = length(local.availability_zones)
# Public subnets (first X subnets)
public_subnet_cidrs = [
for i in range(local.az_count) :
cidrsubnet(local.vpc_cidr, 8, i)
]
# Private app subnets (next X subnets)
private_app_subnet_cidrs = [
for i in range(local.az_count) :
cidrsubnet(local.vpc_cidr, 8, i + 10)
]
# Database subnets (last X subnets)
database_subnet_cidrs = [
for i in range(local.az_count) :
cidrsubnet(local.vpc_cidr, 8, i + 20)
]
# Subnet configurations for different tiers
subnet_configurations = {
public = {
name_prefix = "${local.naming_convention.subnet_prefix}-public"
cidrs = local.public_subnet_cidrs
map_public_ip = true
route_table_type = "public"
tags = merge(local.network_tags, {
SubnetType = "public"
InternetAccess = "direct"
})
}
private_app = {
name_prefix = "${local.naming_convention.subnet_prefix}-private-app"
cidrs = local.private_app_subnet_cidrs
map_public_ip = false
route_table_type = "private"
tags = merge(local.network_tags, {
SubnetType = "private"
InternetAccess = "nat"
Tier = "application"
})
}
database = {
name_prefix = "${local.naming_convention.subnet_prefix}-private-db"
cidrs = local.database_subnet_cidrs
map_public_ip = false
route_table_type = "database"
tags = merge(local.network_tags, {
SubnetType = "private"
InternetAccess = "none"
Tier = "data"
})
}
}
# ============================================================================
# SECURITY CONFIGURATIONS
# ============================================================================
# Security group rules based on environment
security_group_rules = {
# ALB security group
alb = {
name = "${local.naming_convention.sg_prefix}-alb"
description = "Security group for Application Load Balancer"
ingress_rules = [
{
description = "HTTP from internet"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
},
{
description = "HTTPS from internet"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
]
egress_rules = [
{
description = "All outbound traffic"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
]
}
# Application servers security group
app = {
name = "${local.naming_convention.sg_prefix}-app"
description = "Security group for application servers"
ingress_rules = concat(
[
{
description = "HTTP from ALB"
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = local.private_app_subnet_cidrs
}
],
# SSH access only in development
local.current_env.is_production == false ? [
{
description = "SSH from VPC"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [local.vpc_cidr]
}
] : []
)
egress_rules = [
{
description = "All outbound traffic"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
]
}
# Database security group
database = {
name = "${local.naming_convention.sg_prefix}-db"
description = "Security group for database servers"
ingress_rules = [
{
description = "MySQL from app servers"
from_port = 3306
to_port = 3306
protocol = "tcp"
cidr_blocks = local.private_app_subnet_cidrs
}
]
egress_rules = [] # No outbound rules for databases
}
}
# ============================================================================
# COMPUTE CONFIGURATIONS
# ============================================================================
# Instance configuration based on environment
instance_configurations = {
# Launch template configuration
launch_template = {
name_prefix = "${local.naming_convention.prefix}-app"
image_id = data.aws_ami.ubuntu.id
instance_type = local.current_env.is_production ? "t3.large" : "t3.micro"
# User data with complex template variables
user_data_vars = {
project_name = var.project_name
environment = var.environment
region = data.aws_region.current.name
# Environment-specific variables
debug_mode = !local.current_env.is_production
log_level = local.current_env.is_production ? "INFO" : "DEBUG"
monitoring_enabled = local.current_env.monitoring_level != "basic"
# Database configuration
database_host = "localhost" # Replaced later by RDS
database_port = 3306
database_name = replace("${var.project_name}_${var.environment}", "-", "_")
# Application configuration
app_port = 8080
health_check_path = "/health"
metrics_enabled = local.current_env.monitoring_level == "detailed"
# Computed values
instance_role = "application"
deployment_timestamp = local.current_timestamp
}
# Block device mappings
block_device_mappings = [
{
device_name = "/dev/sda1"
volume_size = local.current_env.storage_size
volume_type = "gp3"
encrypted = local.current_env.enable_encryption
iops = local.current_env.is_production ? 3000 : 1000
}
]
# Instance tags
tags = merge(local.compute_tags, {
Name = "${local.naming_convention.prefix}-app"
InstanceProfile = "application"
})
}
# Auto Scaling group configuration
autoscaling_group = {
name = "${local.naming_convention.prefix}-app-asg"
min_size = local.current_env.is_production ? 2 : 1
max_size = local.current_env.is_production ? 10 : 3
desired_capacity = local.current_env.instance_count
# Health check configuration
health_check_type = "ELB"
health_check_grace_period = 300
# Termination policies
termination_policies = ["OldestInstance"]
# Scaling policies
target_group_arns = [] # Set later
tags = merge(local.compute_tags, {
Name = "${local.naming_convention.prefix}-app-asg"
AutoScalingGroup = "application"
})
}
}
# ============================================================================
# DATABASE CONFIGURATIONS
# ============================================================================
# RDS configuration based on environment
database_configurations = {
master = {
identifier = "${local.naming_convention.prefix}-master"
engine = "mysql"
engine_version = "8.0"
instance_class = local.current_env.is_production ? "db.t3.medium" : "db.t3.micro"
# Storage configuration
allocated_storage = local.current_env.storage_size
max_allocated_storage = local.current_env.storage_size * 2
storage_type = "gp3"
storage_encrypted = local.current_env.enable_encryption
# Backup configuration
backup_retention_period = local.current_env.backup_retention
backup_window = "03:00-04:00"
maintenance_window = "sun:04:00-sun:05:00"
# Monitoring configuration
monitoring_interval = local.current_env.monitoring_level == "detailed" ? 15 : 60
performance_insights_enabled = local.current_env.monitoring_level == "detailed"
# Multi-AZ for production
multi_az = local.current_env.is_production
# Database name
db_name = replace("${var.project_name}_${var.environment}", "-", "_")
# Parameter group
parameter_group_family = "mysql8.0"
tags = merge(local.storage_tags, {
Name = "${local.naming_convention.prefix}-master"
DatabaseRole = "master"
Engine = "mysql"
})
}
# Read replica only for production
replica = local.current_env.is_production ? {
identifier = "${local.naming_convention.prefix}-replica"
replicate_source_db = "${local.naming_convention.prefix}-master"
instance_class = "db.t3.medium"
# Monitoring for replica
monitoring_interval = 60
performance_insights_enabled = false
tags = merge(local.storage_tags, {
Name = "${local.naming_convention.prefix}-replica"
DatabaseRole = "replica"
Engine = "mysql"
})
} : null
}
# ============================================================================
# MONITORING AND LOGGING CONFIGURATIONS
# ============================================================================
# CloudWatch configuration
cloudwatch_configurations = {
# Log groups
log_groups = {
application = {
name = "/aws/ec2/${local.naming_convention.prefix}/application"
retention_days = local.current_env.is_production ? 90 : 7
tags = merge(local.base_tags, {
LogType = "application"
})
}
system = {
name = "/aws/ec2/${local.naming_convention.prefix}/system"
retention_days = local.current_env.is_production ? 30 : 7
tags = merge(local.base_tags, {
LogType = "system"
})
}
access = {
name = "/aws/elb/${local.naming_convention.prefix}/access"
retention_days = local.current_env.is_production ? 180 : 14
tags = merge(local.base_tags, {
LogType = "access"
})
}
}
# CloudWatch alarms
alarms = local.current_env.monitoring_level != "basic" ? {
high_cpu = {
alarm_name = "${local.naming_convention.prefix}-high-cpu"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = "2"
metric_name = "CPUUtilization"
namespace = "AWS/EC2"
period = "120"
statistic = "Average"
threshold = "80"
alarm_description = "This metric monitors ec2 cpu utilization"
alarm_actions = [] # SNS topics added later
}
high_memory = {
alarm_name = "${local.naming_convention.prefix}-high-memory"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = "2"
metric_name = "MemoryUtilization"
namespace = "CWAgent"
period = "120"
statistic = "Average"
threshold = "85"
alarm_description = "This metric monitors memory utilization"
alarm_actions = []
}
} : {}
}
# ============================================================================
# COMPUTED OUTPUT STRUCTURES
# ============================================================================
# Computed outputs for other modules
computed_outputs = {
# Network summary
network_summary = {
vpc_id = var.vpc_id # Replaced by the real VPC ID
vpc_cidr = local.vpc_cidr
availability_zones = local.availability_zones
subnet_count = local.az_count
# Subnet information
public_subnets = {
count = local.az_count
cidrs = local.public_subnet_cidrs
}
private_app_subnets = {
count = local.az_count
cidrs = local.private_app_subnet_cidrs
}
database_subnets = {
count = local.az_count
cidrs = local.database_subnet_cidrs
}
}
# Compute summary
compute_summary = {
instance_type = local.instance_configurations.launch_template.instance_type
min_instances = local.instance_configurations.autoscaling_group.min_size
max_instances = local.instance_configurations.autoscaling_group.max_size
desired_instances = local.instance_configurations.autoscaling_group.desired_capacity
storage_size = "${local.current_env.storage_size}GB"
encrypted = local.current_env.enable_encryption
}
# Database summary
database_summary = {
engine = local.database_configurations.master.engine
engine_version = local.database_configurations.master.engine_version
instance_class = local.database_configurations.master.instance_class
storage_size = "${local.database_configurations.master.allocated_storage}GB"
backup_retention = "${local.database_configurations.master.backup_retention_period}days"
multi_az = local.database_configurations.master.multi_az
has_replica = local.database_configurations.replica != null
}
# Environment summary
environment_summary = {
environment = var.environment
is_production = local.current_env.is_production
monitoring_level = local.current_env.monitoring_level
cost_optimized = local.current_env.cost_optimization
backup_retention = local.current_env.backup_retention
encryption_enabled = local.current_env.enable_encryption
}
}
}
Performance optimisation for locals:
# Performance-optimised locals strategies
locals {
# ✅ Good: cache simple calculations
vpc_cidr_base = split("/", var.vpc_cidr)[0]
vpc_cidr_prefix = split("/", var.vpc_cidr)[1]
# ✅ Good: cache complex list operations
filtered_availability_zones = [
for az in data.aws_availability_zones.available.names :
az if length(regexall("us-west-2[abc]", az)) > 0
]
# ✅ Good: cache conditional logic
production_settings = var.environment == "prod" ? {
instance_count = 5
backup_retention = 30
monitoring = "detailed"
} : {
instance_count = 1
backup_retention = 7
monitoring = "basic"
}
# ⚠️ Caution: very complex calculations
# Use these only when they are really needed
complex_subnet_calculation = {
for i, az in local.filtered_availability_zones :
az => {
public_cidr = cidrsubnet(var.vpc_cidr, 8, i)
private_cidr = cidrsubnet(var.vpc_cidr, 8, i + 10)
database_cidr = cidrsubnet(var.vpc_cidr, 8, i + 20)
route_table_id = "rt-${md5("${az}-${i}")}"
}
}
}
Locals organisation strategy (layer model):
┌─────────────────────────────────────────────────────────────┐
│ Locals Organization Strategy │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ 1. BASE CONFIGURATION │ │
│ │ ───────────────────────────────────────────────────── │ │
│ │ - Environment Settings - Common Tags │ │
│ │ - Naming Conventions - Timestamps │ │
│ └───────────────────────────┬───────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ 2. COMPUTED VALUES │ │
│ │ ───────────────────────────────────────────────────── │ │
│ │ - Network CIDRs - Conditional Logic │ │
│ │ - Resource Counts - Transformations │ │
│ └───────────────────────────┬───────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ 3. CONFIGURATIONS │ │
│ │ ───────────────────────────────────────────────────── │ │
│ │ - Security Groups - Database Settings │ │
│ │ - Launch Templates - Monitoring & Alarms │ │
│ └───────────────────────────┬───────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ 4. OUTPUT STRUCTURES │ │
│ │ ───────────────────────────────────────────────────── │ │
│ │ - Summaries - Exports │ │
│ │ - Integration Data - Module Payloads │ │
│ └───────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Locals best practices:
| Best practice | Description | Example |
|---|---|---|
| Logical grouping | Group related calculations | Network, Compute, Database |
| Meaningful names | Self-explanatory local names | current_env instead of env |
| Comments for complexity | Document complex logic | Why particular calculations |
| Watch performance | Cache heavy calculations | Compute once, use often |
| Manage dependencies | Avoid circular references | Clear hierarchy |
🔧 Practical example - locals in real resources:
# Using locals in real resources
resource "aws_subnet" "public" {
count = length(local.availability_zones)
vpc_id = aws_vpc.main.id
cidr_block = local.subnet_configurations.public.cidrs[count.index]
availability_zone = local.availability_zones[count.index]
map_public_ip_on_launch = local.subnet_configurations.public.map_public_ip
tags = merge(
local.subnet_configurations.public.tags,
{
Name = "${local.subnet_configurations.public.name_prefix}-${count.index + 1}"
AvailabilityZone = local.availability_zones[count.index]
}
)
}
resource "aws_launch_template" "app" {
name_prefix = local.instance_configurations.launch_template.name_prefix
image_id = local.instance_configurations.launch_template.image_id
instance_type = local.instance_configurations.launch_template.instance_type
vpc_security_group_ids = [aws_security_group.app.id]
# Use the computed user-data variables
user_data = base64encode(templatefile("${path.module}/userdata.sh",
local.instance_configurations.launch_template.user_data_vars
))
# Use the computed block device mappings
dynamic "block_device_mappings" {
for_each = local.instance_configurations.launch_template.block_device_mappings
content {
device_name = block_device_mappings.value.device_name
ebs {
volume_size = block_device_mappings.value.volume_size
volume_type = block_device_mappings.value.volume_type
encrypted = block_device_mappings.value.encrypted
iops = block_device_mappings.value.iops
}
}
}
tag_specifications {
resource_type = "instance"
tags = local.instance_configurations.launch_template.tags
}
}
Locals performance tips:
- Use locals for values that are used more than once
- Cache complex for-expressions in locals
- Avoid deep nesting in locals
- Use separate locals blocks for better organisation
Locals pitfalls:
| Problem | Symptom | Solution |
|---|---|---|
| Circular dependencies | Error: Cycle in local values |
Restructure dependencies |
| Performance problems | Slow plan times | Optimise complex calculations |
| Unreadable complexity | Hard-to-understand locals | Split into smaller parts |
| Order problems | Error: Reference to undeclared |
Order local definitions |
❗ Locals debugging techniques:
# Show locals values in terraform console
terraform console
> local.current_env
> local.naming_convention.prefix
> local.computed_outputs.network_summary
# Show locals values in the plan
terraform plan | grep "local\."
# Analyse locals dependencies
terraform graph | grep local
With these advanced variable and output features you write not only functional but intelligent Terraform code. Complex validations catch errors before they become expensive, sensitive variables protect your credentials, locals eliminate duplication and keep complex logic maintainable, and advanced outputs make modules flexible and reusable. These techniques are the difference between simple Terraform scripts and professional, production-ready infrastructure definitions.
Built-in functions and expressions
What are built-in functions in Terraform? Built-in functions are ready-made tools integrated into HCL that enable complex operations. They turn static configurations into dynamic, intelligent infrastructure definitions. Terraform offers more than 100 built-in functions for string manipulation, maths, collection processing and conditional logic.
Why are built-in functions decisive? They remove the need for external scripting and make Terraform configurations self-contained. Without them you would have to push complex logic into external tools, which leads to fragile, hard-to-maintain setups. Built-in functions keep everything in Terraform and make infrastructure predictable.
What must you watch for with built-in functions? Functions run on every plan and apply. Complex nesting can hurt performance and reduce readability. Not all functions are available in all Terraform versions. Missing input validation can cause runtime errors.
What do you use built-in functions for? Dynamic naming, data transformation, conditional logic, collection processing, JSON/YAML generation and complex calculations needed when creating infrastructure.
String manipulation and formatting
What does Terraform offer for string operations? Terraform has extensive string functions for formatting, manipulation, validation and transformation. Those functions let you create dynamic configurations that can adapt to different environments.
Why is string manipulation so important? In Terraform you constantly work with strings — resource names, tags, URLs, paths and configuration values. Professional string manipulation makes configurations flexible and maintainable. Without it you get static, hard-to-change definitions.
What should you watch for with string functions? String operations are case-sensitive. Empty strings and null values can produce unexpected results. Regex patterns must be escaped correctly. Performance can suffer on large string operations.
What do you use string functions for? Resource naming, tag generation, URL construction, path manipulation, JSON template creation and data validation.
🔧 Practical example - advanced string functions:
# Extensive string manipulation for infrastructure naming
locals {
# Base variables for string operations
project_name = "e-commerce-platform"
environment = "production"
region = "us-east-1"
team_name = "Platform Engineering"
# ============================================================================
# STRING FORMATTING AND TRANSFORMATION
# ============================================================================
# Base string operations
string_operations = {
# Case manipulation
project_upper = upper(local.project_name) # "E-COMMERCE-PLATFORM"
project_lower = lower(local.project_name) # "e-commerce-platform"
project_title = title(replace(local.project_name, "-", " ")) # "E Commerce Platform"
# String cleanup for different contexts
dns_safe_name = replace(lower(local.project_name), "_", "-") # DNS-safe names
db_safe_name = replace(replace(local.project_name, "-", "_"), ".", "_") # Database-safe names
s3_safe_name = replace(lower(local.project_name), "_", "-") # S3-safe names
# Length validation and adjustment
truncated_name = substr(local.project_name, 0, 10) # "e-commerce" (first 10 characters)
padded_name = format("%-20s", local.project_name) # Left-aligned to 20 characters
# String combinations
full_prefix = join("-", [local.project_name, local.environment, local.region])
compact_prefix = join("", [substr(local.project_name, 0, 3), local.environment, substr(local.region, -1, 1)])
}
# Advanced naming conventions with string functions
naming_patterns = {
# Standard resource names
vpc_name = format("%s-%s-vpc", local.string_operations.dns_safe_name, local.environment)
# Subnet names with index formatting
subnet_pattern = format("%s-%s-subnet-%%02d", local.string_operations.dns_safe_name, local.environment)
# Security group names with role
sg_pattern = format("%s-%s-%%s-sg", local.string_operations.dns_safe_name, local.environment)
# Database names (underscores for SQL compatibility)
database_name = format("%s_%s_db", local.string_operations.db_safe_name, local.environment)
# S3 bucket names (globally unique)
bucket_pattern = format("%s-%s-%%s-%s",
local.string_operations.s3_safe_name,
local.environment,
formatdate("YYYY-MM", timestamp())
)
# CloudWatch log groups
log_group_pattern = format("/aws/%%s/%s-%s", local.string_operations.dns_safe_name, local.environment)
}
# ============================================================================
# ADVANCED STRING VALIDATION AND CLEANUP
# ============================================================================
# String validation with regex
validation_results = {
# DNS name validation
is_valid_dns = can(regex("^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$", local.string_operations.dns_safe_name))
# Email validation
admin_email = "admin@company.com"
is_valid_email = can(regex("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$", local.validation_results.admin_email))
# IP address validation
vpc_cidr = "10.0.0.0/16"
is_valid_cidr = can(cidrhost(local.validation_results.vpc_cidr, 0))
# Password complexity check
temp_password = "TempPass123!"
has_upper = can(regex("[A-Z]", local.validation_results.temp_password))
has_lower = can(regex("[a-z]", local.validation_results.temp_password))
has_digit = can(regex("[0-9]", local.validation_results.temp_password))
has_special = can(regex("[!@#$%^&*()_+-=]", local.validation_results.temp_password))
is_complex_password = (
local.validation_results.has_upper &&
local.validation_results.has_lower &&
local.validation_results.has_digit &&
local.validation_results.has_special &&
length(local.validation_results.temp_password) >= 8
)
}
# String template functions
template_strings = {
# Advanced formatting with multiple variables
resource_description = format(
"Managed by Terraform for %s project in %s environment, deployed to %s region by %s team",
local.project_name,
local.environment,
local.region,
local.team_name
)
# JSON template for policies
policy_template = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"s3:GetObject",
"s3:PutObject"
]
Resource = format("arn:aws:s3:::%s/*", format(local.naming_patterns.bucket_pattern, "data"))
Condition = {
StringEquals = {
"s3:x-amz-server-side-encryption" = "AES256"
}
}
}
]
})
# YAML template for user data
user_data_yaml = yamlencode({
users = [
{
name = "ubuntu"
sudo = "ALL=(ALL) NOPASSWD:ALL"
shell = "/bin/bash"
ssh_authorized_keys = [
format("ssh-rsa %s %s@%s",
"AAAAB3NzaC1yc2EAAAADAQABAAABAQC...",
"admin",
replace(local.project_name, "-", "")
)
]
}
]
packages = [
"curl",
"wget",
"unzip",
"docker.io"
]
runcmd = [
format("echo 'Project: %s' >> /etc/motd", local.project_name),
format("echo 'Environment: %s' >> /etc/motd", local.environment),
"systemctl enable docker",
"systemctl start docker"
]
})
}
# ============================================================================
# URL AND PATH MANIPULATION
# ============================================================================
# URL construction for various services
service_urls = {
# Application URLs
app_base_url = format("https://%s.%s.example.com", local.environment, replace(local.project_name, "-", ""))
api_url = format("%s/api/v1", local.service_urls.app_base_url)
admin_url = format("%s/admin", local.service_urls.app_base_url)
# Monitoring URLs
grafana_url = format("https://grafana-%s.monitoring.example.com", local.environment)
prometheus_url = format("https://prometheus-%s.monitoring.example.com", local.environment)
# Dashboard URLs with URL encoding
cloudwatch_dashboard = format(
"https://console.aws.amazon.com/cloudwatch/home?region=%s#dashboards:name=%s",
local.region,
urlencode(format("%s-%s-dashboard", local.project_name, local.environment))
)
}
# Path manipulation for different contexts
file_paths = {
# Linux paths for configuration
app_config_dir = format("/etc/%s", replace(local.project_name, "-", "_"))
app_log_dir = format("/var/log/%s", replace(local.project_name, "-", "_"))
app_data_dir = format("/opt/%s/data", replace(local.project_name, "-", "_"))
# Relative paths for modules
module_path = format("./modules/%s", local.environment)
template_path = format("${path.module}/templates/%s", local.environment)
# S3 paths for backups
backup_prefix = format("backups/%s/%s", local.environment, formatdate("YYYY/MM/DD", timestamp()))
log_prefix = format("logs/%s/%s", local.environment, formatdate("YYYY/MM/DD", timestamp()))
}
}
# Practical use of string functions in resources
resource "aws_s3_bucket" "data" {
bucket = format(local.naming_patterns.bucket_pattern, "data")
tags = {
Name = format(local.naming_patterns.bucket_pattern, "data")
Description = local.template_strings.resource_description
Environment = local.environment
Project = local.project_name
}
}
resource "aws_s3_bucket_object" "config" {
bucket = aws_s3_bucket.data.bucket
key = format("%s/config.yaml", local.file_paths.backup_prefix)
content = local.template_strings.user_data_yaml
tags = {
ConfigType = "user-data"
CreatedAt = formatdate("YYYY-MM-DD hh:mm:ss ZZZ", timestamp())
}
}
# CloudWatch log group with string manipulation
resource "aws_cloudwatch_log_group" "application" {
name = format(local.naming_patterns.log_group_pattern, "application")
retention_in_days = local.environment == "prod" ? 90 : 7
tags = {
Name = format("Logs for %s", title(replace(local.project_name, "-", " ")))
LogType = "application"
}
}
String function categories:
| Category | Functions | Use case | Performance |
|---|---|---|---|
| Case manipulation | upper(), lower(), title() |
Naming conventions | Very high |
| String operations | substr(), replace(), trim() |
Text transformation | High |
| Formatting | format(), formatdate() |
Template generation | High |
| Validation | regex(), can() |
Input validation | Medium |
| Encoding | urlencode(), base64encode() |
Data transfer | High |
String performance tips:
- Cache complex string operations in locals
- Use format() instead of string interpolation for better readability
- Validate inputs early with can() and regex()
- Use trimspace() to clean user input
Collection functions
What are collection functions? Collection functions process lists, maps and sets. They let you filter, transform, combine and organise data. These functions are the heart of dynamic Terraform configurations.
Why are collection functions indispensable? They eliminate static, repetitive configurations. Instead of defining dozens of similar resources by hand, you create them dynamically based on data. That makes infrastructure more scalable and maintainable.
What must you watch for with collection functions? Performance can suffer with large collections. Nested collection operations can become hard to read. Type consistency matters — do not mix different data types. Null values can produce unexpected results.
What do you use collection functions for? Dynamic resource creation, data aggregation, configuration merging, filtering of inputs and transformation between data formats.
# Advanced collection functions for dynamic infrastructure
locals {
# ============================================================================
# BASE DATA FOR COLLECTION OPERATIONS
# ============================================================================
# Availability zone data
all_availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e", "us-east-1f"]
# Environment configurations
environment_configs = {
dev = {
instance_count = 1
instance_type = "t3.micro"
storage_size = 20
backup_enabled = false
monitoring_level = "basic"
allowed_cidrs = ["10.0.0.0/16", "172.16.0.0/12"]
}
staging = {
instance_count = 2
instance_type = "t3.small"
storage_size = 50
backup_enabled = true
monitoring_level = "standard"
allowed_cidrs = ["10.0.0.0/16", "172.16.0.0/12", "192.168.0.0/16"]
}
prod = {
instance_count = 5
instance_type = "t3.large"
storage_size = 100
backup_enabled = true
monitoring_level = "detailed"
allowed_cidrs = ["10.0.0.0/16"]
}
}
# Application services with different requirements
application_services = {
frontend = {
port = 80
protocol = "HTTP"
health_check_path = "/health"
cpu_request = "100m"
memory_request = "128Mi"
replicas = 3
environment_variables = {
NODE_ENV = "production"
API_URL = "https://api.example.com"
CDN_URL = "https://cdn.example.com"
}
}
backend = {
port = 8080
protocol = "HTTP"
health_check_path = "/api/health"
cpu_request = "500m"
memory_request = "512Mi"
replicas = 5
environment_variables = {
DATABASE_URL = "postgresql://..."
REDIS_URL = "redis://..."
LOG_LEVEL = "info"
}
}
worker = {
port = 9090
protocol = "HTTP"
health_check_path = "/metrics"
cpu_request = "200m"
memory_request = "256Mi"
replicas = 2
environment_variables = {
QUEUE_URL = "redis://..."
WORKER_CONCURRENCY = "10"
LOG_LEVEL = "debug"
}
}
database = {
port = 5432
protocol = "TCP"
health_check_path = null
cpu_request = "1000m"
memory_request = "2Gi"
replicas = 1
environment_variables = {
POSTGRES_DB = "application"
POSTGRES_USER = "app_user"
POSTGRES_MAX_CONNECTIONS = "100"
}
}
}
# ============================================================================
# FOR-EXPRESSIONS FOR DATA TRANSFORMATION
# ============================================================================
# Advanced for-expressions for various use cases
for_expression_examples = {
# Simple list transformation
subnet_cidrs = [
for i, az in slice(local.all_availability_zones, 0, 3) :
cidrsubnet("10.0.0.0/16", 8, i)
]
# Map transformation with conditional logic
environment_instance_types = {
for env, config in local.environment_configs :
env => config.instance_type
if config.instance_count > 0
}
# Complex object transformation
service_configurations = {
for service_name, service_config in local.application_services :
service_name => {
name = service_name
port = service_config.port
replicas = service_config.replicas
resource_requests = {
cpu = service_config.cpu_request
memory = service_config.memory_request
}
environment = [
for key, value in service_config.environment_variables :
{
name = key
value = value
}
]
health_check = service_config.health_check_path != null ? {
path = service_config.health_check_path
port = service_config.port
} : null
}
}
# Nested for-expressions for multi-dimensional data
all_service_environment_combinations = flatten([
for env_name, env_config in local.environment_configs : [
for service_name, service_config in local.application_services : {
key = "${env_name}-${service_name}"
environment = env_name
service = service_name
instance_count = env_config.instance_count
service_replicas = service_config.replicas
total_instances = env_config.instance_count * service_config.replicas
resource_suffix = "${env_name}-${service_name}"
}
if service_name != "database" || env_name == "prod" # Database only in production
]
])
# Conditional for-expressions
production_services = {
for service_name, service_config in local.application_services :
service_name => merge(service_config, {
replicas = service_config.replicas * 2 # Double replicas for production
monitoring_enabled = true
})
if var.environment == "prod"
}
}
# ============================================================================
# FILTER FUNCTIONS FOR DATA SELECTION
# ============================================================================
# Advanced filtering operations
filtered_data = {
# Filter availability zones (only the first 3)
selected_azs = slice(local.all_availability_zones, 0, 3)
# Filter services by type
web_services = {
for name, config in local.application_services :
name => config
if config.protocol == "HTTP"
}
# Identify high-memory services
memory_intensive_services = [
for name, config in local.application_services :
name
if can(regex("Gi", config.memory_request))
]
# Environment-specific allowed CIDRs
current_allowed_cidrs = lookup(local.environment_configs, var.environment, {}).allowed_cidrs
# Services with health checks
services_with_health_checks = [
for name, config in local.application_services :
{
name = name
health_check_path = config.health_check_path
port = config.port
}
if config.health_check_path != null
]
# Conditional service configuration based on environment
environment_specific_services = {
for name, config in local.application_services :
name => merge(config, {
# Development: reduced replicas
replicas = var.environment == "dev" ? 1 : config.replicas
# Production: increased resources
cpu_request = var.environment == "prod" ? "${tonumber(split("m", config.cpu_request)[0]) * 2}m" : config.cpu_request
})
}
}
# ============================================================================
# MERGE FUNCTIONS FOR DATA COMBINATION
# ============================================================================
# Advanced merge operations
merged_configurations = {
# Merge base tags with environment-specific tags
base_tags = {
ManagedBy = "terraform"
Project = var.project_name
CreatedAt = timestamp()
}
environment_tags = {
Environment = var.environment
CostCenter = var.environment == "prod" ? "production" : "development"
BackupPolicy = var.environment == "prod" ? "daily" : "weekly"
}
# Final tags for all resources
final_tags = merge(
local.merged_configurations.base_tags,
local.merged_configurations.environment_tags,
var.additional_tags # External tags
)
# Merge service configurations with environment overrides
final_service_configs = {
for name, config in local.application_services :
name => merge(
config,
lookup(local.environment_configs[var.environment], "service_overrides", {}),
{
# Environment-specific adjustments
replicas = var.environment == "dev" ? 1 : (
var.environment == "staging" ? max(1, config.replicas - 1) : config.replicas
)
monitoring_enabled = var.environment == "prod"
log_level = var.environment == "prod" ? "warn" : "debug"
}
)
}
# Merge network configuration
network_config = merge(
{
# Base network settings
vpc_cidr = "10.0.0.0/16"
enable_nat_gateway = true
enable_vpn_gateway = false
},
# Environment-specific network settings
var.environment == "prod" ? {
enable_nat_gateway = true
enable_vpn_gateway = true
enable_flow_logs = true
enable_dns_hostnames = true
} : {},
# Additional custom settings
var.custom_network_config
)
}
# ============================================================================
# FLATTEN FUNCTIONS FOR HIERARCHICAL DATA
# ============================================================================
# Complex flatten operations
flattened_data = {
# Flatten all service environment-variable combinations
all_environment_variables = flatten([
for service_name, service_config in local.application_services : [
for var_name, var_value in service_config.environment_variables : {
service = service_name
variable_name = var_name
variable_value = var_value
full_key = "${service_name}_${var_name}"
}
]
])
# Flatten security group rules for all services
all_security_group_rules = flatten([
for service_name, service_config in local.application_services : [
{
service = service_name
type = "ingress"
from_port = service_config.port
to_port = service_config.port
protocol = "tcp"
description = "Allow ${service_config.protocol} traffic to ${service_name}"
},
# Health-check port (if different)
service_config.health_check_path != null && service_config.port != 80 ? {
service = service_name
type = "ingress"
from_port = 80
to_port = 80
protocol = "tcp"
description = "Allow health check traffic to ${service_name}"
} : null
]
# Filter out null values
if service_config.port != null
])
# Flatten subnet configurations for all AZs and tiers
all_subnet_configurations = flatten([
for tier in ["public", "private", "database"] : [
for i, az in local.filtered_data.selected_azs : {
name = "${var.project_name}-${var.environment}-${tier}-${i + 1}"
tier = tier
availability_zone = az
cidr_block = cidrsubnet("10.0.0.0/16", 8,
tier == "public" ? i : (
tier == "private" ? i + 10 : i + 20
)
)
map_public_ip = tier == "public"
route_table_association = tier == "public" ? "public" : "private"
}
]
])
# Flatten load balancer target configurations
load_balancer_targets = flatten([
for service_name, service_config in local.filtered_data.web_services : [
for i in range(service_config.replicas) : {
service = service_name
instance_index = i
target_id = "${service_name}-${i}"
port = service_config.port
health_check_path = service_config.health_check_path
}
]
])
}
# ============================================================================
# ADVANCED COLLECTION COMBINATIONS
# ============================================================================
# Combine complex collection operations
advanced_operations = {
# Generate service-discovery configuration
service_discovery_config = {
for service_name, service_config in local.application_services :
service_name => {
name = service_name
port = service_config.port
instances = [
for i in range(service_config.replicas) : {
id = "${service_name}-${i}"
address = "10.0.${i + 1}.${index(keys(local.application_services), service_name) + 10}"
port = service_config.port
health_check = service_config.health_check_path != null ? {
http = "http://10.0.${i + 1}.${index(keys(local.application_services), service_name) + 10}:${service_config.port}${service_config.health_check_path}"
} : null
}
]
load_balancer = {
algorithm = "round_robin"
health_check = service_config.health_check_path != null
}
}
}
# Monitoring configuration for all services
monitoring_targets = merge([
for service_name, service_config in local.application_services : {
for i in range(service_config.replicas) :
"${service_name}-${i}" => {
job_name = service_name
instance = "${service_name}-${i}"
targets = ["10.0.${i + 1}.${index(keys(local.application_services), service_name) + 10}:${service_config.port}"]
labels = {
service = service_name
environment = var.environment
replica = tostring(i)
}
scrape_interval = "30s"
metrics_path = "/metrics"
}
}
if service_config.health_check_path != null # Monitor only services with health checks
]...)
# Calculate resource quotas based on service requirements
total_resource_requirements = {
total_cpu = sum([
for service_name, service_config in local.application_services :
tonumber(split("m", service_config.cpu_request)[0]) * service_config.replicas
])
total_memory_mb = sum([
for service_name, service_config in local.application_services :
tonumber(split("Mi", service_config.memory_request)[0]) * service_config.replicas
])
total_instances = sum([
for service_name, service_config in local.application_services :
service_config.replicas
])
services_count = length(keys(local.application_services))
}
}
}
Collection functions in detail:
| Function | Purpose | Input type | Output type | Performance |
|---|---|---|---|---|
for |
Transformation | List/Map | List/Map | High |
filter |
Selection | List | List | High |
merge |
Combination | Maps | Map | Very high |
flatten |
Hierarchy flattening | List of lists | List | Medium |
slice |
Subset | List | List | Very high |
concat |
Concatenation | Lists | List | High |
🔧 Practical example - applying to resources:
# Dynamic subnet creation with collection functions
resource "aws_subnet" "main" {
for_each = {
for subnet in local.flattened_data.all_subnet_configurations :
subnet.name => subnet
}
vpc_id = aws_vpc.main.id
cidr_block = each.value.cidr_block
availability_zone = each.value.availability_zone
map_public_ip_on_launch = each.value.map_public_ip
tags = merge(
local.merged_configurations.final_tags,
{
Name = each.value.name
Tier = each.value.tier
AvailabilityZone = each.value.availability_zone
}
)
}
# Security group with dynamic rules
resource "aws_security_group" "services" {
name_prefix = "${var.project_name}-${var.environment}-services-"
vpc_id = aws_vpc.main.id
dynamic "ingress" {
for_each = {
for rule in local.flattened_data.all_security_group_rules :
"${rule.service}-${rule.from_port}" => rule
if rule.type == "ingress" && rule != null
}
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ["10.0.0.0/16"]
description = ingress.value.description
}
}
tags = local.merged_configurations.final_tags
}
Collection performance optimisations:
- Use
for_eachinstead ofcountfor better state management - Cache complex collection operations in locals
- Avoid deep nesting of
forexpressions - Use
slice()for large lists instead of iterating everything
Collection pitfalls:
| Problem | Symptom | Solution |
|---|---|---|
| Type inconsistency | Error: Invalid value type |
Explicit type conversion |
| Null values | Error: Invalid function argument |
Null checks with != null |
| Performance | Slow plan times | Limit collection size |
| Complexity | Unreadable for-expressions | Split into smaller operations |
Conditional expressions and dynamic blocks
What are conditional expressions? Conditional expressions enable if-else logic in HCL. They use ternary operator syntax (condition ? true_value : false_value) and make configurations environment-dependent and intelligent.
Why are conditional expressions essential? They remove the need for separate configuration files per environment. A single Terraform configuration can adapt dynamically to different scenarios. That reduces code duplication and makes infrastructure more maintainable.
What are dynamic blocks? Dynamic blocks generate repeating blocks based on data. They are like loops for Terraform resource configurations and make static, repetitive definitions unnecessary.
What must you watch for with conditional logic? Complex nested conditions can become unreadable. Performance can suffer with many dynamic blocks. Type consistency between true and false values matters. Debugging gets harder with complex conditional logic.
What do you use conditional expressions and dynamic blocks for? Environment-specific configurations, feature flags, resource counts, security policies, and anywhere you need flexible, data-driven infrastructure.
# Advanced conditional expressions and dynamic blocks
locals {
# ============================================================================
# BASE CONDITIONAL LOGIC
# ============================================================================
# Environment detection and classification
environment_classification = {
is_production = var.environment == "prod"
is_staging = var.environment == "staging"
is_development = var.environment == "dev"
is_non_production = var.environment != "prod"
# Advanced environment checks
requires_high_availability = contains(["prod", "staging"], var.environment)
allows_experimental_features = contains(["dev", "sandbox"], var.environment)
requires_compliance = var.environment == "prod"
supports_cost_optimization = var.environment != "prod"
}
# Time-based conditionals
time_based_conditions = {
current_hour = tonumber(formatdate("hh", timestamp()))
is_business_hours = local.time_based_conditions.current_hour >= 9 && local.time_based_conditions.current_hour <= 17
is_weekend = contains(["saturday", "sunday"], lower(formatdate("EEEE", timestamp())))
# Auto-shutdown logic for development
should_auto_shutdown = (
local.environment_classification.is_development &&
!local.time_based_conditions.is_business_hours
)
}
# ============================================================================
# ADVANCED CONDITIONAL CONFIGURATIONS
# ============================================================================
# Instance configuration with complex conditional logic
instance_configuration = {
# Instance type based on environment and workload
instance_type = (
local.environment_classification.is_production ? "t3.large" :
local.environment_classification.is_staging ? "t3.medium" :
"t3.micro"
)
# Instance count with business logic
instance_count = (
local.environment_classification.is_production ? 5 :
local.environment_classification.requires_high_availability ? 3 :
1
)
# Storage configuration
storage_configuration = {
volume_type = local.environment_classification.is_production ? "gp3" : "gp2"
volume_size = (
local.environment_classification.is_production ? 100 :
local.environment_classification.is_staging ? 50 :
20
)
iops = local.environment_classification.is_production ? 3000 : null
throughput = local.environment_classification.is_production ? 125 : null
encrypted = local.environment_classification.requires_compliance
}
# Monitoring configuration
monitoring = {
enabled = local.environment_classification.requires_high_availability
detailed_monitoring = local.environment_classification.is_production
log_level = (
local.environment_classification.is_production ? "ERROR" :
local.environment_classification.is_staging ? "WARN" :
"DEBUG"
)
retention_days = (
local.environment_classification.is_production ? 90 :
local.environment_classification.is_staging ? 30 :
7
)
}
}
# Database configuration with conditional logic
database_configuration = {
# Engine-specific configuration
engine_config = {
engine = "mysql"
version = local.environment_classification.is_production ? "8.0.35" : "8.0.28"
instance_class = (
local.environment_classification.is_production ? "db.r5.xlarge" :
local.environment_classification.is_staging ? "db.t3.medium" :
"db.t3.micro"
)
}
# Backup strategy
backup_strategy = {
backup_retention_period = (
local.environment_classification.is_production ? 30 :
local.environment_classification.is_staging ? 7 :
0
)
backup_window = local.environment_classification.requires_high_availability ? "03:00-04:00" : "06:00-07:00"
maintenance_window = local.environment_classification.requires_high_availability ? "sun:04:00-sun:05:00" : "sun:07:00-sun:08:00"
# Cross-region backup only for production
copy_tags_to_snapshot = local.environment_classification.is_production
delete_automated_backups = !local.environment_classification.requires_compliance
}
# Performance configuration
performance_config = {
multi_az = local.environment_classification.requires_high_availability
performance_insights_enabled = local.environment_classification.is_production
monitoring_interval = (
local.environment_classification.is_production ? 15 :
local.environment_classification.is_staging ? 60 :
0
)
}
}
# ============================================================================
# SECURITY CONFIGURATION WITH CONDITIONAL LOGIC
# ============================================================================
# Security policies based on environment
security_configuration = {
# Encryption requirements
encryption_requirements = {
storage_encrypted = local.environment_classification.requires_compliance
kms_key_rotation = local.environment_classification.is_production
backup_encryption = local.environment_classification.requires_compliance
log_encryption = local.environment_classification.is_production
}
# Network security
network_security = {
# SSH access only in development
allow_ssh_from_internet = local.environment_classification.is_development
# VPC Flow Logs for compliance
enable_vpc_flow_logs = local.environment_classification.requires_compliance
# Network ACLs for extra security
enable_network_acls = local.environment_classification.is_production
# Bastion host only when needed
deploy_bastion_host = (
local.environment_classification.requires_high_availability &&
!local.environment_classification.is_development
)
}
# Compliance features
compliance_features = {
enable_cloudtrail = local.environment_classification.requires_compliance
enable_config = local.environment_classification.requires_compliance
enable_guardduty = local.environment_classification.is_production
enable_security_hub = local.environment_classification.is_production
# Audit logging
audit_log_retention = (
local.environment_classification.requires_compliance ? 2555 : # 7 years
local.environment_classification.is_staging ? 365 : # 1 year
30 # 30 days
)
}
}
# ============================================================================
# DYNAMIC BLOCK DATA STRUCTURES
# ============================================================================
# Security group rules for different scenarios
security_group_rules = {
# Base rules for all environments
base_rules = [
{
type = "egress"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "All outbound traffic"
}
]
# Web traffic rules
web_rules = [
{
type = "ingress"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTP from internet"
},
{
type = "ingress"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "HTTPS from internet"
}
]
# Development-specific rules
development_rules = local.environment_classification.is_development ? [
{
type = "ingress"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "SSH for development"
},
{
type = "ingress"
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Development server"
}
] : []
# Monitoring rules for production
monitoring_rules = local.environment_classification.is_production ? [
{
type = "ingress"
from_port = 9090
to_port = 9090
protocol = "tcp"
cidr_blocks = ["10.0.0.0/16"]
description = "Prometheus metrics"
},
{
type = "ingress"
from_port = 3000
to_port = 3000
protocol = "tcp"
cidr_blocks = ["10.0.0.0/16"]
description = "Grafana dashboard"
}
] : []
# Combined rules
all_rules = concat(
local.security_group_rules.base_rules,
local.security_group_rules.web_rules,
local.security_group_rules.development_rules,
local.security_group_rules.monitoring_rules
)
}
# Auto Scaling policies for different scenarios
autoscaling_policies = {
# CPU-based scaling
cpu_policies = local.environment_classification.requires_high_availability ? [
{
name = "scale-up-cpu"
scaling_adjustment = 2
adjustment_type = "ChangeInCapacity"
cooldown = 300
metric_name = "CPUUtilization"
threshold = 70
comparison_operator = "GreaterThanThreshold"
evaluation_periods = 2
period = 120
},
{
name = "scale-down-cpu"
scaling_adjustment = -1
adjustment_type = "ChangeInCapacity"
cooldown = 300
metric_name = "CPUUtilization"
threshold = 30
comparison_operator = "LessThanThreshold"
evaluation_periods = 2
period = 120
}
] : []
# Memory-based scaling for production
memory_policies = local.environment_classification.is_production ? [
{
name = "scale-up-memory"
scaling_adjustment = 1
adjustment_type = "ChangeInCapacity"
cooldown = 600
metric_name = "MemoryUtilization"
threshold = 80
comparison_operator = "GreaterThanThreshold"
evaluation_periods = 3
period = 300
}
] : []
# Schedule-based scaling for cost optimisation
schedule_policies = local.environment_classification.supports_cost_optimization ? [
{
name = "scale-down-night"
min_size = 0
max_size = 1
desired_capacity = 0
recurrence = "0 22 * * MON-FRI" # 22:00 on weekdays
},
{
name = "scale-up-morning"
min_size = 1
max_size = 3
desired_capacity = 1
recurrence = "0 8 * * MON-FRI" # 08:00 on weekdays
}
] : []
# All policies combined
all_policies = concat(
local.autoscaling_policies.cpu_policies,
local.autoscaling_policies.memory_policies,
local.autoscaling_policies.schedule_policies
)
}
}
# Advanced resources with dynamic blocks and conditional logic
resource "aws_instance" "application" {
count = local.instance_configuration.instance_count
ami = data.aws_ami.ubuntu.id
instance_type = local.instance_configuration.instance_type
# Conditional subnet placement
subnet_id = var.subnet_ids[count.index % length(var.subnet_ids)]
# Conditional storage configuration
root_block_device {
volume_type = local.instance_configuration.storage_configuration.volume_type
volume_size = local.instance_configuration.storage_configuration.volume_size
encrypted = local.instance_configuration.storage_configuration.encrypted
# Conditional IOPS and throughput for gp3
iops = local.instance_configuration.storage_configuration.volume_type == "gp3" ? local.instance_configuration.storage_configuration.iops : null
throughput = local.instance_configuration.storage_configuration.volume_type == "gp3" ? local.instance_configuration.storage_configuration.throughput : null
}
# Conditional monitoring
monitoring = local.instance_configuration.monitoring.detailed_monitoring
# Dynamic user data based on environment
user_data = base64encode(templatefile("${path.module}/userdata.sh", {
environment = var.environment
instance_index = count.index
log_level = local.instance_configuration.monitoring.log_level
monitoring_enabled = local.instance_configuration.monitoring.enabled
is_production = local.environment_classification.is_production
}))
# Conditional auto-shutdown for development
dynamic "credit_specification" {
for_each = local.environment_classification.supports_cost_optimization && startswith(local.instance_configuration.instance_type, "t") ? [1] : []
content {
cpu_credits = "unlimited"
}
}
tags = merge(
var.base_tags,
{
Name = "${var.project_name}-${var.environment}-app-${count.index + 1}"
Environment = var.environment
InstanceIndex = count.index
AutoShutdown = local.time_based_conditions.should_auto_shutdown
}
)
lifecycle {
# Conditional lifecycle rules
create_before_destroy = local.environment_classification.requires_high_availability
ignore_changes = local.environment_classification.is_development ? ["ami"] : []
}
}
# Security group with dynamic blocks for flexible rules
resource "aws_security_group" "application" {
name_prefix = "${var.project_name}-${var.environment}-app-"
vpc_id = var.vpc_id
# Dynamic ingress rules
dynamic "ingress" {
for_each = {
for rule in local.security_group_rules.all_rules :
"${rule.type}-${rule.from_port}-${rule.to_port}" => rule
if rule.type == "ingress"
}
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ingress.value.cidr_blocks
description = ingress.value.description
}
}
# Dynamic egress rules
dynamic "egress" {
for_each = {
for rule in local.security_group_rules.all_rules :
"${rule.type}-${rule.from_port}-${rule.to_port}" => rule
if rule.type == "egress"
}
content {
from_port = egress.value.from_port
to_port = egress.value.to_port
protocol = egress.value.protocol
cidr_blocks = egress.value.cidr_blocks
description = egress.value.description
}
}
tags = {
Name = "${var.project_name}-${var.environment}-app-sg"
Environment = var.environment
RuleCount = length(local.security_group_rules.all_rules)
}
}
# Auto Scaling group with dynamic scaling policies
resource "aws_autoscaling_group" "application" {
count = local.environment_classification.requires_high_availability ? 1 : 0
name = "${var.project_name}-${var.environment}-asg"
vpc_zone_identifier = var.subnet_ids
min_size = local.instance_configuration.instance_count
max_size = local.instance_configuration.instance_count * 3
desired_capacity = local.instance_configuration.instance_count
launch_template {
id = aws_launch_template.application.id
version = "$Latest"
}
# Conditional health check
health_check_type = local.environment_classification.is_production ? "ELB" : "EC2"
health_check_grace_period = local.environment_classification.is_production ? 300 : 60
# Dynamic tags
dynamic "tag" {
for_each = merge(
var.base_tags,
{
Name = "${var.project_name}-${var.environment}-asg"
Environment = var.environment
AutoScaling = "enabled"
}
)
content {
key = tag.key
value = tag.value
propagate_at_launch = true
}
}
}
# CloudWatch alarms with dynamic blocks
resource "aws_cloudwatch_metric_alarm" "application_alarms" {
for_each = {
for policy in local.autoscaling_policies.all_policies :
policy.name => policy
if policy.metric_name != null
}
alarm_name = "${var.project_name}-${var.environment}-${each.value.name}"
comparison_operator = each.value.comparison_operator
evaluation_periods = each.value.evaluation_periods
metric_name = each.value.metric_name
namespace = "AWS/EC2"
period = each.value.period
statistic = "Average"
threshold = each.value.threshold
alarm_description = "Auto scaling alarm for ${each.value.name}"
# Conditional alarm actions
alarm_actions = local.environment_classification.requires_high_availability ? [
aws_autoscaling_policy.application_policies[each.key].arn
] : []
# Dynamic dimensions
dynamic "dimensions" {
for_each = local.environment_classification.requires_high_availability ? [1] : []
content {
AutoScalingGroupName = aws_autoscaling_group.application[0].name
}
}
tags = {
Name = "${var.project_name}-${var.environment}-${each.value.name}-alarm"
Environment = var.environment
MetricName = each.value.metric_name
}
}
Conditional logic flow diagram:
┌─────────────────────────────────────────────────────────────┐
│ Conditional Logic Flow │
├─────────────────────────────────────────────────────────────┤
│ │
│ Environment Input │
│ (var.environment) │
│ │ │
│ ▼ │
│ ┌─────────────────────────────┐ │
│ │ Environment Classification │ │
│ │ ─────────────────────────── │ │
│ │ - is_production │ │
│ │ - requires_ha │ │
│ │ - allows_experimental │ │
│ └──────────────┬──────────────┘ │
│ │ │
│ ┌───────────────┴───────────────┐ │
│ ▼ ▼ │
│ ┌───────────────────────┐ ┌───────────────────────┐ │
│ │ Instance Config │ │ Security Config │ │
│ │ ───────────────────── │ │ ───────────────────── │ │
│ │ - type: conditional │ │ - enc: conditional │ │
│ │ - count: conditional │ │ - mon: conditional │ │
│ │ - disk: conditional │ │ - comp: conditional │ │
│ └───────────┬───────────┘ └───────────┬───────────┘ │
│ │ │ │
│ ▼ │ │
│ ┌───────────────────────┐ │ │
│ │ Dynamic Blocks │ │ │
│ │ ───────────────────── │ │ │
│ │ - security_rules │ │ │
│ │ - scaling_policies │ │ │
│ │ - monitoring_targets │ │ │
│ └───────────┬───────────┘ │ │
│ │ │ │
│ └───────────────┬───────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────┐ │
│ │ Resource Configuration │ │
│ │ (Evaluated Final State) │ │
│ └─────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Conditional logic best practices:
| Pattern | Use | Performance | Maintainability |
|---|---|---|---|
| Ternary operator | Simple if-else | Very high | High |
| Complex conditions | Multi-criteria | High | Medium |
| Dynamic blocks | Repeating structure | Medium | High |
| for_each with conditions | Conditional resources | Medium | Very high |
Conditional logic optimisations:
- Use for_each instead of count for better state management
- Cache complex collection operations in locals
- Avoid deep nesting of for-expressions
- Use slice() for large lists instead of iterating everything
Conditional logic pitfalls:
| Problem | Symptom | Solution |
|---|---|---|
| Type mismatch | Error: Inconsistent conditional result types |
Same types in both branches |
| Null conditions | Error: Invalid value for conditional |
Add null checks |
| Performance | Slow evaluation | Cache conditions in locals |
| Debugging | Hard-to-follow logic | Use simpler conditions |
🔧 Practical example - debugging conditional logic:
# Test conditional logic in terraform console
terraform console
> local.environment_classification.is_production
> local.instance_configuration.instance_type
> local.security_group_rules.all_rules
# Inspect dynamic block contents
terraform plan | grep -A 10 "dynamic"
# Validate conditional resources
terraform state list | grep conditional_resource
For-expressions for complex transformations
What are for-expressions? For-expressions are one of the most powerful HCL features for data transformation. They combine the flexibility of loops with the expressiveness of conditional logic and let you build complex data structures in a single, elegant expression. For-expressions are like SQL queries for your Terraform configuration.
Why do for-expressions matter so much? They turn static Terraform configurations into dynamic, data-driven infrastructure definitions. Without them you would have to define repetitive resources by hand or use external scripts. With them you create flexible, scalable configurations that adapt automatically to changing data structures.
What must you watch for with for-expressions? Complex nested for-expressions can be hard to read and debug. Performance can suffer on large datasets. Type safety matters — make sure every iteration returns consistent data types. Missing null checks can cause runtime errors.
What do you use for-expressions for? Dynamic resource generation, data aggregation, complex transformations between data formats, filtering with business logic, and anywhere you must build one data structure from another.
Basic for-expression syntax
# Advanced for-expressions for complex data transformations
locals {
# ============================================================================
# BASE DATA FOR FOR-EXPRESSION EXAMPLES
# ============================================================================
# Complex input data
application_services = {
frontend = {
name = "frontend"
port = 3000
replicas = 3
cpu_limit = "500m"
memory_limit = "512Mi"
environment = "production"
health_check = "/health"
dependencies = ["backend", "cache"]
labels = {
tier = "presentation"
version = "v1.2.3"
team = "frontend-team"
}
volumes = [
{
name = "config"
path = "/app/config"
size = "1Gi"
},
{
name = "logs"
path = "/app/logs"
size = "5Gi"
}
]
}
backend = {
name = "backend"
port = 8080
replicas = 5
cpu_limit = "1000m"
memory_limit = "1Gi"
environment = "production"
health_check = "/api/health"
dependencies = ["database", "cache"]
labels = {
tier = "application"
version = "v2.1.0"
team = "backend-team"
}
volumes = [
{
name = "data"
path = "/app/data"
size = "10Gi"
}
]
}
worker = {
name = "worker"
port = 9000
replicas = 2
cpu_limit = "750m"
memory_limit = "768Mi"
environment = "production"
health_check = "/worker/health"
dependencies = ["database", "queue"]
labels = {
tier = "worker"
version = "v1.0.5"
team = "backend-team"
}
volumes = []
}
cache = {
name = "cache"
port = 6379
replicas = 1
cpu_limit = "250m"
memory_limit = "256Mi"
environment = "production"
health_check = "/ping"
dependencies = []
labels = {
tier = "cache"
version = "v6.2.0"
team = "platform-team"
}
volumes = [
{
name = "cache-data"
path = "/data"
size = "2Gi"
}
]
}
}
# Availability zones and regions
regions = {
"us-east-1" = {
azs = ["us-east-1a", "us-east-1b", "us-east-1c"]
vpc_cidr = "10.0.0.0/16"
instance_types = ["t3.micro", "t3.small", "t3.medium"]
}
"us-west-2" = {
azs = ["us-west-2a", "us-west-2b", "us-west-2c"]
vpc_cidr = "10.1.0.0/16"
instance_types = ["t3.small", "t3.medium", "t3.large"]
}
"eu-west-1" = {
azs = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
vpc_cidr = "10.2.0.0/16"
instance_types = ["t3.medium", "t3.large", "t3.xlarge"]
}
}
# Environment configurations
environments = {
dev = {
replica_multiplier = 0.5
cpu_multiplier = 0.5
memory_multiplier = 0.5
storage_multiplier = 0.5
monitoring_enabled = false
backup_enabled = false
}
staging = {
replica_multiplier = 0.8
cpu_multiplier = 0.8
memory_multiplier = 0.8
storage_multiplier = 0.8
monitoring_enabled = true
backup_enabled = true
}
prod = {
replica_multiplier = 1.0
cpu_multiplier = 1.0
memory_multiplier = 1.0
storage_multiplier = 1.0
monitoring_enabled = true
backup_enabled = true
}
}
# ============================================================================
# BASIC FOR-EXPRESSION PATTERNS
# ============================================================================
# Simple list comprehension
basic_list_transformations = {
# Extract service names
service_names = [
for service_name, config in local.application_services :
service_name
]
# Extract ports
service_ports = [
for service_name, config in local.application_services :
config.port
]
# Service names in uppercase
service_names_upper = [
for service_name, config in local.application_services :
upper(service_name)
]
# Combined values
service_endpoints = [
for service_name, config in local.application_services :
"${service_name}:${config.port}"
]
}
# Simple object comprehension
basic_object_transformations = {
# Port mapping
port_mapping = {
for service_name, config in local.application_services :
service_name => config.port
}
# Health-check mapping
health_check_mapping = {
for service_name, config in local.application_services :
service_name => config.health_check
}
# Replica mapping
replica_mapping = {
for service_name, config in local.application_services :
service_name => config.replicas
}
# Label-tier mapping
tier_mapping = {
for service_name, config in local.application_services :
service_name => config.labels.tier
}
}
# ============================================================================
# CONDITIONAL FOR-EXPRESSIONS
# ============================================================================
# For-expressions with conditional logic
conditional_transformations = {
# Web services only (frontend/backend)
web_services = {
for service_name, config in local.application_services :
service_name => config
if contains(["frontend", "backend"], service_name)
}
# Services with high memory demand
memory_intensive_services = [
for service_name, config in local.application_services :
service_name
if tonumber(split("Mi", config.memory_limit)[0]) > 500
]
# Services with dependencies
services_with_dependencies = {
for service_name, config in local.application_services :
service_name => config.dependencies
if length(config.dependencies) > 0
}
# Services with volumes
services_with_storage = {
for service_name, config in local.application_services :
service_name => {
volumes = config.volumes
total_storage = sum([
for volume in config.volumes :
tonumber(split("Gi", volume.size)[0])
])
}
if length(config.volumes) > 0
}
# Team-based grouping
backend_team_services = [
for service_name, config in local.application_services :
service_name
if config.labels.team == "backend-team"
]
# Environment-specific configuration
production_ready_services = {
for service_name, config in local.application_services :
service_name => merge(config, {
# Production-specific adjustments
replicas = max(2, config.replicas) # Minimum 2 replicas
monitoring_enabled = true
backup_enabled = true
})
if config.environment == "production"
}
}
# ============================================================================
# NESTED FOR-EXPRESSIONS
# ============================================================================
# Complex nested transformations
nested_transformations = {
# All service-volume combinations
all_service_volumes = flatten([
for service_name, config in local.application_services : [
for volume in config.volumes : {
service = service_name
volume_name = volume.name
volume_path = volume.path
volume_size = volume.size
full_name = "${service_name}-${volume.name}"
}
]
])
# Service dependency graph
service_dependency_graph = {
for service_name, config in local.application_services :
service_name => {
depends_on = config.dependencies
dependents = [
for other_service, other_config in local.application_services :
other_service
if contains(other_config.dependencies, service_name)
]
total_connections = length(config.dependencies) + length([
for other_service, other_config in local.application_services :
other_service
if contains(other_config.dependencies, service_name)
])
}
}
# Multi-region service deployment
multi_region_deployments = {
for region_name, region_config in local.regions :
region_name => {
region = region_name
vpc_cidr = region_config.vpc_cidr
services = {
for service_name, service_config in local.application_services :
service_name => {
service = service_name
region = region_name
subnets = [
for i, az in region_config.azs :
{
name = "${service_name}-${region_name}-${i + 1}"
az = az
cidr = cidrsubnet(region_config.vpc_cidr, 8, i)
instance_type = region_config.instance_types[i % length(region_config.instance_types)]
}
]
load_balancer_targets = [
for i, az in region_config.azs :
{
target_id = "${service_name}-${region_name}-${i + 1}"
az = az
port = service_config.port
health_check = service_config.health_check
}
]
}
}
}
}
# Environment-service matrix
environment_service_matrix = {
for env_name, env_config in local.environments :
env_name => {
environment = env_name
services = {
for service_name, service_config in local.application_services :
service_name => {
name = service_name
environment = env_name
replicas = max(1, floor(service_config.replicas * env_config.replica_multiplier))
cpu_limit = "${floor(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier)}m"
memory_limit = "${floor(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier)}Mi"
monitoring_enabled = env_config.monitoring_enabled
backup_enabled = env_config.backup_enabled
volumes = [
for volume in service_config.volumes :
{
name = volume.name
path = volume.path
size = "${floor(tonumber(split("Gi", volume.size)[0]) * env_config.storage_multiplier)}Gi"
}
]
full_name = "${service_name}-${env_name}"
}
}
total_replicas = sum([
for service_name, service_config in local.application_services :
max(1, floor(service_config.replicas * env_config.replica_multiplier))
])
total_cpu = sum([
for service_name, service_config in local.application_services :
floor(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier)
])
total_memory = sum([
for service_name, service_config in local.application_services :
floor(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier)
])
}
}
}
# ============================================================================
# ADVANCED FOR-EXPRESSION PATTERNS
# ============================================================================
# Complex data aggregation
advanced_aggregations = {
# Team-based statistics
team_statistics = {
for team in distinct([
for service_name, config in local.application_services :
config.labels.team
]) :
team => {
team_name = team
services = [
for service_name, config in local.application_services :
service_name
if config.labels.team == team
]
total_replicas = sum([
for service_name, config in local.application_services :
config.replicas
if config.labels.team == team
])
total_cpu = sum([
for service_name, config in local.application_services :
tonumber(split("m", config.cpu_limit)[0])
if config.labels.team == team
])
total_memory = sum([
for service_name, config in local.application_services :
tonumber(split("Mi", config.memory_limit)[0])
if config.labels.team == team
])
service_count = length([
for service_name, config in local.application_services :
service_name
if config.labels.team == team
])
}
}
# Tier-based configuration
tier_configurations = {
for tier in distinct([
for service_name, config in local.application_services :
config.labels.tier
]) :
tier => {
tier_name = tier
services = {
for service_name, config in local.application_services :
service_name => {
port = config.port
replicas = config.replicas
health_check = config.health_check
dependencies = config.dependencies
resource_requirements = {
cpu = config.cpu_limit
memory = config.memory_limit
}
}
if config.labels.tier == tier
}
load_balancer_config = {
enabled = contains(["presentation", "application"], tier)
port = tier == "presentation" ? 80 : 8080
health_check_path = "/health"
targets = [
for service_name, config in local.application_services :
{
service = service_name
port = config.port
health_check = config.health_check
}
if config.labels.tier == tier
]
}
security_group_rules = [
for service_name, config in local.application_services : {
service = service_name
port = config.port
tier = tier
rule = {
type = "ingress"
from_port = config.port
to_port = config.port
protocol = "tcp"
description = "Allow traffic to ${service_name} in ${tier} tier"
source_tier = tier == "presentation" ? "internet" : "application"
}
}
if config.labels.tier == tier
]
}
}
# Dependency-chain analysis
dependency_chains = {
for service_name, config in local.application_services :
service_name => {
service = service_name
direct_dependencies = config.dependencies
indirect_dependencies = flatten([
for dep in config.dependencies : [
for indirect_dep in lookup(local.application_services, dep, {}).dependencies :
indirect_dep
if !contains(config.dependencies, indirect_dep)
]
])
all_dependencies = distinct(concat(
config.dependencies,
flatten([
for dep in config.dependencies : [
for indirect_dep in lookup(local.application_services, dep, {}).dependencies :
indirect_dep
if !contains(config.dependencies, indirect_dep)
]
])
))
dependency_depth = length(distinct(concat(
config.dependencies,
flatten([
for dep in config.dependencies : [
for indirect_dep in lookup(local.application_services, dep, {}).dependencies :
indirect_dep
if !contains(config.dependencies, indirect_dep)
]
])
)))
is_leaf_service = length(config.dependencies) == 0
is_root_service = length([
for other_service, other_config in local.application_services :
other_service
if contains(other_config.dependencies, service_name)
]) == 0
}
}
}
# ============================================================================
# PERFORMANCE-OPTIMISED FOR-EXPRESSIONS
# ============================================================================
# Optimised transformations for large datasets
optimized_transformations = {
# Service lookup table (fast access)
service_lookup = {
for service_name, config in local.application_services :
service_name => {
port = config.port
health_check = config.health_check
tier = config.labels.tier
team = config.labels.team
}
}
# Port index for fast search
port_index = {
for service_name, config in local.application_services :
config.port => service_name
}
# Team index for grouped operations
team_index = {
for team in distinct([
for service_name, config in local.application_services :
config.labels.team
]) :
team => [
for service_name, config in local.application_services :
service_name
if config.labels.team == team
]
}
# Tier index for load balancer configuration
tier_index = {
for tier in distinct([
for service_name, config in local.application_services :
config.labels.tier
]) :
tier => {
services = [
for service_name, config in local.application_services :
service_name
if config.labels.tier == tier
]
total_replicas = sum([
for service_name, config in local.application_services :
config.replicas
if config.labels.tier == tier
])
needs_load_balancer = contains(["presentation", "application"], tier)
}
}
# Cached computed values
resource_summary = {
total_services = length(keys(local.application_services))
total_replicas = sum([
for service_name, config in local.application_services :
config.replicas
])
total_cpu_millicores = sum([
for service_name, config in local.application_services :
tonumber(split("m", config.cpu_limit)[0]) * config.replicas
])
total_memory_mi = sum([
for service_name, config in local.application_services :
tonumber(split("Mi", config.memory_limit)[0]) * config.replicas
])
total_storage_gi = sum([
for service_name, config in local.application_services :
sum([
for volume in config.volumes :
tonumber(split("Gi", volume.size)[0])
])
])
unique_teams = length(distinct([
for service_name, config in local.application_services :
config.labels.team
]))
unique_tiers = length(distinct([
for service_name, config in local.application_services :
config.labels.tier
]))
}
}
}
For-expression syntax variants
| Syntax | Purpose | Example | Output type |
|---|---|---|---|
[for item in list : expression] |
List comprehension | [for s in services : s.name] |
List |
{for key, value in map : key => expression} |
Object comprehension | {for k, v in map : k => v.port} |
Map |
[for item in list : expression if condition] |
Conditional list | [for s in services : s if s.enabled] |
List |
{for key, value in map : key => expression if condition} |
Conditional object | {for k, v in map : k => v if v.active} |
Map |
Practical use in real resources
# Dynamic Kubernetes deployments with for-expressions
resource "kubernetes_deployment" "services" {
for_each = local.conditional_transformations.web_services
metadata {
name = each.key
labels = merge(
each.value.labels,
{
managed-by = "terraform"
environment = var.environment
}
)
}
spec {
replicas = each.value.replicas
selector {
match_labels = {
app = each.key
}
}
template {
metadata {
labels = merge(
each.value.labels,
{
app = each.key
}
)
}
spec {
container {
name = each.key
image = "${each.key}:${each.value.labels.version}"
port {
container_port = each.value.port
}
# Dynamic environment variables
dynamic "env" {
for_each = {
for key, value in merge(
{
SERVICE_NAME = each.key
SERVICE_PORT = tostring(each.value.port)
ENVIRONMENT = var.environment
},
# Service-specific environment variables
lookup(local.service_environment_variables, each.key, {})
) :
key => value
}
content {
name = env.key
value = env.value
}
}
# Dynamic volume mounts
dynamic "volume_mount" {
for_each = {
for volume in each.value.volumes :
volume.name => volume
}
content {
name = volume_mount.key
mount_path = volume_mount.value.path
}
}
# Health check
liveness_probe {
http_get {
path = each.value.health_check
port = each.value.port
}
initial_delay_seconds = 30
period_seconds = 10
}
readiness_probe {
http_get {
path = each.value.health_check
port = each.value.port
}
initial_delay_seconds = 5
period_seconds = 5
}
resources {
limits = {
cpu = each.value.cpu_limit
memory = each.value.memory_limit
}
requests = {
cpu = "${floor(tonumber(split("m", each.value.cpu_limit)[0]) * 0.5)}m"
memory = "${floor(tonumber(split("Mi", each.value.memory_limit)[0]) * 0.5)}Mi"
}
}
}
# Dynamic volumes
dynamic "volume" {
for_each = {
for volume in each.value.volumes :
volume.name => volume
}
content {
name = volume.key
persistent_volume_claim {
claim_name = "${each.key}-${volume.key}-pvc"
}
}
}
}
}
}
}
# Load balancer services with for-expressions
resource "kubernetes_service" "load_balancers" {
for_each = local.nested_transformations.multi_region_deployments["us-east-1"].services
metadata {
name = "${each.key}-lb"
labels = {
service = each.key
type = "load-balancer"
}
}
spec {
selector = {
app = each.key
}
# Dynamic ports
dynamic "port" {
for_each = {
for target in each.value.load_balancer_targets :
target.target_id => target
}
content {
name = "http"
port = 80
target_port = port.value.port
protocol = "TCP"
}
}
type = "LoadBalancer"
}
}
# Persistent volume claims with for-expressions
resource "kubernetes_persistent_volume_claim" "service_storage" {
for_each = {
for volume_config in local.nested_transformations.all_service_volumes :
volume_config.full_name => volume_config
}
metadata {
name = "${each.key}-pvc"
labels = {
service = each.value.service
volume = each.value.volume_name
}
}
spec {
access_modes = ["ReadWriteOnce"]
resources {
requests = {
storage = each.value.volume_size
}
}
storage_class_name = "gp2"
}
}
For-expression performance optimisation:
Data preparation
- Use small datasets
- Remove unnecessary fields
- Indexing for lookups
- Caching in locals
Expression optimisation
- Prefer simple expressions
- Minimise nesting
- Apply conditional logic early
- Flatten only when needed
Iteration efficiency
- for_each > count
- Distinct() for duplicates
- Slice() for subsets
- Lookup() for access
Memory management
- Avoid large lists
- Use streaming patterns
- Watch garbage collection
- Minimise state size
For-expression debugging strategies:
| Debugging technique | Use | Example |
|---|---|---|
| Step-by-step build | Split complex expressions | One transformation per step |
| Terraform console | Interactive tests | terraform console |
| Print debugging | Outputs for intermediate results | output "debug" { value = local.test } |
| Type validation | Check consistency | can() for type checks |
For-expression best practices:
- Use meaningful variable names in iterations
- Keep expressions as simple as possible
- Cache complex calculations in locals
- Use conditional logic for performance optimisation
- Document complex transformations in detail
For-expression pitfalls:
| Problem | Symptom | Solution |
|---|---|---|
| Circular dependencies | Error: Cycle in values |
Restructure dependencies |
| Type inconsistency | Error: Inconsistent types |
Explicit type conversion |
| Performance problems | Slow plan times | Optimise expressions |
| Memory exhaustion | Error: Out of memory |
Shrink datasets |
| Complex nested loops | Unreadable expressions | Split into several steps |
For-expression debugging techniques
# Test for-expression results in terraform console
terraform console
> [for s in local.application_services : s.name]
> {for k, v in local.application_services : k => v.port}
> local.nested_transformations.all_service_volumes
# Step-by-step debugging
terraform console
> local.application_services
> [for s in local.application_services : s]
> [for s in local.application_services : s.name]
> [for s in local.application_services : s.name if s.replicas > 2]
# Performance analysis
terraform plan -detailed-exitcode
time terraform plan
Advanced for-expression patterns
# Advanced patterns for production code
locals {
# Pattern 1: flatten hierarchical data
flattened_configuration = flatten([
for region_name, region in local.regions : [
for service_name, service in local.application_services : {
key = "${region_name}-${service_name}"
region = region_name
service = service_name
az_count = length(region.azs)
vpc_cidr = region.vpc_cidr
service_port = service.port
service_replicas = service.replicas
full_config = merge(service, {
region = region_name
azs = region.azs
})
}
]
])
# Pattern 2: lookup tables for performance
service_by_port = {
for service_name, config in local.application_services :
config.port => service_name
}
# Pattern 3: conditional aggregation
resource_allocation = {
for env_name, env_config in local.environments :
env_name => {
services = {
for service_name, service_config in local.application_services :
service_name => {
replicas = max(1, ceil(service_config.replicas * env_config.replica_multiplier))
cpu_total = ceil(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
memory_total = ceil(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
}
}
totals = {
cpu = sum([
for service_name, service_config in local.application_services :
ceil(tonumber(split("m", service_config.cpu_limit)[0]) * env_config.cpu_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
])
memory = sum([
for service_name, service_config in local.application_services :
ceil(tonumber(split("Mi", service_config.memory_limit)[0]) * env_config.memory_multiplier * max(1, ceil(service_config.replicas * env_config.replica_multiplier)))
])
replicas = sum([
for service_name, service_config in local.application_services :
max(1, ceil(service_config.replicas * env_config.replica_multiplier))
])
}
}
}
# Pattern 4: error handling with can()
safe_transformations = {
for service_name, config in local.application_services :
service_name => {
cpu_millicores = can(tonumber(split("m", config.cpu_limit)[0])) ? tonumber(split("m", config.cpu_limit)[0]) : 0
memory_mi = can(tonumber(split("Mi", config.memory_limit)[0])) ? tonumber(split("Mi", config.memory_limit)[0]) : 0
has_health_check = can(config.health_check) && config.health_check != null
volume_count = can(length(config.volumes)) ? length(config.volumes) : 0
has_dependencies = can(length(config.dependencies)) ? length(config.dependencies) > 0 : false
}
}
}
For-expressions are the most powerful tool for data transformation in Terraform. They let you turn static configurations into dynamic, data-driven infrastructure. With list comprehensions, object comprehensions, conditional logic and nested loops you implement complex transformations in elegant, readable expressions. That command of for-expressions is what makes HCL configurations adapt automatically to changing requirements.
Command Reference (Cheatsheet)
For quick access while developing complex Terraform configurations, the following table summarises the most important HCL built-in functions, data transformations and expression syntax patterns:
| Function / syntax | Category | Description and practical use |
|---|---|---|
lookup(map, key, default) |
Map function | Reads a key from a map and returns a default if it is missing. |
merge(map1, map2, ...) |
Collection | Merges several maps; later values overwrite earlier keys. |
flatten(list_of_lists) |
Collection | Reduces multi-dimensional nested lists to a flat one-dimensional list. |
keys(map) / values(map) |
Map function | Returns all keys or all values of a map as a list. |
element(list, index) |
List function | Returns the element at an index with automatic wrap-around (modulo). |
slice(list, start, end) |
List function | Extracts a sublist from the start index to the end index. |
concat(list1, list2, ...) |
List function | Joins two or more lists into a single list. |
distinct(list) |
List function | Removes all duplicates from a list and keeps the original order. |
compact(list_of_strings) |
List function | Removes all empty strings "" from a list of strings. |
contains(list, value) |
Collection | Checks whether a given value exists in a list or set (true/false). |
try(expr1, expr2, ...) |
Error handling | Evaluates expressions in order and returns the first error-free value. |
can(expression) |
Error handling | Checks whether an expression can be evaluated without error (ideal for validations). |
coalesce(v1, v2, ...) |
Conditional | Returns the first value that is neither null nor an empty string "". |
templatefile(path, vars) |
String function | Renders an external template file and substitutes HCL variables. |
jsonencode(value) |
Encoding | Serialises arbitrary HCL data structures (objects, lists) into valid JSON. |
jsondecode(string) |
Encoding | Parses a JSON string into native HCL data structures. |
yamldecode(string) |
Encoding | Parses YAML documents directly into HCL maps or lists. |
sensitive(value) |
Security | Marks a value as sensitive so it is masked in the plan and in CLI logs. |
nonsensitive(value) |
Security | Removes the sensitive flag for non-critical follow-on operations. |
[for x in list : f(x)] |
For-expression | List comprehension: transforms every element of a list into a new format. |
{for k, v in map : k => f(v)} |
For-expression | Map comprehension: builds a new map with modified keys or values. |
[for x in list : x if condition] |
For-expression | Filters elements based on a boolean condition. |
dynamic "block" { for_each = ... } |
Dynamic block | Generates repeated resource blocks dynamically from collections. |
Further Resources
The following official documentation, registry directories and best-practice guides go deeper into the advanced HCL concepts covered above:
| Resource | Description |
|---|---|
| Terraform Configuration Language | Official language specification for HCL syntax, expressions and the type system. |
| Terraform Built-in Functions Reference | Complete reference of all mathematical, string and collection functions. |
| Terraform Developer Hub | Central place for HashiCorp documentation, tutorials and best practices. |
| Terraform CLI Documentation | Command reference for the Terraform CLI and debugging options. |
| Terraform Registry | Public directory for official and verified providers and modules. |
| AWS Provider Documentation | Official documentation of all AWS resources, attributes and IAM roles. |
| Azure Provider Documentation | Specification of the Microsoft Azure Resource Manager (azurerm) provider. |
| Google Cloud Terraform Documentation | Guide for Google Cloud Platform infrastructure with HashiCorp Terraform. |
💡 Tip: Start with the official documentation and the free resources. Once you have practical experience, deeper books and architecture blueprints are a worthwhile investment for architectural understanding.
Conclusion
With advanced HCL syntax in place you have the foundation for professional Terraform development. Complex data structures, intelligent validations, secure credential management and powerful built-in functions turn infrastructure definitions from static scripts into dynamic, self-adapting systems.
That language expertise is the prerequisite for every further Terraform challenge. You now write maintainable, reusable code that adapts cleanly to different environments and stays readable.
💡 Coming next: Part three of the series covers Terraform state management — the core of every professional Terraform implementation. Remote state, locking mechanisms, team workflows and the practices that make infrastructure scalable and team-ready.
👉 Overview: All DevOps articles and guides