A module interface is an API. The difference between a module your platform team trusts and one they fork is almost never the resources inside it; it’s whether the inputs are typed, invalid combinations are rejected at plan time, and optional knobs default sanely instead of forcing every caller to specify everything. Terraform’s type system, dynamic blocks, and the validation/precondition/postcondition family let you build that interface entirely in HCL. They also give you enough rope to write code nobody can read. This is a guide to using the sharp tools well and knowing when to put them down.
I’ll assume Terraform 1.9 or later. optional() defaults landed stable in 1.3, precondition/postcondition in 1.2, and cross-object input validation in 1.9, so version pinning matters for several of these patterns.
1. Design module inputs with object types and optional() defaults
The weakest module interface is a pile of scalar variables: bucket_name, bucket_versioning, bucket_lifecycle_days, bucket_logging_target. Callers can’t tell which scalars belong together, and you can’t express “if logging is on, a target is required.” Model the input as a typed object instead, and use optional() to give attributes defaults so callers only specify what they need.
variable "buckets" {
description = "Map of logical name => bucket configuration."
type = map(object({
name = string
versioning = optional(bool, false)
force_destroy = optional(bool, false)
storage_class = optional(string, "STANDARD")
lifecycle_rules = optional(list(object({
id = string
prefix = optional(string, "")
expiration_days = optional(number)
transition_to_ia_days = optional(number)
noncurrent_expiration = optional(number)
})), [])
tags = optional(map(string), {})
}))
default = {}
}
Two properties of optional() matter. First, the second argument is a default that Terraform fills in during type conversion before your code sees the value, so inside the module you write each.value.versioning (guaranteed non-null) instead of lookup(each.value, "versioning", false). Second, defaults apply at the level they’re declared: a caller who omits lifecycle_rules gets [], and a caller who supplies one rule with only id and expiration_days gets prefix = "" filled in automatically.
A caller’s input collapses to the essentials:
module "data_lake" {
source = "./modules/s3-buckets"
buckets = {
raw = {
name = "acme-data-lake-raw"
versioning = true
lifecycle_rules = [
{ id = "expire-tmp", prefix = "tmp/", expiration_days = 7 },
]
}
curated = {
name = "acme-data-lake-curated"
# everything else defaulted
}
}
}
A typed object that rejects unknown attributes is a feature, not a limitation. If a caller misspells
versionning, conversion fails with a clear error instead of silently ignoring the key the way an untypedmap(any)would.
2. Generate nested blocks with dynamic and the for_each iterator
Once lifecycle_rules is a list, you can’t write a fixed number of nested blocks. dynamic generates them. It takes the block label as its own label, a for_each collection, and a content {} template that runs once per element. Inside content, the iterator object exposes .key and .value.
resource "aws_s3_bucket" "this" {
for_each = var.buckets
bucket = each.value.name
force_destroy = each.value.force_destroy
tags = each.value.tags
}
resource "aws_s3_bucket_lifecycle_configuration" "this" {
# Only create the config resource for buckets that actually have rules.
for_each = { for k, v in var.buckets : k => v if length(v.lifecycle_rules) > 0 }
bucket = aws_s3_bucket.this[each.key].id
dynamic "rule" {
for_each = each.value.lifecycle_rules
content {
id = rule.value.id
status = "Enabled"
filter {
prefix = rule.value.prefix
}
# Nested dynamic: emit expiration only when a day count is set.
dynamic "expiration" {
for_each = rule.value.expiration_days != null ? [rule.value.expiration_days] : []
content {
days = expiration.value
}
}
dynamic "transition" {
for_each = rule.value.transition_to_ia_days != null ? [rule.value.transition_to_ia_days] : []
content {
days = transition.value
storage_class = "STANDARD_IA"
}
}
}
}
}
The iterator is named after the block by default (rule, expiration, transition). When a nested dynamic label collides with an outer one or the auto-derived name reads badly, rename it explicitly with iterator:
dynamic "setting" {
for_each = var.app_settings
iterator = cfg
content {
namespace = cfg.value.namespace
name = cfg.key
value = cfg.value.value
}
}
The single most important idiom here is the conditional single-element list: for_each = condition ? [value] : []. A dynamic block over an empty list produces zero blocks; over a one-element list, exactly one. That is how you include an optional sub-block only when its data is present, without a separate resource. Reach for dynamic only when block count or content genuinely varies by input; if a block is always present, write it literally.
3. Conditional blocks, null handling, and the merge/coalesce toolkit
Optional inputs mean nulls, and nulls behave differently across HCL functions in ways that bite. Keep three rules straight:
coalesce(a, b, c)returns the first non-null and non-empty argument. It errors if every argument is null or empty, so it’s for picking a value, not for defaulting to empty.coalescelist(a, b)is the list analogue and returns the first non-empty list.try(expr, fallback)swallows errors (including indexing into something that doesn’t exist), not nulls.try(local.x.y, "default")is the safe way to reach into a structure that may not have attributey.
A common mistake is using coalesce to default to an empty string or map; it throws because the empty value is treated as absent. Use a conditional or try for that case.
locals {
# Merge precedence: caller tags win over module defaults.
common_tags = {
ManagedBy = "terraform"
Module = "s3-buckets"
Environment = var.environment
}
# coalesce picks the first real value; never feed it only-empty args.
resolved_kms_key = coalesce(var.kms_key_arn, var.default_kms_key_arn)
}
resource "aws_s3_bucket" "this" {
for_each = var.buckets
bucket = each.value.name
# Later keys override earlier ones in merge().
tags = merge(local.common_tags, each.value.tags)
}
merge() is right-biased: later maps win on key collisions, which is the precedence you want when layering caller overrides on top of module defaults. For conditionally including whole attributes, merge an empty map:
locals {
encryption_block = var.kms_key_arn != null ? {
sse_algorithm = "aws:kms"
kms_master_key_id = var.kms_key_arn
} : {
sse_algorithm = "AES256"
}
}
4. Variable validation with multiple validation blocks and regex
A single variable can carry as many validation blocks as you have rules. Terraform evaluates all of them and surfaces every failure, so write one rule per failure mode with a specific error_message. The condition must reference the variable and must return true when valid.
variable "environment" {
type = string
description = "Deployment environment slug."
validation {
condition = contains(["dev", "staging", "prod"], var.environment)
error_message = "environment must be one of: dev, staging, prod."
}
}
variable "bucket_prefix" {
type = string
description = "DNS-compatible prefix for bucket names."
validation {
condition = can(regex("^[a-z0-9][a-z0-9-]{1,40}[a-z0-9]$", var.bucket_prefix))
error_message = "bucket_prefix must be 3-42 lowercase alphanumerics or hyphens, not starting/ending with a hyphen."
}
validation {
condition = !strcontains(var.bucket_prefix, "--")
error_message = "bucket_prefix must not contain consecutive hyphens."
}
}
can(regex(...)) is the workhorse: regex() throws when there’s no match, and can() converts that throw into false. Two accuracy notes. Use the ^...$ anchors deliberately, because regex matches anywhere in the string otherwise and "ab_CD" would pass a pattern meant to forbid underscores. And remember that as of Terraform 1.9, a validation condition may reference other variables and data sources, not just var.<self>, which lets you express cross-field rules at the input layer:
variable "replica_count" {
type = number
default = 1
}
variable "multi_az" {
type = bool
default = false
validation {
# Cross-variable rule: multi-AZ is meaningless with a single replica.
condition = var.multi_az == false || var.replica_count >= 2
error_message = "multi_az requires replica_count >= 2."
}
}
Keep validation for things knowable from input alone: formats, enums, ranges, mutually exclusive flags. Anything that depends on a computed attribute or a remote lookup belongs in a precondition.
5. Precondition and postcondition lifecycle checks on resources and outputs
validation runs against raw input. precondition and postcondition live in a lifecycle block (or in check blocks and output blocks) and run during plan/apply, so they can reference computed values, other resources, and data sources. A precondition asserts an invariant must hold before Terraform acts on the resource; a postcondition asserts something about the result.
data "aws_caller_identity" "current" {}
resource "aws_s3_bucket" "this" {
for_each = var.buckets
bucket = each.value.name
lifecycle {
# Catch a foot-gun before the destroy is planned, not after.
precondition {
condition = each.value.force_destroy == false || var.environment != "prod"
error_message = "force_destroy must not be enabled for buckets in prod (${each.value.name})."
}
}
}
resource "aws_s3_bucket_public_access_block" "this" {
for_each = aws_s3_bucket.this
bucket = each.value.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
lifecycle {
postcondition {
condition = self.block_public_policy && self.restrict_public_buckets
error_message = "Public access block did not fully apply; refusing to proceed."
}
}
}
self is only available inside postcondition and refers to the resource the block is attached to. Postconditions on outputs are an underused guardrail: they let a module assert its own contract before exporting a value, so a downstream module never receives garbage.
output "bucket_arns" {
description = "ARNs of all managed buckets, keyed by logical name."
value = { for k, b in aws_s3_bucket.this : k => b.arn }
precondition {
condition = length(aws_s3_bucket.this) == length(var.buckets)
error_message = "Bucket count mismatch; some buckets failed to materialize."
}
}
For invariants that aren’t tied to a single resource’s lifecycle, use a standalone check block. Unlike preconditions, a failing check assertion emits a warning and does not block apply, which is the right severity for advisory drift signals and post-deploy health probes.
check "tls_minimum" {
data "http" "endpoint" {
url = "https://${var.public_endpoint}/healthz"
}
assert {
condition = data.http.endpoint.status_code == 200
error_message = "Health endpoint returned ${data.http.endpoint.status_code}."
}
}
6. Transform data with for expressions, flatten, and setproduct
Module internals frequently need to reshape a nested input into a flat collection suitable for for_each. Two functions do most of the heavy lifting: flatten collapses one level of nesting, and setproduct builds the Cartesian product of two or more sets. The canonical pattern is a nested for that emits objects, wrapped in flatten, then re-keyed into a map.
locals {
# Input: map of bucket => list of CORS origins.
# Goal: one flat list of {bucket, origin} pairs for a per-rule resource.
bucket_cors = flatten([
for bucket_key, cfg in var.buckets : [
for origin in cfg.cors_origins : {
bucket = bucket_key
origin = origin
}
]
])
# Re-key with a STABLE composite key so addressing is deterministic.
bucket_cors_map = {
for pair in local.bucket_cors :
"${pair.bucket}:${pair.origin}" => pair
}
}
setproduct is the tool when you genuinely want every combination, for example expanding regions x environments:
locals {
regions = ["us-east-1", "eu-west-1"]
environments = ["dev", "prod"]
# Returns a list of [region, env] tuples.
deployments = {
for pair in setproduct(local.regions, local.environments) :
"${pair[0]}-${pair[1]}" => {
region = pair[0]
environment = pair[1]
}
}
}
The non-obvious rule: the value of a composite key must be unique, and the key string you build must be stable across runs. Building keys from list indices ("${idx}") is the classic trap; if the input list reorders, every key shifts and Terraform plans a destroy/create storm. Always key from intrinsic identity (names, IDs), never position.
7. Avoid the count vs for_each addressing trap on collections
This is the single most expensive HCL mistake in production, so it gets its own step. count indexes resources by integer position: aws_s3_bucket.this[0], [1], [2]. for_each indexes by map key: aws_s3_bucket.this["raw"], ["curated"]. When you manage a collection of similar things, the addressing scheme decides what happens when the collection changes.
Consider three buckets managed with count = length(var.bucket_names) over ["raw", "curated", "archive"]. Remove "curated" from the middle. The list becomes ["raw", "archive"], so index [1] flips from curated to archive and index [2] disappears. Terraform reads that as: modify [1] (rename curated -> archive in place, which for an S3 bucket means destroy and recreate) and destroy [2]. You lost the archive bucket to delete the curated one.
# FRAGILE: positional addressing on a mutable collection.
resource "aws_s3_bucket" "bad" {
count = length(var.bucket_names)
bucket = var.bucket_names[count.index]
}
# CORRECT: stable, key-based addressing.
resource "aws_s3_bucket" "good" {
for_each = toset(var.bucket_names)
bucket = each.value
}
With for_each, the address is aws_s3_bucket.good["curated"]. Remove that key and Terraform destroys exactly that one resource and leaves the others untouched. The rule is mechanical: use count only for “create N copies” or a boolean on/off toggle (count = var.enabled ? 1 : 0); use for_each for every collection of distinct things. If you’ve already shipped count over a collection and need to migrate, do it with moved blocks mapping each index to its new key so the change is config-driven and reviewable rather than a state mv by hand.
moved {
from = aws_s3_bucket.bad[0]
to = aws_s3_bucket.good["raw"]
}
8. Readability tradeoffs: when to push logic out of HCL
Every technique above puts computation in configuration, and that has a ceiling. A triple-nested for inside a flatten inside a merge, gated by a ternary, is technically correct and operationally hostile: the next engineer can’t predict the plan, and a typo produces a 400-line diff. Some honest limits:
- HCL has no named, unit-testable functions or loops. Past a couple of
forlevels, intent disappears into bracket soup. dynamicblocks are harder to debug than static ones because the rendered block isn’t in your source.- Heavy data munging in
localsis invisible toterraform planuntil something breaks; there’s no step-through.
| Logic lives best in… | When |
|---|---|
| Static HCL | Fixed structure, no variation by input |
dynamic + for |
Block count/content varies by typed input, shallow nesting |
validation / preconditions |
Asserting invariants and rejecting bad input early |
| External data / templates | Generation, lookups, anything you’d want to unit-test |
| A real language (CDKTF, a generator) | Combinatorial expansion, complex transforms, reuse across stacks |
When you find yourself reaching for the fourth nested loop, that’s the signal to generate the configuration upstream, model the data with an external data source, or move to a programmatic tool. The goal of advanced HCL is a simple interface backed by just enough cleverness, not a demonstration of how much the language can do.
Verify
Validate the type system, validation rules, and lifecycle checks before you trust the module. These commands are non-destructive.
# 1. Syntax and internal consistency, including type/validation wiring.
terraform init -backend=false
terraform validate
# 2. Canonical formatting (catches dynamic/for indentation drift).
terraform fmt -recursive -check -diff
# 3. Prove validation/precondition messages fire on bad input.
# Expect a non-zero exit and your custom error_message.
terraform plan -var 'environment=production' # not in [dev,staging,prod]
# 4. Inspect the resolved type structure of an input, defaults included.
echo 'var.buckets' | terraform console
Use terraform console to confirm optional() defaults materialize as expected; type an input expression and watch Terraform fill the missing attributes. To prove the count-vs-for_each addressing, run terraform state list after an apply and confirm resources are addressed by key (["raw"]), not index ([0]). For the full module surface, a native test file asserts both happy-path planning and that invalid inputs fail:
# tests/validation.tftest.hcl
run "rejects_bad_environment" {
command = plan
variables {
environment = "production"
buckets = {}
}
expect_failures = [var.environment]
}
Run it with terraform test. The expect_failures list turns a failing validation into a passing test, which is how you regression-guard your guardrails.