Kustomize in Depth: Overlays, Components, Strategic Merge Patches, and Secret/Config Generators

Helm templates YAML by stringifying it; Kustomize never does. It reads valid Kubernetes manifests, applies declarative transformations, and emits valid manifests. That difference matters: your base files always parse, kubectl understands them natively, and there is no Go templating language wedged between you and the API server. This article walks through structuring real multi-environment manifests with bases and overlays, choosing between strategic-merge and JSON 6902 patches, factoring opt-in features into Components, generating hashed ConfigMaps and Secrets that force safe rollouts, and wiring the whole thing into Argo CD.

Everything here assumes the modern Kustomize bundled with kubectl (v5.x of the standalone binary, kustomize.config.k8s.io/v1beta1 for kustomization.yaml and the stable Component kind). Avoid the deprecated bases, patchesStrategicMerge, and vars fields - all three are still parsed but emit warnings and vars is on its way out.

1. Why Kustomize: patch, don’t template

Two philosophies dominate Kubernetes configuration. Templating engines treat manifests as text and interpolate variables; Kustomize treats manifests as structured data and overlays patches. The practical consequences:

• Your base is real YAML. You can kubectl apply -f base/deployment.yaml directly during development. There is no {{ .Values.foo }} that only resolves at render time.
• Diffs are semantic. A patch says “set spec.replicas to 5 on this Deployment,” not “replace lines 23-24.” Refactoring whitespace in the base never breaks an overlay.
• It ships with kubectl. kubectl apply -k, kubectl kustomize, and kustomize build all consume the same kustomization.yaml. No extra controller, no Tiller history.

The cost is that Kustomize is deliberately not a programming language. There are no loops, no conditionals, no arithmetic. If you need to render twenty near-identical objects from a list, Kustomize will frustrate you and Helm (or cdk8s) is the better tool. Where Kustomize wins is the common enterprise case: one application, a handful of environments, and a need to keep the diff between prod and dev small, reviewable, and obviously correct.

2. Structuring bases and per-environment overlays

A base holds the environment-agnostic truth. Overlays hold only what differs. The directory layout that has held up across many teams:

app/
  base/
    kustomization.yaml
    deployment.yaml
    service.yaml
    configmap.yaml
  overlays/
    dev/
      kustomization.yaml
      replicas-patch.yaml
    staging/
      kustomization.yaml
    prod/
      kustomization.yaml
      replicas-patch.yaml
      resources-patch.yaml

The base kustomization.yaml simply enumerates resources and any labels that apply everywhere:

# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml
  - service.yaml
  - configmap.yaml

labels:
  - pairs:
      app.kubernetes.io/name: checkout
      app.kubernetes.io/part-of: storefront
    includeSelectors: false

Use the labels transformer rather than the older commonLabels. commonLabels always writes into selectors, which is dangerous: a Deployment’s spec.selector is immutable after creation, so changing a common label that lands in a selector forces you to delete and recreate the workload. labels with includeSelectors: false adds metadata labels without touching selectors. Reserve selector labels for a small, stable set you set once.

Each overlay references the base by relative path and layers its differences:

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: storefront-prod

resources:
  - ../../base

patches:
  - path: replicas-patch.yaml
  - path: resources-patch.yaml

images:
  - name: registry.internal/checkout
    newTag: "1.42.0"

Note resources: [../../base] - the modern field for referencing another kustomization is resources, not the deprecated bases. Promotion between environments becomes a one-line change to newTag, which is exactly the small reviewable diff you want flowing through pull requests.

3. Strategic merge patches vs JSON 6902 patches

Kustomize supports two patch dialects under the single patches field, and choosing the right one per change keeps overlays readable.

A strategic merge patch is a partial manifest. You write just enough of the object to identify it (apiVersion, kind, name) plus the fields you want to change, and Kustomize merges it into the matching resource using the same merge semantics kubectl apply uses:

# overlays/prod/resources-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
spec:
  template:
    spec:
      containers:
        - name: checkout
          resources:
            requests:
              cpu: "500m"
              memory: 512Mi
            limits:
              memory: 1Gi

Strategic merge understands Kubernetes list semantics: because containers has a merge key of name, the container named checkout is matched and patched in place rather than replaced. This is why strategic merge is the default choice for the 80% case - it reads like the resource it modifies.

A JSON 6902 patch is an explicit list of operations (add, remove, replace, move, copy, test) against JSON paths. It shines where strategic merge is awkward: deleting a single field, surgically editing one element of a plain list, or modifying a CRD that has no registered strategic-merge metadata.

# overlays/prod/kustomization.yaml (excerpt)
patches:
  - target:
      kind: Deployment
      name: checkout
    patch: |-
      - op: add
        path: /spec/template/spec/topologySpreadConstraints
        value:
          - maxSkew: 1
            topologyKey: topology.kubernetes.io/zone
            whenUnsatisfiable: DoNotSchedule
            labelSelector:
              matchLabels:
                app.kubernetes.io/name: checkout
      - op: remove
        path: /spec/template/spec/containers/0/livenessProbe

The target selector lets one JSON 6902 patch hit many objects at once - by kind, name, namespace, labelSelector, or annotationSelector. That is impossible with a strategic merge patch, which can only target the single object it names.

Concern	Strategic merge	JSON 6902
Readability	High (looks like the resource)	Lower (op/path/value)
Deleting a field	Awkward (null directives)	Trivial (op: remove)
Editing one list element	Needs merge key	Precise by index
Targeting many objects	No (one named object)	Yes (label/kind selector)
Works on arbitrary CRDs	Sometimes (needs schema)	Always (pure JSON path)

A useful rule: reach for strategic merge first; switch to JSON 6902 the moment you need to delete something, touch an indexed list element, or fan a change across multiple resources.

4. Reusable Components for opt-in features

Overlays are linear - each one extends a base. But cross-cutting features (sidecar injection, a PodDisruptionBudget, mTLS annotations) are orthogonal: you want them in some environments regardless of which base. That is what Component is for. A Component is a kustomization with kind: Component that can add resources and patches, and is pulled in by an overlay rather than building on top of one.

# components/pdb/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

resources:
  - pdb.yaml

patches:
  - target:
      kind: Deployment
      name: checkout
    patch: |-
      - op: add
        path: /spec/template/metadata/annotations/prometheus.io~1scrape
        value: "true"

# components/pdb/pdb.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: checkout

An overlay opts in with the components field. Because components apply in order, after the resources are loaded, staging and prod can share the same component without copy-paste:

# overlays/prod/kustomization.yaml (excerpt)
components:
  - ../../components/pdb
  - ../../components/otel-sidecar

The Component kind is stable in practice but still carries the v1alpha1 group version - that is the correct, current value and not a sign of instability. Note the ~1 in the JSON pointer above: it is the RFC 6901 escape for a literal /, required because prometheus.io/scrape contains a slash. Forgetting that escape is one of the most common JSON 6902 mistakes.

5. configMapGenerator and secretGenerator with content hashes

Hard-coding a ConfigMap and then editing it is a rollout footgun: the Deployment’s pod template never changes, so pods keep running with stale config until something happens to restart them. Generators fix this. configMapGenerator and secretGenerator build the object and append a hash of its contents to the name, so any change to the data produces a new object name, which changes the pod template, which triggers a rolling update automatically.

# base/kustomization.yaml (excerpt)
configMapGenerator:
  - name: checkout-config
    literals:
      - LOG_LEVEL=info
    files:
      - app.properties=config/app.properties

secretGenerator:
  - name: checkout-secrets
    type: Opaque
    envs:
      - secrets/checkout.env

generatorOptions:
  disableNameSuffixHash: false
  labels:
    app.kubernetes.io/managed-by: kustomize

Build this and the generated name looks like checkout-config-7h8t4k2mfd. Crucially, Kustomize rewrites every reference to checkout-config (in envFrom, volumes, valueFrom) to the hashed name in the same build. You reference the logical name in your Deployment; Kustomize wires up the hashed name. This is the single most valuable Kustomize feature and the one templating engines cannot replicate without extra tooling.

An overlay can patch a generator’s contents using behavior: merge (or replace), keyed by the logical name:

# overlays/prod/kustomization.yaml (excerpt)
configMapGenerator:
  - name: checkout-config
    behavior: merge
    literals:
      - LOG_LEVEL=warn
      - FEATURE_FAST_CHECKOUT=true

behavior: merge adds or overrides keys on top of the base generator; behavior: replace discards the base entirely. Set disableNameSuffixHash: true only for objects that genuinely must have a fixed name (for example a ConfigMap consumed by name from outside this kustomization) - and accept that you then own restarts manually.

For real secrets, do not commit plaintext .env files. The generator pattern composes cleanly with the Kustomize KSOPS plugin or with External Secrets Operator, where the generator references an already-decrypted file produced earlier in the pipeline.

6. Cross-cutting transformers: namePrefix, images, replacements

Beyond patches, Kustomize ships built-in transformers that rewrite fields across every resource in the build.

• namePrefix / nameSuffix prepend or append to every resource name and fix up references (Service names in Ingress backends, ConfigMap names in volumes). Use these to namespace objects per environment when you cannot use distinct Kubernetes namespaces.
• namespace sets metadata.namespace on every namespaced object at once.
• images overrides image name, newName, newTag, or digest without patching each container by hand. Prefer digest in production for immutability.

# overlays/staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namePrefix: stg-
namespace: storefront-staging

resources:
  - ../../base

images:
  - name: registry.internal/checkout
    newName: registry.internal/checkout
    digest: sha256:6c3e... # pinned by CI on promotion

7. Variable substitution with replacements

Older Kustomize had vars for copying a value from one object into another (for example, putting a generated Service name into a container env var). vars is deprecated and the documentation steers everyone to replacements, which is strictly more capable: it copies a value from a source field on one object to one or more target field paths on other objects.

# overlays/prod/kustomization.yaml (excerpt)
replacements:
  - source:
      kind: ConfigMap
      name: checkout-config
      fieldPath: metadata.name        # the hashed name
    targets:
      - select:
          kind: Deployment
          name: checkout
        fieldPaths:
          - spec.template.spec.containers.[name=checkout].env.[name=CONFIG_REF].value

The [name=checkout] syntax selects a list element by a field value, so you target the right container and env entry without relying on array indices. Because replacements runs after generators, the fieldPath: metadata.name source resolves to the hashed ConfigMap name - giving you the post-hash value that vars could never reliably reach. If you still have vars in a repo, migrating to replacements is the single highest-value cleanup you can make.

8. Integrating with Argo CD

Argo CD has first-class Kustomize support: point an Application at the overlay directory and Argo runs kustomize build for you, then syncs the output. No render step, no committed rendered manifests.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: checkout-prod
  namespace: argocd
spec:
  project: storefront
  source:
    repoURL: https://git.internal/storefront/deploy.git
    targetRevision: main
    path: app/overlays/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: storefront-prod
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Argo CD detects a Kustomize app by the presence of kustomization.yaml at path. You can override images directly from the Application spec under source.kustomize.images, which is how many teams wire their image promotion: a CI job patches the Argo CD Application (or a values file an ApplicationSet reads) rather than committing into the overlay. Either pattern works; pick one and be consistent so the source of truth for the running tag is never ambiguous.

If your generators need a plugin (KSOPS, for example), the repo-server must run with --enable-helm/--enable-alpha-plugins style configuration and the plugin binary present. Stock generators (configMapGenerator, secretGenerator) need no special flags.

Verify

Never trust an overlay you have not built. The whole point of Kustomize is that build produces exactly what the cluster will see, so render and inspect it.

# Render the full prod overlay to stdout (kubectl-native, no extra binary).
kubectl kustomize app/overlays/prod

# Identical output via the standalone binary, which tracks newer features first.
kustomize build app/overlays/prod

# Server-side dry run: validate against the live API without applying.
kubectl apply -k app/overlays/prod --dry-run=server

# Confirm the generated names carry a content hash.
kustomize build app/overlays/prod | grep -E 'name: checkout-(config|secrets)-'

# Diff two environments to prove the overlay delta is small and intentional.
diff <(kustomize build app/overlays/staging) <(kustomize build app/overlays/prod)

For CI, fail the build on schema violations by piping into a validator:

kustomize build app/overlays/prod | kubeconform -strict -summary -

A green kubeconform plus a human-reviewed diff between overlays is the gate that catches the overwhelming majority of Kustomize mistakes before they reach a cluster.

Enterprise scenario

A payments platform team ran a 30-service estate across dev, staging, and three regional production clusters, all on Kustomize. Their pain was image promotion: each promotion was a hand-edit to newTag in an overlay, and a tired engineer once promoted an unscanned tag straight to a prod overlay during an incident. Audit could not answer “which exact image is in eu-prod right now?” because the answer lived in whatever the last commit happened to say, sometimes a floating tag.

The constraint was hard: every production image had to be pinned by digest, set only by the CI pipeline after a passing supply-chain scan, with the running digest queryable from Git at any moment. Humans were not allowed to type tags into prod overlays.

They moved image selection out of the overlay entirely and into a per-cluster Argo CD Application, written by CI on a successful scan:

# Patched by the promotion pipeline, never by hand.
spec:
  source:
    path: app/overlays/prod
    kustomize:
      images:
        - registry.internal/checkout@sha256:6c3eb1f0a9...   # pinned digest

CI ran cosign verify against the digest, then patched the Application via argocd app set ... --kustomize-image. The overlay’s own images block was reduced to a placeholder for local builds. Result: prod was always pinned by digest, the running digest was a one-line kubectl get application away, and the human-typed-a-tag failure mode was structurally impossible. The Kustomize overlays stayed clean and identical across regions; the only per-cluster variation was the digest the pipeline injected.

Checklist

• Base contains only environment-agnostic, valid manifests that kubectl apply -f on their own.
• Overlays reference the base with resources:, not the deprecated bases:.
• Selector labels are set once and small; metadata labels use labels: with includeSelectors: false.
• Strategic merge patches for additive changes; JSON 6902 for deletions, indexed lists, and multi-target edits.
• JSON pointers escape / as ~1 and ~ as ~0.
• Opt-in features live in Components and are pulled in via components:, not copied across overlays.
• ConfigMaps and Secrets come from generators with the name-suffix hash enabled.
• No plaintext secrets in Git; generators feed from KSOPS/ESO-decrypted sources.
• Production images pinned by digest, not a floating tag.
• vars migrated to replacements.
• CI runs kustomize build | kubeconform -strict and a cross-overlay diff.
• Argo CD Application points at the overlay directory; image promotion path is single and unambiguous.