Securing the Software Supply Chain: SBOMs, Sigstore Signing, and SLSA Provenance in CI/CD

Supply-chain attacks do not break your code; they break the path your code travels from commit to cluster. This guide wires up the four controls that actually move the needle: a software bill of materials, keyless artifact signing, build provenance, and fail-closed verification at deploy time.

1. Anatomy of a supply-chain attack and SLSA as a roadmap

The classic attacks (SolarWinds, Codecov, the event-stream npm hijack, and the 2024 xz backdoor) share a shape: the attacker compromises the build or distribution step, not the source. The malicious artifact is signed by your legitimate pipeline and ships looking exactly like the real thing. Source review never sees it because the tampering happens after git push.

SLSA (Supply-chain Levels for Software Artifacts) gives you a maturity ladder to close that gap. The Build track is what most teams target:

Build level	What it requires	What it stops
L1	Provenance exists and is distributed	“Where did this come from?” guesswork
L2	Provenance is signed; build runs on a hosted, authenticated service	Casual forgery; build run on a laptop
L3	Provenance generated in an isolated, non-falsifiable build; secrets unforgeable by the build steps	A compromised build step tampering with its own provenance

The deliverables map cleanly to tools: SBOMs answer what is inside, signatures answer who produced it, and provenance answers how and from where it was built. You need all three, plus an enforcement point that rejects anything missing them.

2. Generating SBOMs with Syft

A software bill of materials is the dependency inventory you query later when the next CVE drops. Syft scans source trees and container images and emits SPDX or CycloneDX. Generate it against the exact image digest you are about to ship, never a floating tag.

# Scan the built image by digest and emit CycloneDX JSON
syft "registry.example.com/app@${IMAGE_DIGEST}" \
  -o cyclonedx-json=sbom.cdx.json \
  -o spdx-json=sbom.spdx.json

Producing the SBOM is not the goal; binding it to the artifact is. There are two complementary patterns. Embed it at build time as an OCI attestation, and also attach it as a signed Sigstore attestation (covered in step 4). With BuildKit you can capture an SBOM as part of the build itself:

docker buildx build \
  --sbom=true \
  --provenance=mode=max \
  -t "registry.example.com/app:${GIT_SHA}" \
  --push .

Callout: SBOM accuracy depends on the cataloguers matching your ecosystem. Validate that Syft actually detects your language’s packages (Go modules, Python wheels, jars) before you trust the output. An empty or partial SBOM is worse than none because it looks complete.

3. Keyless signing with cosign, Fulcio, and Rekor

Long-lived signing keys are a liability: they leak, they expire unnoticed, and rotating them across a fleet is painful. Sigstore’s keyless flow removes the standing key entirely. cosign obtains an OIDC token from your CI’s workload identity, exchanges it at Fulcio for a short-lived (roughly 10-minute) X.509 certificate bound to that identity, signs, and records the signature in the Rekor transparency log. The private key never outlives the signing operation.

In GitHub Actions, the ambient OIDC token drives this. The job needs id-token: write, and you should pin the cosign version rather than tracking main.

jobs:
  sign:
    runs-on: ubuntu-latest
    permissions:
      id-token: write   # mint the OIDC token Fulcio verifies
      packages: write    # push signature/attestation to GHCR
    steps:
      - uses: sigstore/cosign-installer@v3
        with:
          cosign-release: "v2.4.1"
      - name: Sign by digest
        env:
          COSIGN_EXPERIMENTAL: "1"
        run: |
          cosign sign --yes \
            "ghcr.io/${{ github.repository }}@${{ steps.build.outputs.digest }}"

Always sign by digest, never by tag. Tags are mutable; a signature over :latest proves nothing once the tag is repointed. cosign in modern releases defaults to the public good Fulcio and Rekor instances, so no key material is configured.

Callout: The signed identity is the OIDC subject (for example, the workflow path plus ref), not a key fingerprint. Verification policy therefore asserts “signed by this workflow, via this issuer” rather than “signed by this key.” That is the whole point of keyless: you verify provenance of identity, not custody of a secret.

4. Attaching the SBOM as a signed attestation

With keyless wired up, bind the SBOM to the image as an in-toto attestation so verifiers can demand it. cosign wraps the SBOM in a DSSE envelope, signs it the same keyless way, and logs it to Rekor.

cosign attest --yes \
  --predicate sbom.cdx.json \
  --type cyclonedx \
  "ghcr.io/org/app@${IMAGE_DIGEST}"

Now the SBOM is not a loose file in an artifact store that can be swapped; it is a signed predicate attached to the exact digest and anchored in a transparency log.

5. Producing SLSA provenance

Provenance is the cryptographically verifiable record of how the artifact was built: the source repo and commit, the builder identity, and the build parameters. Generating it inside the same job that runs your build only gets you so far, because a compromised build step could lie about its own inputs. SLSA Build L3 requires the provenance to be generated by an isolated component the build steps cannot influence.

The slsa-framework/slsa-github-generator provides reusable workflows that run the provenance generation on a separate, isolated runner and produce non-forgeable provenance. For container images, call the generator and hand it the image and digest your build produced:

  provenance:
    needs: [build]
    permissions:
      actions: read     # read workflow run metadata
      id-token: write   # keyless signing of the provenance
      packages: write    # write the provenance attestation
    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v2.0.0
    with:
      image: ghcr.io/org/app
      digest: ${{ needs.build.outputs.digest }}
    secrets:
      registry-password: ${{ secrets.GITHUB_TOKEN }}

Because the reusable workflow runs in its own job with its own token, the build job cannot tamper with the provenance it emits. The result is a signed SLSA provenance attestation attached to the image digest, verifiable later with cosign verify-attestation or slsa-verifier.

Callout: Build L3 hinges on isolation between the thing that builds and the thing that attests. If you generate provenance with an inline run: step in the same job as the build, you are at L2 at best, regardless of what the predicate claims.

6. Hardening the build environment

Signing a build that runs on a tampered machine just produces a trustworthy signature over malware. The build environment is itself part of the threat model.

• Ephemeral runners. Each job gets a fresh VM that is destroyed afterwards, so a compromise cannot persist into the next run. GitHub-hosted runners are ephemeral by default; for self-hosted, use the --ephemeral flag or an autoscaling controller (Actions Runner Controller) that recycles pods per job.
• Pin everything by digest. Pin Actions to a full commit SHA and base images to a digest. A mutable @v4 tag on a third-party action is an unguarded write into your build.
• Hermetic builds. Disallow network access during the build so dependencies come only from a vetted, pre-fetched cache. This makes builds reproducible and removes the “fetch a fresh malicious dependency at build time” vector.
• Least-privilege tokens. Set permissions: {} at the workflow root and grant scopes per job. The build job rarely needs id-token: write; only the signing and provenance jobs do.

permissions: {}   # deny-all default; opt in per job

jobs:
  build:
    permissions:
      contents: read   # nothing more

7. Admission-time verification with Kyverno

Producing all this metadata is wasted effort unless the cluster refuses to run anything that lacks it. Kyverno ships a verifyImages rule that performs cosign verification at admission and, critically, mutates the image reference to its digest so the verified bits are exactly the bits that run, defeating tag-swap races.

This policy demands a keyless signature from a specific GitHub Actions workflow and OIDC issuer:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-images
spec:
  validationFailureAction: Enforce   # fail closed
  webhookTimeoutSeconds: 30
  rules:
    - name: verify-cosign-keyless
      match:
        any:
          - resources:
              kinds: ["Pod"]
      verifyImages:
        - imageReferences:
            - "ghcr.io/org/*"
          mutateDigest: true       # pin to verified digest
          required: true
          attestors:
            - entries:
                - keyless:
                    issuer: "https://token.actions.githubusercontent.com"
                    subject: "https://github.com/org/app/.github/workflows/release.yml@refs/heads/main"
                    rekor:
                      url: "https://rekor.sigstore.dev"

You can extend the same rule with an attestations block to require the SLSA provenance predicate and assert fields inside it (for example, that the source repo matches), so an image signed by the right identity but built from the wrong repo is still rejected. The cosign policy controller (formerly Sigstore’s policy-controller) is a reasonable alternative if you prefer a Sigstore-native enforcement point over Kyverno.

Callout: validationFailureAction: Enforce is what makes this real. In Audit mode Kyverno only logs violations. Run in Audit first to find the unsigned images already in your cluster, then flip to Enforce once the backlog is clean.

8. Vulnerability gates: failing closed without locking yourself out

Signatures prove origin, not safety. A correctly signed image full of critical CVEs should still be blocked. Run Grype against the SBOM you already produced (faster and more accurate than re-scanning the image) and fail the pipeline on threshold.

grype "sbom:sbom.cdx.json" \
  --fail-on critical \
  --output table

“Fail closed” is correct, but a naive gate becomes an outage when a zero-day lands in a base image overnight and every deploy halts. Build the escape hatches in advance:

• A reviewed, expiring allowlist (.grype.yaml ignore rules with a justification and a follow-up date), not a blanket bypass.
• A documented break-glass path that still requires a signature, so the audit trail survives.
• Threshold tuned by severity and fix availability. Failing on an unfixable low-severity finding just trains people to disable the gate.

The deploy gate is then a logical AND: a valid signature, the required provenance, and a clean (or explicitly waived) vulnerability scan. Any one missing is a hard stop.

Enterprise scenario

A fintech platform team rolled keyless cosign across ~140 services and flipped Kyverno to Enforce. Within hours, a handful of production Deployments wedged: pods stuck ImagePullBackOff with no matching signatures on images that were demonstrably signed and present in Rekor. The signatures were real; verification at admission still failed.

The cause was the signature discovery model. cosign stores a signature as a sibling tag derived from the image digest (sha256-<digest>.sig). The platform fronted GHCR with a pull-through cache (Harbor proxy) for egress control. The cache happily mirrored the image manifest on first pull but had never been asked for the .sig tag, so it returned a 404 and Kyverno read that as “unsigned.” Worse, mutateDigest: true had pinned the running pods to a digest whose signature the cache couldn’t serve.

The fix was to make the proxy treat signatures as first-class artifacts and warm them alongside the image, plus point Kyverno’s verification at the upstream registry for the cosign lookup rather than the cache:

verifyImages:
  - imageReferences: ["harbor.corp/ghcr-proxy/org/*"]
    mutateDigest: true
    attestors:
      - entries:
          - keyless:
              issuer: "https://token.actions.githubusercontent.com"
              subject: "https://github.com/org/app/.github/workflows/release.yml@refs/heads/main"
    repository: "ghcr.io/org"   # resolve .sig/.att from upstream, not the cache

The durable lesson: signatures and attestations are OCI artifacts with their own tags. Any registry mirror, replication rule, or retention policy that operates on “the image” will silently drop them unless it is explicitly artifact-aware. Audit your caching and GC paths for .sig and .att before you enforce.

Verify

Prove the chain end to end before you trust it.

# 1. Verify the keyless signature and identity
cosign verify \
  --certificate-identity-regexp "https://github.com/org/app/.+" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  "ghcr.io/org/app@${IMAGE_DIGEST}"

# 2. Verify the SLSA provenance attestation
cosign verify-attestation \
  --type slsaprovenance \
  --certificate-identity-regexp "https://github.com/org/app/.+" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  "ghcr.io/org/app@${IMAGE_DIGEST}"

# 3. Confirm the entry is in the Rekor transparency log
cosign tree "ghcr.io/org/app@${IMAGE_DIGEST}"

# 4. Prove the cluster fails closed: an unsigned image must be rejected
kubectl run unsigned --image=nginx:latest --dry-run=server
#   expect: admission webhook "...kyverno..." denied the request

A passing signature check, a verified provenance predicate, a Rekor entry, and an admission denial for the unsigned image mean the controls are live, not theoretical.

Auditing and incident response

When a key identity or build is suspected compromised, the transparency log is your forensic record. Query Rekor for every artifact a given identity or workflow ever signed:

# Find all log entries signed by a given identity (email/SPIFFE/OIDC subject)
rekor-cli search --email "ci-bot@org.example"

# Inspect a specific entry by its log index
rekor-cli get --log-index 123456789

Because Rekor is append-only, you cannot delete a bad entry. You revoke trust instead: tighten the verification policy to exclude the compromised identity, repo, or time window, then rebuild and re-sign affected artifacts with a clean identity. Keyless makes this far less painful than rotating a leaked long-lived key, since there is no key to revoke across a fleet.

Rollout checklist

• SBOMs generated with Syft against the image digest, in CycloneDX and SPDX
• Keyless signing via cosign with id-token: write, signing by digest only
• SBOM and SLSA provenance attached as signed, Rekor-logged attestations
• Provenance generated by an isolated reusable workflow (Build L3), not inline
• Actions and base images pinned by full SHA / digest, root permissions: {}
• Kyverno (or cosign policy controller) in Audit mode to baseline violations
• Enforce flipped on with mutateDigest: true after the backlog is clean
• Vulnerability gate wired with an expiring allowlist and a break-glass path

Pitfalls and next steps

• Signing by tag. A signature over :latest is worthless the instant the tag moves. Sign and verify digests, always.
• Verifying the key, not the identity. Keyless verification asserts an OIDC subject and issuer. Pin both tightly; a loose identity regexp trusts more workflows than you think.
• Provenance generated inline. Without isolation between build and attestation you are at L2, no matter what the predicate says.
• Enforce before audit. Flipping Kyverno to Enforce with unsigned images still in the cluster causes an immediate self-inflicted outage. Baseline first.
• A fail-closed gate with no escape hatch. The first base-image zero-day will halt every deploy. Build the reviewed, expiring allowlist and break-glass path before you need them.

Once this holds for one service, promote the signing, provenance, and verification jobs into a reusable workflow and a shared Kyverno policy so every team inherits Build L3 by default. The goal is not a one-off hardened pipeline; it is making “signed, attested, and verified” the only way anything reaches production.