containerd Image Signing & Verification

Cosign Notary Sigstore

Secure your container supply chain with image signing and verification in containerd. This guide covers Cosign, Notary, Sigstore integration, and policy enforcement for trusted image pulls.

Cosign Notary Sigstore Image Verification

Why Image Signing Matters

Container images are the building blocks of modern applications. Without verification, you risk running malicious or tampered images from untrusted sources. Image signing uses cryptographic signatures to:

Verify image authenticity - Confirm the image comes from a trusted publisher.
Ensure image integrity - Detect tampering or corruption.
Prevent supply chain attacks - Stop compromised images from running.
Enable policy enforcement - Only allow signed images in production.

Image signing is a critical security control for production environments. Combine with vulnerability scanning for comprehensive supply chain security.

Image Signing Tools Comparison

Cosign

Ecosystem: Sigstore (open source)

Format: OCI artifacts (in-registry)

Key Mgmt: Public/private keys, KMS, GitHub OIDC

Pros: Simple, modern, CI/CD friendly

Notary v1

Ecosystem: Docker (CNCF)

Format: TUF (The Update Framework)

Key Mgmt: Private/public keys, delegation

Pros: Mature, supports delegation

Sigstore

Ecosystem: Linux Foundation

Format: Fulcio (short-lived certs), Rekor (transparency log)

Key Mgmt: OIDC identity, ephemeral keys

Pros: No key management, transparency

Cosign: Getting Started

Cosign is the most widely used image signing tool. It's part of the Sigstore project and works with any OCI registry.

                # Install Cosign
curl -L https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64 -o /usr/local/bin/cosign
chmod +x /usr/local/bin/cosign

# Generate a key pair
cosign generate-key-pair

# Sign an image
cosign sign --key cosign.key ghcr.io/user/nginx:latest

# Verify an image
cosign verify --key cosign.pub ghcr.io/user/nginx:latest

# Sign with keyless (OIDC)
cosign sign ghcr.io/user/nginx:latest

# Verify keyless
cosign verify ghcr.io/user/nginx:latest
            

Cosign supports keyless signing using GitHub or Google OIDC. This eliminates the need to manage private keys in CI/CD pipelines.

Notary: The Update Framework

Notary v1 implements TUF (The Update Framework) for secure image signing. It's used by Docker Hub and many enterprise registries. Notary v2 (a.k.a. Notation) is the next generation, using OCI artifacts.

                # Install Notary
curl -L https://github.com/docker/notary/releases/latest/download/notary-linux-amd64 -o /usr/local/bin/notary
chmod +x /usr/local/bin/notary

# Initialize Notary client
notary init

# Add a trusted key
notary key add -n mykey

# Publish a signed image
notary publish -s https://notary.docker.io -d ~/.docker/trust docker.io/user/nginx

# Verify with Notary
notary verify docker.io/user/nginx

# Notary v2 (Notation)
notation sign --key mykey ghcr.io/user/nginx:latest
notation verify ghcr.io/user/nginx:latest
            

Notary v1 is being deprecated in favor of Notation (Notary v2). Notation uses OCI artifacts and is fully compatible with Cosign.

Sigstore: Keyless Signing with Fulcio & Rekor

Sigstore provides keyless signing using ephemeral certificates from Fulcio and transparency logging with Rekor. This eliminates the need to manage private keys.

                # Install Sigstore components
# Cosign already includes Fulcio and Rekor support

# Keyless signing (using GitHub OIDC)
cosign sign ghcr.io/user/nginx:latest

# Verify with transparency log
cosign verify ghcr.io/user/nginx:latest

# Check Rekor transparency log
cosign verify-attestation --type https://slsa.dev/provenance/v1 \
  ghcr.io/user/nginx:latest

# Verify using Rekor
rekor-cli get --uuid 

# Enable identity verification
cosign verify --certificate-identity-regexp ".*@yourorg.com" \
  ghcr.io/user/nginx:latest
            

Keyless signing is the future of image security. It eliminates the overhead of key management and provides transparency through public logs.

Integrating Image Verification with containerd

containerd can verify image signatures before pulling. This ensures only trusted images are used in your clusters.

                # Configure containerd for image verification (config.toml)
[plugins."io.containerd.grpc.v1.cri".registry]
  # Registry configuration
  config_path = "/etc/containerd/certs.d"

  # Enable image verification (using Cosign)
  [plugins."io.containerd.grpc.v1.cri".registry.configs."ghcr.io".auth]
    username = "user"
    password = "token"

  [plugins."io.containerd.grpc.v1.cri".registry.configs."ghcr.io".tls]
    insecure_skip_verify = false

# Using cosign with containerd
# Pull and verify in one step
cosign verify ghcr.io/user/nginx:latest && ctr image pull ghcr.io/user/nginx:latest

# Use with crictl (Kubernetes)
crictl pull ghcr.io/user/nginx:latest

# Configuration for image verification policy
[plugins."io.containerd.grpc.v1.cri".registry.configs."ghcr.io".auth]
  auth = "base64_credentials"

# For Kubernetes, use ImagePolicyWebhook
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: image-policy
webhooks:
- name: image-policy.io
  rules:
  - apiGroups: [""]
    apiVersions: ["v1"]
    operations: ["CREATE"]
    resources: ["pods"]
            

For Kubernetes clusters, use the ImagePolicyWebhook admission controller to enforce image signing policies at the cluster level.

Policy Enforcement in Kubernetes

Enforce image signing policies in Kubernetes using admission controllers and policy engines.

                # ImagePolicyWebhook configuration
# /etc/kubernetes/admission/image-policy/config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
  configuration:
    imagePolicy:
      kubeConfigFile: /etc/kubernetes/admission/image-policy/webhook.yaml
      allowTTL: 50
      denyTTL: 50
      retryBackoff: 500
      defaultAllow: false

# Webhook configuration
# /etc/kubernetes/admission/image-policy/webhook.yaml
apiVersion: v1
kind: Config
clusters:
- cluster:
    server: https://image-policy-webhook:8443
  name: image-policy
contexts:
- context:
    cluster: image-policy
    user: image-policy
  name: image-policy
current-context: image-policy

# Policy as Code (OPA/Gatekeeper)
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: require-image-signature
spec:
  crd:
    spec:
      names:
        kind: ImageSignature
  targets:
  - target: admission.k8s.gatekeeper.sh
    rego: |
      package image_signature
      violation[{"msg": msg}] {
        container := input.review.object.spec.containers[_]
        not signed(container.image)
        msg := sprintf("Image %v is not signed", [container.image])
      }
      signed(image) {
        # Check signature using Cosign verification
        cosign.verify(image)
      }
            

CI/CD Integration

Integrate image signing into your CI/CD pipelines for automated security.

                # GitHub Actions workflow
name: Build and Sign Image
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: docker/setup-buildx-action@v2
    
    # Build image
    - uses: docker/build-push-action@v4
      with:
        push: true
        tags: ghcr.io/user/app:${{ github.sha }}
    
    # Sign with Cosign (keyless)
    - uses: sigstore/cosign-installer@v3
    - run: cosign sign ghcr.io/user/app:${{ github.sha }}
    
    # Verify signature
    - run: cosign verify ghcr.io/user/app:${{ github.sha }}

# GitLab CI
cosign:
  stage: sign
  image: sigstore/cosign:latest
  script:
    - cosign sign --key ${COSIGN_PRIVATE_KEY} ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA}
  only:
    - main
            

Attestations: Beyond Signatures

Attestations provide additional metadata about images: build provenance, SBOM, vulnerability reports, etc. Cosign supports multiple attestation types.

                # Generate SBOM attestation
cosign attest --type spdxjson \
  --predicate sbom.json \
  ghcr.io/user/nginx:latest

# Generate SLSA provenance attestation
cosign attest --type https://slsa.dev/provenance/v1 \
  --predicate provenance.json \
  ghcr.io/user/nginx:latest

# Verify attestations
cosign verify-attestation --type spdxjson \
  ghcr.io/user/nginx:latest

# Attestation types:
# - SLSA provenance
# - SBOM (SPDX, CycloneDX)
# - Vulnerability scan results
# - Test results
            

Troubleshooting Image Signing

                # Check if image is signed
cosign verify --key cosign.pub ghcr.io/user/nginx:latest

# View signatures (OCI artifacts)
crane manifest ghcr.io/user/nginx:latest | jq .

# Check Rekor transparency log
rekor-cli search --sha 

# Debug signing failure
cosign sign --verbose ghcr.io/user/nginx:latest

# Check registry permissions
crane auth ghcr.io

# Verify with debug output
cosign verify --debug ghcr.io/user/nginx:latest
            

Frequently Asked Questions

What's the difference between Cosign and Notary?

Cosign is newer, simpler, and uses OCI artifacts directly in registries. Notary v1 uses TUF and requires a separate trust store. Notary v2 (Notation) is converging with Cosign to use OCI artifacts.

Do I need to manage private keys with Cosign?

Not necessarily! Cosign supports keyless signing using OIDC (GitHub, Google). Your identity is used to sign images, eliminating private key management.

How does Sigstore improve security?

Sigstore provides transparency logs (Rekor) and ephemeral certificates (Fulcio). This ensures all signatures are publicly auditable and keys are short-lived, reducing risk.

Can containerd verify signatures natively?

containerd doesn't have built-in signature verification, but you can integrate with Cosign or Notary via admission controllers or custom policy engines in Kubernetes.

What is SLSA provenance?

SLSA (Supply-chain Levels for Software Artifacts) provenance provides metadata about how an artifact was built. It includes information about the build process, sources, and dependencies.

Should I sign all images?

Yes! Sign all production images and any images pulled from public registries. For development, signing is still recommended but may be less critical.

What happens if verification fails?

Policies determine the outcome. With ImagePolicyWebhook, the pod creation fails. With manual verification, the image won't be pulled. Always have a fallback plan for critical images.

Is image signing enough for supply chain security?

No. Image signing is one layer. Combine it with vulnerability scanning, SBOM generation, and runtime security monitoring for comprehensive supply chain security.

Previous: Security Best Practices Next: Secrets Management

Image signing and verification are essential for securing the container supply chain. Start with Cosign and Sigstore for modern, keyless signing with transparency.