Chart Provenance

Chart Signing Verification Provenance Files

A comprehensive guide to Helm chart provenance covering chart signing, verification, provenance files, and best practices for ensuring chart authenticity and integrity.

Chart Signing Verification Provenance Files

What is Chart Provenance?

Chart provenance is Helm's mechanism for ensuring chart authenticity and integrity. It uses PGP (Pretty Good Privacy) signatures to verify that a chart was created by a trusted source and hasn't been tampered with.

Provenance provides:

Authenticity: Confirms the chart was signed by the claimed author
Integrity: Ensures the chart hasn't been modified since signing
Non-repudiation: The signer cannot deny signing the chart
Trust: Enables verification against trusted public keys

Key Concept: Chart provenance is like a digital signature for your Helm charts. It provides cryptographic proof of the chart's origin and integrity.

How Provenance Works

PGP Key Pair

Generate keys

Create a PGP key pair for signing charts. The private key signs charts; the public key verifies them.

Chart Signing

Sign with private key

When packaging, Helm signs the chart with the private key, creating a .prov file.

Provenance File

Contains signature and metadata

The .prov file contains the chart hash, signature, and metadata for verification.

Verification

Verify with public key

Use the public key to verify the chart's signature and ensure integrity.

                # Provenance Workflow
# 1. Developer creates a PGP key pair
#    - Private key (secret): Used for signing
#    - Public key (shared): Used for verification

# 2. Developer signs the chart during packaging
helm package ./my-chart --sign --key "Developer Name"

# 3. This creates two files:
#    - my-chart-1.0.0.tgz (the chart)
#    - my-chart-1.0.0.tgz.prov (the provenance file)

# 4. User verifies the chart before installation
helm verify my-chart-1.0.0.tgz

# 5. Helm checks:
#    - The signature is valid (signed by trusted key)
#    - The chart hash matches the signed hash (no tampering)
#    - The chart metadata is consistent

# 6. If verification passes, the chart can be installed
helm install my-release my-chart-1.0.0.tgz
            

Generating PGP Keys

                # Generate a new PGP key pair
gpg --full-generate-key

# Follow the prompts:
# - Key type: RSA and RSA (default)
# - Key size: 4096 (recommended for security)
# - Expiration: 2y (2 years, recommended)
# - Real name: Your Name
# - Email: your.email@example.com
# - Comment: Helm Chart Signing Key

# List generated keys
gpg --list-keys
gpg --list-secret-keys

# Get the key fingerprint
gpg --fingerprint "Your Name"

# Export the public key (for sharing)
gpg --export --armor "Your Name" > public-key.asc

# Export the private key (keep secure!)
gpg --export-secret-keys --armor "Your Name" > private-key.asc

# Import a public key (for verification)
gpg --import public-key.asc

# Import a private key (for signing)
gpg --import private-key.asc

# Check the keyring
gpg --list-keys

# Example output:
# /home/user/.gnupg/pubring.kbx
# ----------------------------
# pub   rsa4096 2024-01-15 [SC] [expires: 2026-01-15]
#       ABC123DEF4567890ABC123DEF4567890ABC12345
# uid           [ultimate] Your Name 
# sub   rsa4096 2024-01-15 [E] [expires: 2026-01-15]
            

Key Security:

Never share your private key
Store private keys securely (HSM, Vault)
Use strong passphrases for private keys
Back up your keys in a secure location
Rotate keys periodically
Revoke compromised keys immediately

Signing Charts

                # Package and sign a chart
helm package ./my-chart --sign --key "Your Name" --keyring ~/.gnupg/secring.gpg

# This creates:
# - my-chart-1.0.0.tgz (the chart)
# - my-chart-1.0.0.tgz.prov (the provenance file)

# Sign with a specific key ID
helm package ./my-chart --sign --key "ABC123DEF456" --keyring ~/.gnupg/secring.gpg

# Sign with a specific keyring
helm package ./my-chart --sign --key "Your Name" --keyring /path/to/keyring.gpg

# Sign with a passphrase (will prompt)
helm package ./my-chart --sign --key "Your Name"

# View the provenance file
cat my-chart-1.0.0.tgz.prov

# Example provenance file content:
# -----BEGIN PGP SIGNED MESSAGE-----
# Hash: SHA512
#
# apiVersion: v2
# name: my-chart
# description: A Helm chart for my application
# type: application
# version: 1.0.0
# appVersion: 1.0.0
# ...
# files:
#   my-chart-1.0.0.tgz: sha256:abc123def456...
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAEBCgAdFiEE...
# -----END PGP SIGNATURE-----

# Sign a chart from a repository
helm pull my-repo/my-chart --version 1.0.0
helm package ./my-chart --sign --key "Your Name"

# Sign an existing chart package
# Note: Helm doesn't support signing existing packages directly
# You need to re-package with the --sign flag

# Sign in CI/CD pipelines
# Use environment variables for passphrase
export GPG_TTY=$(tty)
echo "your-passphrase" | gpg --batch --passphrase-fd 0 --import private-key.asc
helm package ./my-chart --sign --key "Your Name"
            

Signing Best Practices:

Sign all production charts
Use strong keys (4096-bit RSA)
Set key expiration dates
Store private keys securely
Automate signing in CI/CD
Document signing procedures

Verifying Charts

                # Import the public key
gpg --import public-key.asc

# Verify a chart
helm verify my-chart-1.0.0.tgz

# Verify with a specific keyring
helm verify --keyring ~/.gnupg/pubring.gpg my-chart-1.0.0.tgz

# Verify a chart from a repository
helm pull my-repo/my-chart --version 1.0.0
helm verify my-chart-1.0.0.tgz

# Verify while pulling
helm pull my-repo/my-chart --verify

# Verify during installation
helm install my-release my-chart-1.0.0.tgz --verify

# Example verification output:
# Signed by: Your Name 
# Using Key With Fingerprint: ABC123DEF456...
# Chart Hash Verified: sha256:abc123def456...

# Verify with GPG directly
gpg --verify my-chart-1.0.0.tgz.prov

# Check the signature
gpg --verify my-chart-1.0.0.tgz.prov my-chart-1.0.0.tgz

# Verify the chart hash
sha256sum my-chart-1.0.0.tgz
# Compare with hash in provenance file

# Automate verification in CI/CD
helm verify my-chart-1.0.0.tgz || exit 1
helm install my-release my-chart-1.0.0.tgz
            

Verification Steps:

Helm computes the SHA256 hash of the chart
Helm extracts the signed hash from the provenance file
Helm verifies the PGP signature
If the signature is valid and hashes match, verification passes
If any check fails, verification fails

Verification Best Practices:

Always verify charts before installation
Use trusted public keys
Automate verification in CI/CD
Verify in production deployments
Monitor for verification failures
Keep public keys updated

Provenance File Contents

                # Example provenance file
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

apiVersion: v2
name: my-chart
description: A Helm chart for my application
type: application
version: 1.0.0
appVersion: 1.0.0
home: https://example.com
icon: https://example.com/icon.png
keywords:
  - web
  - microservice
maintainers:
  - name: John Doe
    email: john@example.com
sources:
  - https://github.com/example/my-chart
dependencies:
  - name: postgresql
    version: 11.0.0
    repository: https://charts.bitnami.com/bitnami
files:
  my-chart-1.0.0.tgz: sha256:abc123def4567890abc123def4567890abc123def4567890abc123def4567890
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEE...
-----END PGP SIGNATURE-----

# Key components:
# 1. Hash algorithm (SHA512)
# 2. Chart metadata (from Chart.yaml)
# 3. File hash (SHA256 of the chart)
# 4. PGP signature

# Verify the provenance file structure
helm verify --debug my-chart-1.0.0.tgz

# Extract the provenance file
tar -xzf my-chart-1.0.0.tgz.prov

# View the signed content
gpg --decrypt my-chart-1.0.0.tgz.prov
            

Hash Algorithm

The algorithm used for hashing (SHA512 for the message, SHA256 for the chart).

Chart Metadata

Complete Chart.yaml content, including name, version, dependencies, and maintainers.

File Hash

SHA256 hash of the packaged chart file, used to verify integrity.

PGP Signature

The PGP signature that authenticates the content.

CI/CD Integration

                # GitHub Actions workflow for signing
name: Sign Helm Chart
on:
  push:
    tags:
      - 'v*'

jobs:
  sign:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    - name: Install Helm
      uses: azure/setup-helm@v3
    
    - name: Import GPG Key
      run: |
        echo "${{ secrets.GPG_PRIVATE_KEY }}" | gpg --batch --import
        echo "${{ secrets.GPG_PASSPHRASE }}" | gpg --batch --passphrase-fd 0 --pinentry-mode loopback --import
    
    - name: Package and Sign
      run: |
        helm package ./my-chart --sign --key "${{ secrets.GPG_KEY_NAME }}"
      env:
        GPG_TTY: /dev/tty
    
    - name: Verify Chart
      run: |
        helm verify my-chart-*.tgz
    
    - name: Upload Chart
      uses: actions/upload-artifact@v3
      with:
        name: chart
        path: |
          my-chart-*.tgz
          my-chart-*.tgz.prov

# GitLab CI
sign:
  stage: sign
  image: alpine/helm:latest
  before_script:
    - apk add --no-cache gnupg
    - echo "$GPG_PRIVATE_KEY" | gpg --batch --import
  script:
    - helm package ./my-chart --sign --key "$GPG_KEY_NAME"
    - helm verify my-chart-*.tgz
  artifacts:
    paths:
      - my-chart-*.tgz
      - my-chart-*.tgz.prov

# Jenkins Pipeline
pipeline {
    agent any
    stages {
        stage('Sign') {
            steps {
                withCredentials([
                    string(credentialsId: 'gpg-private-key', variable: 'GPG_KEY'),
                    string(credentialsId: 'gpg-passphrase', variable: 'GPG_PASS')
                ]) {
                    sh '''
                        echo "$GPG_KEY" | gpg --batch --import
                        helm package ./my-chart --sign --key "$GPG_KEY_NAME"
                        helm verify my-chart-*.tgz
                    '''
                }
            }
        }
    }
}
            

CI/CD Best Practices:

Use secrets for GPG keys and passphrases
Automate signing in CI/CD pipelines
Verify charts after signing
Store signed charts in artifact repositories
Publish public keys for verification
Monitor signing failures

Provenance vs Other Security Measures

Security Measure	Purpose	Protects Against	Complexity
Chart Provenance	Chart authenticity and integrity	Tampering, impersonation	Medium
Secrets Management	Protect sensitive data	Data breaches, leaks	High
RBAC	Access control	Unauthorized access	Medium
Vulnerability Scanning	Identify known vulnerabilities	Exploits, CVEs	Low
Policy Enforcement	Enforce security policies	Misconfigurations	High

Frequently Asked Questions

What is chart provenance in Helm?

Chart provenance is Helm's mechanism for ensuring chart authenticity and integrity using PGP signatures. It creates a .prov file that contains the chart's hash and signature.

How do I sign a Helm chart?

Use helm package --sign --key "Your Name" to sign a chart. This creates both the .tgz chart and the .prov provenance file.

How do I verify a signed chart?

Use helm verify my-chart-1.0.0.tgz after importing the public key. Helm checks the signature and chart hash.

What is in a provenance file?

The provenance file contains the chart metadata, SHA256 hash of the chart, and a PGP signature. It's used to verify the chart's authenticity and integrity.

Can I sign charts in CI/CD pipelines?

Yes! Import the GPG private key in CI/CD and use helm package --sign. Store the key and passphrase as secrets.

What happens if verification fails?

If verification fails, Helm will not install the chart (when using --verify). The chart may have been tampered with or signed by an untrusted key.

Is chart provenance required for Helm charts?

No, it's optional. However, it's highly recommended for production charts to ensure authenticity and integrity.

What is the difference between chart signing and image signing?

Chart signing verifies the Helm chart package. Image signing verifies container images. Both are important for supply chain security.

Chart Provenance

PGP Key Pair

Chart Signing

Provenance File

Verification

Hash Algorithm

Chart Metadata

File Hash

PGP Signature

What is Helm?

Helm Security

Secrets in Helm

Testing Charts

Helm Best Practices