Secrets in Helm

Sealed Secrets helm-secrets Plugin External Secrets

A comprehensive guide to managing secrets in Helm covering sealed secrets, helm-secrets plugin, external secrets management, and best practices for secure Helm deployments.

Sealed Secrets helm-secrets External Secrets

Why Secrets Management Matters

Managing secrets securely is one of the most critical aspects of Helm deployments. Secrets include passwords, API keys, TLS certificates, and other sensitive data. Improper handling can lead to:

Data Breaches: Exposed secrets in Git repositories
Unauthorized Access: Compromised credentials
Compliance Violations: Failing audits and regulations
Financial Loss: Cloud resource abuse
Reputation Damage: Loss of customer trust

Key Principle: Never store secrets in Helm values files or chart templates. Use dedicated secrets management solutions that encrypt secrets at rest and in transit.

The Problem with Helm Secrets

                # BAD PRACTICE: Secrets in values.yaml
# These are stored in plaintext in Git and Helm release secrets!
database:
  username: admin
  password: supersecretpassword  # EXPOSED!
  host: db.example.com

apiKey: sk-1234567890abcdef      # EXPOSED!
tlsKey: |
  -----BEGIN PRIVATE KEY-----
  MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQ...
  -----END PRIVATE KEY-----      # EXPOSED!

# BAD PRACTICE: Secrets in --set
# Visible in:
# - Process list (ps aux)
# - Shell history (~/.bash_history)
# - CI/CD logs
# - Helm release secrets
helm install my-release ./my-chart \
  --set database.password=supersecret \
  --set apiKey=sk-1234567890abcdef

# BAD PRACTICE: Secrets in chart templates
# Hardcoded secrets in templates are visible to anyone with chart access
apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
data:
  password: c3VwZXJzZWNyZXQ=  # EXPOSED!
  apiKey: c2stMTIzNDU2Nzg5MA==  # EXPOSED!

# What happens to secrets in Helm:
# 1. Helm stores release info as Kubernetes Secrets
# 2. Secrets in values are stored in plaintext
# 3. Anyone with access to the namespace can read them
kubectl get secret sh.helm.release.v1.my-release.v1 -o yaml
# The release secret contains all values, including secrets!
            

Critical: Helm stores release information (including values) as Kubernetes Secrets. If you store secrets in values, they're visible to anyone with access to the namespace. Always use external secret management.

Secrets Management Solutions

Sealed Secrets

Encrypt secrets for GitOps

Bitnami Sealed Secrets allows you to encrypt secrets so they can be safely stored in Git. Only the controller in the cluster can decrypt them.

helm-secrets Plugin

Encrypt values files with SOPS

Helm plugin that uses SOPS to encrypt values files. Supports AWS KMS, GCP KMS, Azure Key Vault, and PGP keys.

External Secrets Operator

Sync external secrets to Kubernetes

Syncs secrets from external providers (Vault, AWS Secrets Manager, GCP Secret Manager) into Kubernetes Secrets.

HashiCorp Vault

Enterprise secrets management

Centralized secrets management with dynamic secrets, encryption as a service, and fine-grained access control.

Sealed Secrets

Sealed Secrets allow you to encrypt secrets so they can be safely stored in Git. The Sealed Secrets controller in the cluster decrypts them into regular Kubernetes Secrets.

                # Install Sealed Secrets Controller
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm repo update
helm install sealed-secrets sealed-secrets/sealed-secrets \
  --namespace kube-system \
  --set fullnameOverride=sealed-secrets-controller

# Install kubeseal CLI
# macOS
brew install kubeseal

# Linux
wget https://github.com/bitnami-labs/sealed-secrets/releases/latest/download/kubeseal-linux-amd64
chmod +x kubeseal-linux-amd64
sudo mv kubeseal-linux-amd64 /usr/local/bin/kubeseal

# Create a Kubernetes Secret (plaintext)
kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=supersecret \
  --dry-run=client -o yaml > secret.yaml

# Seal the secret
kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# The sealed secret is safe to commit to Git
cat sealed-secret.yaml
# apiVersion: bitnami.com/v1alpha1
# kind: SealedSecret
# metadata:
#   name: db-credentials
# spec:
#   encryptedData:
#     username: AgBy... (encrypted)
#     password: AgBy... (encrypted)

# Apply the sealed secret
kubectl apply -f sealed-secret.yaml

# The controller decrypts it into a regular Secret
kubectl get secret db-credentials

# Use sealed secrets in Helm
# 1. Create sealed secret template
# templates/sealed-secret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: {{ include "my-app.fullname" . }}-db-credentials
  namespace: {{ .Release.Namespace }}
spec:
  encryptedData:
    username: {{ .Values.secrets.username }}
    password: {{ .Values.secrets.password }}
  template:
    metadata:
      name: {{ include "my-app.fullname" . }}-db-credentials
    type: Opaque

# 2. Use in Helm values (encrypted values)
secrets:
  username: AgBy...  # Encrypted with kubeseal
  password: AgBy...  # Encrypted with kubeseal

# 3. Install with Helm
helm install my-release ./my-chart -f values-sealed.yaml

# 4. Use the secret in your deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-app.fullname" . }}
spec:
  template:
    spec:
      containers:
      - name: app
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: {{ include "my-app.fullname" . }}-db-credentials
              key: username
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: {{ include "my-app.fullname" . }}-db-credentials
              key: password
            

Sealed Secrets Best Practices:

Commit sealed secrets to Git
Never commit unsealed secrets
Back up the controller's private key
Use namespace-scoped sealed secrets
Rotate sealing keys periodically
Use with ArgoCD or Flux for GitOps

helm-secrets Plugin

The helm-secrets plugin uses SOPS (Secrets OPerationS) to encrypt values files. It supports AWS KMS, GCP KMS, Azure Key Vault, and PGP keys.

                # Install helm-secrets plugin
helm plugin install https://github.com/jkroepke/helm-secrets

# Install SOPS
# macOS
brew install sops

# Linux
wget https://github.com/getsops/sops/releases/latest/download/sops-v3.8.1.linux.amd64
chmod +x sops-v3.8.1.linux.amd64
sudo mv sops-v3.8.1.linux.amd64 /usr/local/bin/sops

# Generate a PGP key (or use AWS KMS, GCP KMS, etc.)
gpg --full-generate-key

# Get the fingerprint
gpg --list-secret-keys --fingerprint

# Create .sops.yaml configuration
cat > .sops.yaml < secrets/prod.yaml <
            



            
                 SOPS Supported Key Providers:
                
                    AWS KMS: Use AWS Key Management Service
                    GCP KMS: Use Google Cloud KMS
                    Azure Key Vault: Use Azure Key Vault
                    PGP: Use PGP/GPG keys
                    Age: Use age encryption
                
            

            
                 helm-secrets Best Practices:
                
                    Use .sops.yaml for configuration
                    Commit encrypted files to Git
                    Never commit decrypted files
                    Use cloud KMS for production
                    Rotate encryption keys regularly
                    Use with CI/CD pipelines



        
        
            
                
                External Secrets Operator
            
            External Secrets Operator (ESO) syncs secrets from external providers into Kubernetes Secrets. It supports Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, and more.

            
                # Install External Secrets Operator
helm repo add external-secrets https://charts.external-secrets.io
helm repo update
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets \
  --create-namespace

# Create a SecretStore (Vault)
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-store
  namespace: default
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "my-app-role"
          serviceAccountRef:
            name: my-app-sa

# Create an ExternalSecret
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: default
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-store
    kind: SecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  data:
  - secretKey: username
    remoteRef:
      key: app/db
      property: username
  - secretKey: password
    remoteRef:
      key: app/db
      property: password

# Use in Helm chart
# templates/external-secret.yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: {{ include "my-app.fullname" . }}-db-credentials
  namespace: {{ .Release.Namespace }}
spec:
  refreshInterval: {{ .Values.externalSecrets.refreshInterval | default "1h" }}
  secretStoreRef:
    name: {{ .Values.externalSecrets.secretStore }}
    kind: SecretStore
  target:
    name: {{ include "my-app.fullname" . }}-db-credentials
  data:
  - secretKey: username
    remoteRef:
      key: {{ .Values.externalSecrets.path }}/db
      property: username
  - secretKey: password
    remoteRef:
      key: {{ .Values.externalSecrets.path }}/db
      property: password

# Use in deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-app.fullname" . }}
spec:
  template:
    spec:
      containers:
      - name: app
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: {{ include "my-app.fullname" . }}-db-credentials
              key: username
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: {{ include "my-app.fullname" . }}-db-credentials
              key: password

# Values for External Secrets
externalSecrets:
  enabled: true
  secretStore: vault-store
  refreshInterval: 1h
  path: app/db
            

            
                 External Secrets Best Practices:
                
                    Use external secret stores for all secrets
                    Configure refresh intervals appropriately
                    Use RBAC for ExternalSecret access
                    Monitor sync status
                    Use with GitOps for automated sync
                    Implement secret rotation
                
            
        

        
        
            
                
                Secrets Management Comparison
            
            
                
                    
                        Feature
                        Sealed Secrets
                        helm-secrets
                        External Secrets
                    
                
                
                    
                        Encryption
                        Asymmetric (public key)
                        SOPS (KMS/PGP)
                        External provider
                    
                    
                        GitOps Safe
                        Yes
                        Yes
                        Yes
                    
                    
                        Key Management
                        Controller manages keys
                        External KMS/PGP
                        External provider
                    
                    
                        Rotation
                        Manual
                        Manual
                        Automatic
                    
                    
                        Complexity
                        Low
                        Medium
                        High
                    
                    
                        Best For
                        Simple GitOps
                        Encrypted values
                        Enterprise
                    
                
            
        

        
        
            
                
                Frequently Asked Questions
            

            
                 Why can't I store secrets in values.yaml?
                Values.yaml is stored in plaintext in Git and Helm release secrets. Anyone with access to the repository or namespace can read the secrets. Always use external secret management.
            

            
                 What is the difference between Sealed Secrets and helm-secrets?
                Sealed Secrets encrypts Kubernetes Secrets for GitOps. helm-secrets encrypts values files using SOPS. Sealed Secrets is simpler; helm-secrets is more flexible with multiple KMS providers.
            

            
                 Can I use Sealed Secrets with Helm?
                Yes! Create SealedSecret resources in your Helm templates. The controller decrypts them into regular Kubernetes Secrets that your application can use.
            

            
                 How does helm-secrets work?
                helm-secrets is a plugin that wraps Helm commands and decrypts values files using SOPS. It supports AWS KMS, GCP KMS, Azure Key Vault, and PGP keys.
            

            
                 What is External Secrets Operator?
                External Secrets Operator syncs secrets from external providers (Vault, AWS Secrets Manager, GCP Secret Manager) into Kubernetes Secrets. It provides automatic rotation and centralized management.
            

            
                 Which secrets management solution should I use?
                For simple GitOps: Sealed Secrets. For encrypted values files: helm-secrets. For enterprise: External Secrets Operator or HashiCorp Vault. Choose based on your requirements.
            

            
                 How do I rotate secrets in Helm?
                Use External Secrets Operator for automatic rotation. For other solutions, update the secret and redeploy. Use helm upgrade with updated values or secrets.
            

            
                 Can I use secrets with Helm hooks?
                Yes, but be careful. Hooks run during installation/upgrade. Ensure secrets are available before hooks execute. Use pre-install hooks for secret creation if needed.
            
        

        
        
            
                
                Related Topics
            
            
                
                    
                    What is Helm?
                    Kubernetes package manager
                
                
                    
                    Helm Security
                    Security best practices
                
                
                    
                    Chart Provenance
                    Chart signing, verification
                
                
                    
                    Chart Hooks
                    Pre-install, post-install hooks
                
                
                    
                    Helm Best Practices
                    Production guidelines
                
            
        

        
        
             Previous: Helm Security
             Next: Chart Provenance
        

         Proper secrets management is essential for secure Helm deployments. Choose the right solution for your needs and never store secrets in plaintext.

Feature	Sealed Secrets	helm-secrets	External Secrets
Encryption	Asymmetric (public key)	SOPS (KMS/PGP)	External provider
GitOps Safe	Yes	Yes	Yes
Key Management	Controller manages keys	External KMS/PGP	External provider
Rotation	Manual	Manual	Automatic
Complexity	Low	Medium	High
Best For	Simple GitOps	Encrypted values	Enterprise