Kubernetes Storage

PersistentVolumes PVCs StorageClasses CSI Drivers

A comprehensive guide to Kubernetes storage covering PersistentVolumes, PersistentVolumeClaims, StorageClasses, CSI drivers, volume modes, and practical implementation strategies for stateful applications.

Persistent Volumes PVCs CSI Drivers Storage Classes

Understanding Kubernetes Storage

Kubernetes provides a comprehensive storage system that enables stateful applications to persist data beyond the lifecycle of individual pods. The storage system abstracts the underlying storage infrastructure and provides a consistent interface for applications.

Key concepts in Kubernetes storage:

PersistentVolume (PV): A piece of storage in the cluster provisioned by an administrator
PersistentVolumeClaim (PVC): A request for storage by a user
StorageClass: Defines classes of storage with different QoS, backup policies, or performance
CSI Drivers: Container Storage Interface drivers that enable external storage providers

Key Principle: Kubernetes storage follows a claim-based model. Users create PVCs, and Kubernetes binds them to available PVs based on storage requirements. This decouples application storage requests from the underlying infrastructure.

Storage Components Overview

PersistentVolumes (PV)

Storage resource in the cluster

A PV is a piece of storage in the cluster that has been provisioned by an administrator. It is a cluster resource, independent of any pod, and has a lifecycle independent of any pod.

Storage provisioning

PersistentVolumeClaims (PVC)

Storage request by users

A PVC is a request for storage by a user. It specifies the amount of storage, access modes, and other requirements. Kubernetes binds PVCs to matching PVs.

Application storage requests

StorageClasses

Storage class definitions

StorageClasses define different classes of storage with varying QoS, backup policies, or performance characteristics. They enable dynamic provisioning of PVs.

Storage tiering, QoS

CSI Drivers

Container Storage Interface

CSI drivers enable external storage providers to integrate with Kubernetes. They handle volume provisioning, attachment, and mounting operations.

External storage integration

PersistentVolumes: Cluster Storage Resources

PersistentVolumes are cluster-level resources that represent storage. They can be provisioned statically (by an administrator) or dynamically (using StorageClasses).

                # Static PV Example (hostPath - for development)
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-hostpath-1
  labels:
    type: local
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: standard
  hostPath:
    path: /data/pv-1

# Static PV Example (NFS)
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-1
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nfs
  nfs:
    path: /exports/data
    server: nfs-server.example.com

# Static PV Example (AWS EBS)
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-aws-ebs
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: gp2
  awsElasticBlockStore:
    volumeID: vol-xxxx
    fsType: ext4

# Static PV Example (GCE PD)
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-gce-pd
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: standard
  gcePersistentDisk:
    pdName: pd-name
    fsType: ext4

# PV Reclaim Policies
# Retain - Keep volume for manual reclamation
# Recycle - Basic scrub (deprecated)
# Delete - Delete the volume
            

PV Reclaim Policies: When a PVC is deleted, the PV's reclaim policy determines what happens:

Retain: Volume is preserved for manual reclamation
Delete: Volume is deleted (cloud providers automatically free resources)
Recycle: Basic scrub (deprecated)

PersistentVolumeClaims: User Storage Requests

PVCs are user requests for storage. They specify storage requirements including size, access modes, and optionally StorageClass. Kubernetes binds PVCs to matching PVs.

                # Basic PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-basic
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: standard

# PVC with specific PV (binding)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-specific
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: standard
  volumeName: pv-hostpath-1  # Bind to specific PV

# PVC in a Pod
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx
    volumeMounts:
    - name: storage
      mountPath: /data
  volumes:
  - name: storage
    persistentVolumeClaim:
      claimName: pvc-basic

# Check PVC status
kubectl get pvc
kubectl describe pvc pvc-basic

# Check PV binding
kubectl get pv
            

PVC Best Practices:

Always specify the StorageClass for dynamic provisioning
Use the smallest required size to optimize storage usage
Consider using volume expansion if supported
Use namespaces to isolate storage resources
Monitor PVC usage and capacity

StorageClasses: Dynamic Provisioning

StorageClasses enable dynamic provisioning of PVs. They define different classes of storage with varying performance, availability, and cost characteristics.

                # AWS EBS StorageClass (gp2/gp3)
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp2
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
  fsType: ext4
  encrypted: "true"
  iopsPerGB: "10"

# AWS gp3 StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  fsType: ext4
  encrypted: "true"
  iops: "3000"       # Baseline performance
  throughput: "125"   # MB/s
volumeBindingMode: WaitForFirstConsumer

# GCE PD StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
  fstype: ext4
  replication-type: none

# Azure Disk StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azure-disk
provisioner: disk.csi.azure.com
parameters:
  skuname: StandardSSD_LRS
  kind: Managed
  cachingMode: ReadOnly

# NFS StorageClass (using external provisioner)
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs
provisioner: nfs.csi.k8s.io
parameters:
  server: nfs-server.example.com
  share: /exports/data
  mountOptions: "vers=4.1,hard,noatime"
reclaimPolicy: Delete
volumeBindingMode: Immediate
            

StorageClass Considerations: StorageClasses vary by cloud provider and CSI driver. Check your cloud provider's documentation for specific parameters. The default StorageClass is automatically used if not specified in PVC.

CSI Drivers: Container Storage Interface

CSI (Container Storage Interface) is a standard for exposing arbitrary block and file storage systems to containerized workloads. CSI drivers enable external storage providers to integrate with Kubernetes.

                # Install AWS EBS CSI Driver
helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver
helm repo update
helm install aws-ebs-csi-driver aws-ebs-csi-driver/aws-ebs-csi-driver \
  --namespace kube-system \
  --set enableVolumeScheduling=true \
  --set enableVolumeResizing=true \
  --set enableVolumeSnapshotting=true

# Install GCE PD CSI Driver
helm repo add gcp-pd-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/master/charts
helm repo update
helm install gcp-pd-csi-driver gcp-pd-csi-driver/gcp-compute-persistent-disk-csi-driver \
  --namespace kube-system

# Install Azure Disk CSI Driver
helm repo add azuredisk-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/charts
helm repo update
helm install azuredisk-csi-driver azuredisk-csi-driver/azuredisk-csi-driver \
  --namespace kube-system

# Install NFS CSI Driver
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner
helm repo update
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
  --set nfs.server=nfs-server.example.com \
  --set nfs.path=/exports/data

# Verify CSI Driver
kubectl get csidrivers
kubectl get pods -n kube-system | grep csi
            

CSI Benefits:

Standardized interface for storage providers
Dynamic provisioning of volumes
Volume snapshots and cloning
Volume resizing (expansion)
Consistent management across providers

Volume Snapshots and Backups

CSI drivers support volume snapshots, enabling point-in-time backups of persistent volumes. This is essential for disaster recovery and data protection.

                # VolumeSnapshotClass
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: ebs-snapshot-class
driver: ebs.csi.aws.com
deletionPolicy: Delete
parameters:
  force-create: "true"

# VolumeSnapshot
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: pvc-snapshot
  namespace: default
spec:
  volumeSnapshotClassName: ebs-snapshot-class
  source:
    persistentVolumeClaimName: pvc-basic

# Restore from snapshot (create PVC from snapshot)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-restored
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  dataSource:
    name: pvc-snapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io

# Check snapshots
kubectl get volumesnapshotclasses
kubectl get volumesnapshots
kubectl describe volumesnapshot pvc-snapshot

# Velero for application-level backups
velero install --provider aws --bucket backup-bucket --backup-location-config region=us-east-1
velero backup create app-backup --include-namespaces default,production
velero restore create --from-backup app-backup
            

Backup Best Practices:

Regularly take volume snapshots for critical data
Use Velero for application-level backups (includes resources + volumes)
Test restore procedures in a non-production environment
Implement backup retention policies
Monitor backup success and failures

Volume Modes: Filesystem vs Block

Kubernetes supports two volume modes: Filesystem (default) and Block. Block volumes are useful for applications that manage their own filesystems (databases).

                # Filesystem mode (default)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-filesystem
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 10Gi
  storageClassName: gp2

# Block mode
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-block
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Block
  resources:
    requests:
      storage: 100Gi
  storageClassName: gp2

# Block volume in Pod
apiVersion: v1
kind: Pod
metadata:
  name: database-pod
spec:
  containers:
  - name: database
    image: postgres:13
    volumeDevices:
    - name: data
      devicePath: /dev/xvda
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: pvc-block
            

Volume Mode Considerations: Not all storage providers support block volumes. Check your CSI driver or cloud provider documentation. Databases often benefit from block volumes due to direct raw device access and better performance.

Stateful Applications with StatefulSets

StatefulSets are the recommended way to run stateful applications on Kubernetes. They provide stable network identities and ordered, graceful deployment and scaling.

                # StatefulSet with persistent storage
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 50Gi
      storageClassName: gp2

# Headless Service for StatefulSet
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  clusterIP: None
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432
            

StatefulSet Best Practices:

Use volumeClaimTemplates for automatic PVC creation
Create a headless service for stable network identity
Plan for storage capacity and scaling requirements
Test backup and restore procedures
Use PodDisruptionBudgets for availability

Storage Comparison

Storage Type	Use Case	Access Modes	Provisioning	Persistence
EmptyDir	Pod temporary storage	ReadWriteOnce	Automatic	Pod lifecycle only
hostPath	Development, node-specific	ReadWriteOnce	Manual	Node lifecycle
PersistentVolume	Persistent storage	RWO, ROX, RWX	Static/Dynamic	Cluster lifecycle
CSI Volumes	Cloud-native storage	RWO, ROX, RWX	Dynamic	Cluster lifecycle
ConfigMap	Configuration data	ROX	Manual	ConfigMap lifecycle
Secret	Sensitive data	ROX	Manual	Secret lifecycle

Frequently Asked Questions

What is the difference between PV and PVC?

A PersistentVolume (PV) is a piece of storage in the cluster provisioned by an administrator. A PersistentVolumeClaim (PVC) is a request for storage by a user. PVCs bind to PVs based on storage requirements, similar to how pods request compute resources.

What is a StorageClass and why is it important?

A StorageClass defines different classes of storage with varying QoS, performance, or backup policies. It enables dynamic provisioning of PVs, allowing users to request storage without pre-provisioning volumes. It's essential for cloud-native storage management.

What are the access modes for volumes?

Access modes define how volumes can be accessed: ReadWriteOnce (RWO) - single node read/write, ReadOnlyMany (ROX) - multiple nodes read-only, ReadWriteMany (RWX) - multiple nodes read/write. Not all storage providers support all modes.

What is the CSI (Container Storage Interface)?

CSI is a standard for exposing arbitrary block and file storage systems to containerized workloads. CSI drivers enable external storage providers to integrate with Kubernetes, providing features like dynamic provisioning, snapshots, and volume resizing.

How do I backup persistent volumes?

Use volume snapshots (CSI supported) or Velero (application-level backup). Velero backs up both Kubernetes resources and persistent volume data. Always test restore procedures. For cloud providers, use native snapshot features with CSI drivers.

Can I resize a PVC after creation?

Yes, if your StorageClass supports volume expansion. Enable the feature in the StorageClass and ensure the CSI driver supports it. Some cloud providers require additional settings. Not all volume modes support expansion (block volumes may not).

What is the difference between volume mode Filesystem and Block?

Filesystem mode mounts a filesystem (ext4, xfs) on the volume. Block mode exposes the raw block device without a filesystem. Block mode is useful for databases that manage their own filesystem and can improve performance.

How do I choose the right storage for my application?

Consider: storage performance (IOPS, throughput), access mode requirements (RWO, RWX), data durability, backup/restore capabilities, and cost. Use different StorageClasses for different workload tiers (SSD for databases, HDD for archival).

Previous: CNI Comparison Next: Stateful Applications

Kubernetes storage is a powerful and flexible system for managing persistent data. Master these concepts to build reliable stateful applications and implement effective data management strategies.