Cluster Autoscaling

Cluster Autoscaler HPA VPA Cost Management

A comprehensive guide to Kubernetes cluster autoscaling covering Cluster Autoscaler, HPA, VPA, resource optimization, cost management, and practical implementation strategies for efficient cluster operations.

Cluster Autoscaler HPA VPA Cost Management

Why Autoscaling Matters

Autoscaling is essential for running Kubernetes cost-effectively while maintaining performance and availability. It automatically adjusts resources based on demand, ensuring:

Cost Optimization: Scale down during low demand to save costs
Performance: Scale up during peak demand to maintain responsiveness
Availability: Handle traffic spikes without manual intervention
Resource Efficiency: Right-size resources for workloads
Operational Simplicity: Reduce manual scaling operations

Three Levels of Autoscaling:

Cluster Autoscaler: Scales the number of nodes in the cluster
HPA (Horizontal Pod Autoscaler): Scales the number of pod replicas
VPA (Vertical Pod Autoscaler): Scales pod resource requests/limits

Autoscaling Components

Cluster Autoscaler

Node-level scaling

Cluster Autoscaler automatically adjusts the size of the Kubernetes cluster by adding or removing nodes based on pending pods and node utilization.

Infrastructure scaling

Horizontal Pod Autoscaler (HPA)

Pod replica scaling

HPA automatically scales the number of pod replicas based on observed CPU utilization, memory usage, or custom metrics. Supports both scale-up and scale-down.

Application scaling

Vertical Pod Autoscaler (VPA)

Pod resource adjustments

VPA automatically adjusts CPU and memory requests/limits for pods based on historical usage. Recommended for stateful applications and workloads with variable resource needs.

Resource optimization

Metrics Server

Resource metrics API

Metrics Server collects resource metrics from nodes and pods, providing the data needed for HPA and other autoscaling components to make scaling decisions.

Metrics collection

Cluster Autoscaler: Node-Level Scaling

Cluster Autoscaler automatically adjusts the number of nodes in your cluster based on the resource requirements of pending pods and node utilization.

                # Install Cluster Autoscaler (AWS)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      serviceAccountName: cluster-autoscaler
      containers:
      - image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.28.0
        name: cluster-autoscaler
        command:
        - ./cluster-autoscaler
        - --v=4
        - --stderrthreshold=info
        - --cloud-provider=aws
        - --skip-nodes-with-local-storage=false
        - --expander=least-waste
        - --balance-similar-node-groups
        - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
        env:
        - name: AWS_REGION
          value: us-east-1
        resources:
          requests:
            cpu: 100m
            memory: 300Mi

# Install Cluster Autoscaler (GCP)
# --cloud-provider=gce
# --node-group-auto-discovery=mig:name-starting-with=gke-my-cluster

# Install Cluster Autoscaler (Azure)
# --cloud-provider=azure
# --node-group-auto-discovery=azure:resourceGroup=my-rg,vmss=my-vmss

# Cluster Autoscaler Configuration
# Asg tags for AWS:
# - k8s.io/cluster-autoscaler/enabled: true
# - k8s.io/cluster-autoscaler/my-cluster: owned

# Check Cluster Autoscaler status
kubectl get pods -n kube-system -l app=cluster-autoscaler
kubectl logs -n kube-system -l app=cluster-autoscaler --tail=50

# Manual scale-up (to test)
# Create a deployment with many replicas
kubectl scale deployment busybox --replicas=50

# Manual scale-down (remove pods)
kubectl scale deployment busybox --replicas=5

# Cluster Autoscaler logs
# "Scale up" events
kubectl get events | grep -i scaling
kubectl describe pod -n kube-system cluster-autoscaler-pod
            

Cluster Autoscaler Best Practices:

Use node group auto-discovery for dynamic scaling
Set --balance-similar-node-groups for even distribution
Use --expander to control scaling behavior (least-waste, priority)
Enable --skip-nodes-with-local-storage for pods with PVCs
Configure pod disruption budgets for critical workloads
Use cluster-autoscaler.kubernetes.io/safe-to-evict: false annotation for critical pods

Horizontal Pod Autoscaler (HPA)

HPA automatically scales the number of pod replicas based on observed metrics. It supports CPU, memory, and custom metrics.

                # Install Metrics Server
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Verify Metrics Server
kubectl get pods -n kube-system -l k8s-app=metrics-server
kubectl top nodes
kubectl top pods

# Basic HPA - CPU based
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

# HPA with Memory
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa-memory
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

# HPA with Custom Metrics (Prometheus)
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa-custom
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: 100

# HPA with External Metrics (e.g., SQS queue length)
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-worker
  minReplicas: 1
  maxReplicas: 20
  metrics:
  - type: External
    external:
      metric:
        name: sqs_queue_length
      target:
        type: AverageValue
        averageValue: 10

# Check HPA status
kubectl get hpa
kubectl describe hpa app-hpa

# HPA events
kubectl get events --field-selector involvedObject.kind=HorizontalPodAutoscaler
            

HPA Considerations:

HPA requires Metrics Server or custom metrics adapter
Set appropriate min and max replicas
Use behavior configuration for fine-grained control
Test HPA with load testing before production
Monitor HPA decisions to avoid thrashing

Vertical Pod Autoscaler (VPA)

VPA automatically adjusts CPU and memory requests/limits for pods based on historical usage patterns. It's particularly useful for stateful applications and workloads with variable resource needs.

                # Install Vertical Pod Autoscaler
git clone https://github.com/kubernetes/autoscaler.git
cd autoscaler/vertical-pod-autoscaler
./hack/vpa-up.sh

# Verify VPA installation
kubectl get pods -n kube-system -l app=vpa

# VPA Configuration (Recommender mode)
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: app-vpa
  namespace: default
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Off"  # Off, Initial, Recreate, Auto
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 100m
        memory: 100Mi
      maxAllowed:
        cpu: 2000m
        memory: 2Gi
      controlledResources: ["cpu", "memory"]

# VPA with Auto mode (recommended for production)
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: app-vpa-auto
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Auto"  # Automatically apply recommendations
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 100m
        memory: 100Mi
      maxAllowed:
        cpu: 4000m
        memory: 4Gi

# VPA with Initial mode (set initial resources)
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: app-vpa-initial
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Initial"  # Apply only at pod creation
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 200m
        memory: 256Mi

# Check VPA recommendations
kubectl describe vpa app-vpa

# View VPA recommendations
kubectl get vpa app-vpa -o yaml | grep -A 10 recommendation

# VPA with Prometheus (for better recommendations)
# Configure VPA to use Prometheus as metric source
# --metric-provider=prometheus --prometheus-address=http://prometheus:9090
            

VPA vs HPA:

HPA: Scales the number of pods (horizontal)
VPA: Scales the resources per pod (vertical)
Use HPA for stateless applications with variable load
Use VPA for stateful applications or when pods can't be easily replicated
Use both cautiously - they can conflict

Advanced Scaling Behavior

                # HPA Behavior Configuration (Kubernetes 1.18+)
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa-behavior
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300  # Wait 5 minutes before scaling down
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60  # Max 10% scale down per minute
      - type: Pods
        value: 2
        periodSeconds: 60
      selectPolicy: Min
    scaleUp:
      stabilizationWindowSeconds: 0  # Scale up immediately
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15  # Double pods every 15 seconds
      - type: Pods
        value: 4
        periodSeconds: 15
      selectPolicy: Max

# Cluster Autoscaler expanders
# --expander=least-waste (default)
# --expander=random
# --expander=most-pods
# --expander=price
# --expander=priority

# Priority expander example
apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-autoscaler-priority-expander
  namespace: kube-system
data:
  priorities: |
    10:
      - .*-spot.*
    5:
      - .*-ondemand.*
    1:
      - .*-reserved.*
            

Scaling Behavior Best Practices:

Use stabilization windows to prevent thrashing
Set different policies for scale-up (faster) and scale-down (slower)
Use percent-based policies for large deployments
Test scaling behavior with load testing
Monitor scaling events for anomalies

Cost Optimization with Autoscaling

Spot Instances

Lower-cost capacity

Use spot instances for fault-tolerant workloads. Configure Cluster Autoscaler to use spot instance groups with priority expanders. Combine with HPA for flexible scaling.

Cost savings

Right-Sizing

Optimize resource requests

Use VPA recommendations to right-size resource requests. Analyze actual usage and adjust requests to avoid over-provisioning and waste. Use tools like KubeCost for visibility.

Resource efficiency

Scheduled Scaling

Time-based scaling

Use CronHPA or KEDA for scheduled scaling based on predictable traffic patterns. Scale down during off-hours and weekends to reduce costs.

Predictable workloads

Utilization Monitoring

Track and optimize

Monitor node and pod utilization to identify waste. Use metrics like CPU/memory usage, pod density, and node efficiency. Automate scaling based on utilization trends.

Continuous optimization

                # KEDA ScaledObject with schedule
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: scheduled-scaling
spec:
  scaleTargetRef:
    name: my-app
    kind: Deployment
  minReplicaCount: 0
  maxReplicaCount: 10
  triggers:
  - type: cron
    metadata:
      timezone: America/New_York
      start: 0 9 * * *
      end: 0 17 * * *
      desiredReplicas: "5"

# Cluster Autoscaler with spot instances
# AWS ASG with mixed instances policy
# Use node group with spot price cap
# Configure expander priority for spot

# Kubecost for cost monitoring
helm repo add kubecost https://kubecost.github.io/cost-analyzer
helm repo update
helm install kubecost kubecost/cost-analyzer --namespace kubecost --create-namespace

# Check cost report
kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090:9090

# Resource optimization tips
# - Use KubeCost to identify waste
# - Right-size with VPA recommendations
# - Use spot instances for fault-tolerant workloads
# - Implement HPA for demand-based scaling
# - Use committed use discounts for steady workloads
            

Cost Optimization Tips:

Use spot instances for stateless, fault-tolerant workloads
Right-size resource requests with VPA recommendations
Schedule scaling for predictable traffic patterns
Monitor and alert on cost anomalies
Use committed use discounts for baseline capacity
Regularly review and optimize resource usage

Autoscaling Comparison

Feature	Cluster Autoscaler	HPA	VPA	KEDA
Scaling Level	Nodes	Pods (replicas)	Pods (resources)	Pods (replicas)
Metrics	Pending pods, utilization	CPU, Memory, Custom	Historical usage	Events, Queues, Schedules
Scale-up Speed	Slow (minutes)	Fast (seconds)	Slow (hours)	Fast
Scale-down	Yes	Yes	Yes	Yes
Use Case	Infrastructure	Stateless apps	Stateful apps	Event-driven
Cost Impact	High	Medium	Medium	Medium
Complexity	Medium	Low	Medium	Medium

Autoscaling Strategy: Use all three autoscaling mechanisms together: Cluster Autoscaler for infrastructure, HPA for application scaling, and VPA for resource optimization. This provides comprehensive cost optimization and performance management.

Frequently Asked Questions

What is the difference between HPA and VPA?

HPA scales the number of pod replicas (horizontal scaling). VPA scales the CPU/memory requests per pod (vertical scaling). HPA is for stateless applications, VPA is for stateful applications. They can be used together but may conflict.

How does Cluster Autoscaler decide to scale up?

Cluster Autoscaler monitors pending pods (pods that cannot be scheduled). If there are pending pods due to resource constraints, it attempts to add nodes. It also considers node utilization for scale-down decisions.

What metrics does HPA support?

HPA supports CPU utilization, memory utilization, custom metrics (via Prometheus), and external metrics (like queue length). It also supports pod metrics and object metrics for more advanced scenarios.

How do I prevent HPA from scaling down too aggressively?

Use behavior configuration with a stabilization window (stabilizationWindowSeconds). This prevents rapid scale-down and reduces thrashing. Use scaleDown policies to control the rate of scale-down.

What is KEDA and how does it differ from HPA?

KEDA (Kubernetes Event-driven Autoscaling) is an extension that provides advanced autoscaling based on event sources (queues, streams, cron schedules). It extends HPA with support for more metrics and event-driven workloads.

How do I optimize costs with autoscaling?

Use a combination of strategies: Cluster Autoscaler for node scaling, HPA for application scaling, VPA for resource right-sizing, spot instances for cost savings, and scheduled scaling for predictable workloads. Monitor cost usage with tools like KubeCost.

What are the prerequisites for Cluster Autoscaler?

Cloud provider permissions (IAM roles), node group auto-discovery configuration, and pod anti-affinity for the autoscaler pod. Also, ensure pods have appropriate resource requests and limits.

Can I use VPA and HPA together?

Yes, but carefully. VPA is recommended for stateful applications, HPA for stateless. They can conflict when used together. Use VPA in recommendation mode (updateMode: Off) and manually adjust resources if needed.

Previous: Alerting Best Practices Next: Resource Management

Autoscaling is essential for running Kubernetes cost-effectively while maintaining performance. Implement these best practices to optimize resource usage, reduce costs, and ensure your applications scale seamlessly with demand.