Kubernetes Resource Management

Requests Limits Quotas Limit Ranges Burst Management

A comprehensive guide to Kubernetes resource management covering resource requests, limits, quotas, limit ranges, burst management, and QoS classes with practical implementation strategies for production clusters.

Requests Limits Quotas Burst Management

Why Resource Management Matters

Resource management is critical for running Kubernetes workloads efficiently and reliably. Proper resource management ensures:

Performance: Applications get the resources they need
Stability: Prevent resource starvation and pod eviction
Cost Efficiency: Avoid over-provisioning and waste
Fairness: Ensure all workloads get fair resource allocation
Predictability: Consistent application behavior
Cluster Health: Prevent resource exhaustion

Resource Management Layers:

Pod Level: Resource requests and limits per container
Namespace Level: ResourceQuotas and LimitRanges
Cluster Level: Node capacity and cluster autoscaling
QoS Classes: Guaranteed, Burstable, BestEffort

Resource Requests and Limits

Requests are the minimum resources guaranteed to a container. Limits are the maximum resources a container can use. Both are essential for resource management and scheduling.

                # Pod with CPU and Memory Requests/Limits
apiVersion: v1
kind: Pod
metadata:
  name: resource-demo
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "128Mi"
        cpu: "250m"
      limits:
        memory: "512Mi"
        cpu: "500m"

# Understanding CPU units
# 1 CPU = 1 vCPU/Core
# 100m = 0.1 CPU
# 250m = 0.25 CPU

# Understanding Memory units
# Mi = Mebibyte (1024 * 1024 bytes)
# Mi = 1024Ki
# Gi = 1024Mi

# Container with only requests (Burstable QoS)
apiVersion: v1
kind: Pod
metadata:
  name: burstable-pod
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "256Mi"
        cpu: "250m"

# Container with only limits (Burstable QoS)
apiVersion: v1
kind: Pod
metadata:
  name: burstable-limit-only
spec:
  containers:
  - name: app
    image: nginx
    resources:
      limits:
        memory: "512Mi"
        cpu: "500m"

# No requests or limits (BestEffort QoS)
apiVersion: v1
kind: Pod
metadata:
  name: best-effort-pod
spec:
  containers:
  - name: app
    image: nginx

# Multiple containers in a pod
apiVersion: v1
kind: Pod
metadata:
  name: multi-container
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "128Mi"
        cpu: "250m"
      limits:
        memory: "256Mi"
        cpu: "500m"
  - name: sidecar
    image: fluentd
    resources:
      requests:
        memory: "64Mi"
        cpu: "100m"
      limits:
        memory: "128Mi"
        cpu: "200m"
            

Critical: Always set both requests and limits for production workloads. Requests determine scheduling decisions. Limits prevent resource starvation. Never set limits without requests (it can lead to unpredictable scheduling).

Quality of Service (QoS) Classes

QoS classes determine how pods are prioritized when resources are scarce. They are automatically assigned based on resource requests and limits.

Guaranteed

Highest priority

Both CPU and memory requests = limits. These pods have the highest priority and are least likely to be evicted. Recommended for critical workloads.

Critical applications

Burstable

Medium priority

Requests < limits or only requests set. These pods can burst up to their limits when resources are available. Most common class for production workloads.

Standard applications

BestEffort

Lowest priority

No requests or limits set. These pods get whatever resources are available. First to be evicted under pressure. Not recommended for production.

Development, testing

                # Guaranteed QoS (requests = limits)
apiVersion: v1
kind: Pod
metadata:
  name: guaranteed-pod
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "256Mi"
        cpu: "500m"
      limits:
        memory: "256Mi"
        cpu: "500m"

# Check QoS class
kubectl get pod guaranteed-pod -o jsonpath='{.status.qosClass}'

# List pods by QoS
kubectl get pods -o json | jq '.items[] | {name: .metadata.name, qos: .status.qosClass}'

# Eviction order (from most likely to least)
# 1. BestEffort
# 2. Burstable (with highest usage-to-request ratio)
# 3. Guaranteed

# Pod priority vs QoS
# Priority classes can override QoS for eviction decisions
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000000
globalDefault: false
description: "High priority for critical workloads"
            

QoS Best Practices:

Use Guaranteed QoS for critical workloads (databases, API gateways)
Use Burstable QoS for most applications
Avoid BestEffort for production (no resource guarantees)
Consider priority classes for additional control
Monitor QoS distribution to ensure proper resource allocation

Resource Quotas: Namespace Limits

ResourceQuotas limit total resource consumption per namespace. They prevent a single namespace from consuming all cluster resources.

                # ResourceQuota for a namespace
apiVersion: v1
kind: ResourceQuota
metadata:
  name: namespace-quota
  namespace: production
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 10Gi
    limits.cpu: "20"
    limits.memory: 20Gi
    persistentvolumeclaims: "10"
    pods: "50"
    services: "10"
    services.loadbalancers: "2"
    services.nodeports: "5"
    count/deployments.apps: "10"
    count/statefulsets.apps: "5"

# Check quota usage
kubectl describe resourcequota namespace-quota -n production
kubectl get resourcequota -n production

# Multiple quotas in a namespace
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: production
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 10Gi
    limits.cpu: "20"
    limits.memory: 20Gi

apiVersion: v1
kind: ResourceQuota
metadata:
  name: object-quota
  namespace: production
spec:
  hard:
    persistentvolumeclaims: "10"
    pods: "50"
    services: "10"

# Quota with scopes
apiVersion: v1
kind: ResourceQuota
metadata:
  name: quota-with-scopes
  namespace: production
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 10Gi
  scopes:
  - NotTerminating
  - NotBestEffort

# Check quota status
kubectl get resourcequota -n production
kubectl describe resourcequota compute-quota -n production
            

Quota Considerations:

Quotas are namespace-scoped
Quotas can prevent pod creation if exceeded
Use separate quotas for compute and object resources
Set quotas for all namespaces to prevent resource contention
Monitor quota usage to adjust as needed

Limit Ranges: Default Resource Settings

LimitRanges set default resource requests and limits for pods in a namespace. They also enforce minimum and maximum resource constraints.

                # LimitRange with default values
apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: production
spec:
  limits:
  - default:
      cpu: 500m
      memory: 512Mi
    defaultRequest:
      cpu: 250m
      memory: 256Mi
    max:
      cpu: 2
      memory: 2Gi
    min:
      cpu: 100m
      memory: 128Mi
    maxLimitRequestRatio:
      cpu: 4
      memory: 4
    type: Container

# LimitRange for pods
apiVersion: v1
kind: LimitRange
metadata:
  name: pod-limits
  namespace: production
spec:
  limits:
  - max:
      cpu: 4
      memory: 4Gi
    min:
      cpu: 100m
      memory: 128Mi
    type: Pod

# LimitRange for persistent volumes
apiVersion: v1
kind: LimitRange
metadata:
  name: pvc-limits
  namespace: production
spec:
  limits:
  - max:
      storage: 100Gi
    min:
      storage: 1Gi
    type: PersistentVolumeClaim

# Check LimitRange
kubectl describe limitrange default-limits -n production
kubectl get limitrange -n production

# Effect: Pods without requests/limits get defaults
# Effect: Pods with requests/limits must comply with min/max
            

LimitRange Benefits:

Ensures all pods have resource requests/limits
Prevents pods from requesting too much or too little
Sets consistent resource policies across namespace
Reduces human error in resource configuration
Enforces minimum quality of service standards

Burst Management

Burst management controls how containers can temporarily exceed their CPU requests when resources are available. It's controlled by CPU limits and CFS (Completely Fair Scheduler) settings.

                # CPU Burst with Limit
apiVersion: v1
kind: Pod
metadata:
  name: burst-example
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: 250m
      limits:
        cpu: 500m  # Can burst up to 500m (2x request)

# CPU Manager policy (static)
# Allows guaranteed pods to use dedicated CPUs
apiVersion: v1
kind: ConfigMap
metadata:
  name: kubelet-config
  namespace: kube-system
data:
  cpu-manager-policy: static

# CPU Burst with CFS Quota
# Set CPUCFSQuotaPeriod for finer control
apiVersion: v1
kind: Pod
metadata:
  name: cfs-example
  annotations:
    cpu.cfs.quota.us: "100000"  # 100ms
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: 250m
      limits:
        cpu: 500m

# Node-level CPU manager
# Allocate guaranteed pods to dedicated cores
apiVersion: v1
kind: Pod
metadata:
  name: dedicated-cpu
  annotations:
    cpu-policy: dedicated
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: 1
      limits:
        cpu: 1

# Memory burst (memory is not compressible)
# Limits are hard limits for memory
apiVersion: v1
kind: Pod
metadata:
  name: memory-example
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: 256Mi
      limits:
        memory: 512Mi  # Can use up to 512Mi, OOMKilled at 512Mi
            

Burst Management Best Practices:

Set CPU limits higher than requests for burstable workloads
Use CPU Manager for latency-sensitive workloads
Memory limits are hard limits (OOMKill if exceeded)
Monitor CPU throttling to adjust limits
Use node-specific policies for performance-critical workloads

Resource Monitoring and Optimization

                # Check node resource usage
kubectl top nodes

# Check pod resource usage
kubectl top pods --all-namespaces

# Check pod resource usage with sorting
kubectl top pods --all-namespaces --sort-by=cpu
kubectl top pods --all-namespaces --sort-by=memory

# Check resource requests and limits
kubectl get pods -o json | jq '.items[] | {name: .metadata.name, resources: .spec.containers[].resources}'

# Identify pods with no requests
kubectl get pods -o json | jq '.items[] | select(.spec.containers[].resources.requests == null) | .metadata.name'

# Identify pods with no limits
kubectl get pods -o json | jq '.items[] | select(.spec.containers[].resources.limits == null) | .metadata.name'

# Resource usage by namespace
kubectl top pods --all-namespaces | awk '{arr[$1]+=$2} END {for (i in arr) print i, arr[i]}'

# Check resource quota usage
kubectl describe resourcequota --all-namespaces
            

Monitoring Best Practices:

Monitor actual vs requested resource usage
Identify pods with high resource usage
Adjust requests and limits based on usage patterns
Use VPA for automated recommendations
Monitor node utilization to detect resource shortages
Set up alerts for resource exhaustion

Resource Management Comparison

Feature	Requests	Limits	Quotas	LimitRanges
Purpose	Minimum guaranteed	Maximum allowed	Namespace caps	Defaults & constraints
Scope	Container	Container	Namespace	Namespace
Enforcement	Scheduling	Runtime	Admission	Admission
Default Values	None	None	None	Yes
QoS Impact	High	High	Indirect	Indirect
Eviction Priority	Yes	Yes	No	No
Resource Types	CPU, Memory	CPU, Memory	CPU, Memory, Objects	CPU, Memory

Frequently Asked Questions

What is the difference between requests and limits?

Requests are the minimum resources guaranteed to a container. Limits are the maximum resources a container can use. Requests determine scheduling decisions; limits prevent resource starvation.

What are QoS classes and why are they important?

QoS classes (Guaranteed, Burstable, BestEffort) determine pod priority for eviction under resource pressure. They are automatically assigned based on requests and limits. Higher QoS means lower eviction priority.

What is the difference between ResourceQuota and LimitRange?

ResourceQuota limits total resource consumption in a namespace. LimitRange sets default values and constraints for individual pods. Use both for comprehensive resource management.

What happens when a container exceeds its memory limit?

The container is terminated with an OOMKilled status. This is a hard limit. CPU limits cause throttling, not termination.

How do I choose the right resource requests?

Start with VPA recommendations or use historical usage data. Monitor actual usage and adjust gradually. Set requests based on steady-state usage, limits based on peak usage.

Can I set different CPU and memory limits?

Yes, CPU and memory are independent. CPU is compressible, memory is not. This means CPU can be throttled, memory excess causes OOMKill.

What are the recommended resource values for a basic application?

Start with CPU: 250m request, 500m limit; Memory: 256Mi request, 512Mi limit. Adjust based on actual usage. Use VPA for production recommendations.

How do I prevent resource starvation in a namespace?

Use ResourceQuotas to set limits on total resource consumption. Use LimitRanges for default pod constraints. Monitor usage and adjust quotas as needed.

Previous: Cluster Autoscaling Next: Node Maintenance

Effective resource management is the foundation of a healthy Kubernetes cluster. Implement these best practices to ensure your applications get the resources they need while maintaining cluster stability and cost efficiency.