Kubernetes Deployment Strategies

Rolling Updates Blue-Green Canary A/B Testing Feature Flags

A comprehensive guide to Kubernetes deployment strategies covering rolling updates, blue-green, canary, A/B testing, feature flags, and progressive delivery with detailed explanations, practical examples, and best practices for zero-downtime deployments.

Rolling Updates Blue-Green Canary A/B Testing Feature Flags

Why Deployment Strategies Matter

Deployment strategies define how new versions of applications are rolled out to production. The right strategy ensures zero downtime, reduces risk, enables fast rollbacks, and allows for testing in production. In modern cloud-native environments, deployment strategies are critical for maintaining reliability and user experience.

Kubernetes provides native support for rolling updates and, with additional tools like Argo Rollouts, Istio, and Flagger, enables advanced deployment patterns like canary, blue-green, and A/B testing.

Key Principles:

Zero Downtime: Users should not experience interruptions during deployments
Risk Reduction: Minimize the blast radius of failed deployments
Fast Rollback: Ability to quickly revert to previous versions
Validation: Test new versions with real traffic before full rollout

Deployment Strategies Overview

Rolling Update

Default Kubernetes deployment

Gradually replaces old pods with new ones while maintaining application availability. Native to Kubernetes Deployments.

Simple, no extra infrastructure, built-in

Limited control, no traffic splitting

Stateless applications, standard deployments

Blue-Green

Two identical environments

Maintains two environments (blue=current, green=new). Traffic is switched at once after validation. Instant rollback by switching back.

Instant switch, easy rollback

Double infrastructure cost

Critical applications, immediate rollback needed

Canary Deployment

Incremental rollout with monitoring

Rolls out new version to a small subset of users first, monitors for issues, then gradually increases traffic. Advanced with Istio/Argo Rollouts.

Low risk, real-user validation

Complex, requires traffic management

High-risk changes, performance-sensitive apps

A/B Testing

User-specific routing

Routes users to different versions based on headers, cookies, or user attributes. Used for feature validation and user experience testing.

Data-driven decisions, user-specific

Requires routing logic, complex setup

Feature validation, UX testing

Feature Flags

Feature toggles

Controls features in code without deployment. Enables dark launches, gradual rollouts, and instant toggling of features. Used with LaunchDarkly, Flagsmith, etc.

No deployment needed, instant control

Code complexity, flag management

Feature management, dark launches

Shadow Deployment

Real traffic, no impact

Sends real production traffic to the new version without affecting users. Used for performance testing and validation.

Real testing, no user impact

Resource intensive, no user feedback

Performance testing, capacity planning

Rolling Updates: The Default Strategy

Kubernetes Deployments use rolling updates by default. This strategy gradually replaces old pods with new ones while ensuring a minimum number of pods remain available.

                # Rolling Update Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1           # Max pods above desired count
      maxUnavailable: 0     # Max unavailable pods during update
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx
        image: nginx:1.25   # New version

# Update image (triggers rolling update)
kubectl set image deployment/nginx nginx=nginx:1.26

# Check rollout status
kubectl rollout status deployment/nginx

# Pause rollout (for testing)
kubectl rollout pause deployment/nginx

# Resume rollout
kubectl rollout resume deployment/nginx

# Rollback if needed
kubectl rollout undo deployment/nginx

# View rollout history
kubectl rollout history deployment/nginx
            

Rolling Update Best Practices:

Set maxSurge and maxUnavailable based on your workload requirements
Use maxUnavailable: 0 for zero-downtime deployments
Implement health checks (readiness probes) to prevent routing traffic to unhealthy pods
Use kubectl rollout pause for manual validation before completing the rollout

Blue-Green Deployment

Blue-Green deployment uses two identical environments. The "blue" environment is the current production version, and the "green" environment is the new version. After validation, traffic is switched from blue to green in one operation.

                # Blue Deployment (current)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-blue
  labels:
    app: nginx
    version: blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
      version: blue
  template:
    metadata:
      labels:
        app: nginx
        version: blue
    spec:
      containers:
      - name: nginx
        image: nginx:1.24

---
# Green Deployment (new)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-green
  labels:
    app: nginx
    version: green
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
      version: green
  template:
    metadata:
      labels:
        app: nginx
        version: green
    spec:
      containers:
      - name: nginx
        image: nginx:1.25

---
# Service pointing to blue
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
    version: blue   # Switch to green when ready
  ports:
  - port: 80
    targetPort: 80

# Switch traffic from blue to green
kubectl patch service nginx-service -p '{"spec":{"selector":{"version":"green"}}}'

# After validation, scale down blue
kubectl scale deployment nginx-blue --replicas=0
            

Blue-Green Considerations: Blue-Green requires double the infrastructure resources during the transition. Database migrations must be backward-compatible or handled separately. Consider using a service mesh for more sophisticated traffic switching.

Canary Deployment

Canary deployments roll out new versions to a small subset of users first, monitor for issues, and gradually increase traffic. This strategy minimizes the blast radius of failures and enables real-world validation before full rollout.

                # Using Argo Rollouts for Canary
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: nginx-canary
spec:
  replicas: 5
  strategy:
    canary:
      maxSurge: 1
      maxUnavailable: 0
      steps:
      - setWeight: 10        # Start with 10% traffic
      - pause: {duration: 30s}
      - setWeight: 25
      - pause: {duration: 1m}
      - setWeight: 50
      - pause: {duration: 2m}
      - setWeight: 100       # Full rollout
      trafficRouting:
        istio:
          virtualService:
            name: nginx-vs
            routes:
            - primary
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.25

---
# Istio VirtualService for canary
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: nginx-vs
spec:
  hosts:
  - nginx-service
  http:
  - route:
    - destination:
        host: nginx-canary-primary
        subset: stable
      weight: 90
    - destination:
        host: nginx-canary-canary
        subset: canary
      weight: 10

---
# Flagger Canary Configuration
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: nginx-canary
spec:
  provider: istio
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  progressDeadlineSeconds: 60
  service:
    port: 80
  analysis:
    interval: 30s
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
    - name: request-success-rate
      threshold: 99
      interval: 1m
    - name: request-duration
      threshold: 500
      interval: 1m
    webhooks:
    - name: acceptance-test
      url: http://flagger-loadtester.test/acceptance
      timeout: 5s
    - name: load-test
      url: http://flagger-loadtester.test/load
      timeout: 5s
            

Canary Best Practices:

Start with very low traffic (1-5%) for high-risk changes
Monitor key metrics (error rate, latency) during canary phases
Use automated rollback on metric threshold breaches
Implement analysis phases with real user traffic
Use tools like Argo Rollouts, Flagger, or Istio for automation

A/B Testing with Service Mesh

A/B testing routes users to different versions based on request attributes (headers, cookies, user ID). This enables data-driven feature validation and user experience testing.

                # Istio A/B Testing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: app-ab
spec:
  hosts:
  - app-service
  http:
  - match:
    - headers:
        user-agent:
          regex: "^.*Mobile.*$"
    route:
    - destination:
        host: app-service
        subset: v2
      weight: 100
  - match:
    - headers:
        x-test-group:
          exact: "A"
    route:
    - destination:
        host: app-service
        subset: v1
      weight: 100
  - match:
    - headers:
        x-test-group:
          exact: "B"
    route:
    - destination:
        host: app-service
        subset: v2
      weight: 100
  - route:
    - destination:
        host: app-service
        subset: v1
      weight: 80
    - destination:
        host: app-service
        subset: v2
      weight: 20

---
# DestinationRule for subsets
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: app-dr
spec:
  host: app-service
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

# NGINX Ingress A/B Testing
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ab-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "x-test-group"
    nginx.ingress.kubernetes.io/canary-by-header-value: "B"
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-v2
            port:
              number: 80
            

A/B Testing Best Practices:

Use consistent hashing (based on user ID or session) to maintain user experience
Define clear success metrics for each test group
Monitor both technical metrics (latency, errors) and business metrics (conversion)
Set a time limit for tests to avoid stale configurations
Collect and analyze data before making decisions

Feature Flags (Feature Toggles)

Feature flags enable you to toggle features on and off without deploying new code. This enables dark launches, gradual rollouts, and instant rollback of features.

                # LaunchDarkly Feature Flag Example
# application code
from launchdarkly import LDClient

client = LDClient("sdk-xxx")

# Check feature flag
if client.variation("new-checkout-flow", user, False):
    # New checkout flow
    show_new_checkout()
else:
    # Old checkout flow
    show_old_checkout()

# Flagsmith Self-Hosted Feature Flags
apiVersion: v1
kind: ConfigMap
metadata:
  name: flagsmith-config
data:
  flagsmith.json: |
    {
      "features": [
        {
          "name": "new-checkout-flow",
          "enabled": false,
          "environments": {
            "production": false,
            "staging": true
          },
          "segments": [
            {
              "name": "beta-users",
              "condition": "user.email contains @example.com",
              "enabled": true
            }
          ]
        },
        {
          "name": "dark-mode",
          "enabled": false,
          "gradual_rollout": {
            "percentage": 10,
            "increment": 5,
            "schedule": "daily"
          }
        }
      ]
    }

# Kubernetes ConfigMap for feature flags
apiVersion: v1
kind: ConfigMap
metadata:
  name: feature-flags
data:
  new-feature-enabled: "false"
  beta-feature-enabled: "true"
  maintenance-mode: "false"

# Use in pod
apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  containers:
  - name: app
    image: myapp:latest
    env:
    - name: NEW_FEATURE_ENABLED
      valueFrom:
        configMapKeyRef:
          name: feature-flags
          key: new-feature-enabled
            

Feature Flag Best Practices:

Use feature flag management tools (LaunchDarkly, Flagsmith, Split)
Implement a clean-up process for old feature flags
Monitor performance impact of feature flag evaluation
Use feature flags for dark launches (release to internal users first)
Combine with canary deployment for progressive delivery

Deployment Strategy Comparison

Strategy	Zero Downtime	Risk Level	Rollback Speed	Infrastructure Cost	Complexity
Rolling Update	Yes	Medium	Medium	Low	Low
Blue-Green	Yes	Low	Instant	High (2x)	Medium
Canary	Yes	Very Low	Fast	Low-Medium	High
A/B Testing	Yes	Low	Fast	Low-Medium	High
Feature Flags	Yes	Very Low	Instant	Low	Medium
Shadow	Yes	Very Low	N/A	High (2x)	High

Progressive Delivery: The Modern Approach

Progressive Delivery is the evolution of continuous delivery that combines multiple strategies (canary, A/B testing, feature flags) with automated analysis to gradually roll out changes with minimal risk.

Automated Analysis

Metrics-based decisions

Use metrics (error rate, latency, throughput) to automatically decide whether to continue, rollback, or pause a rollout. Flagger and Argo Rollouts provide this capability.

Automated rollback, safe deployments

Observability Integration

Real-time feedback

Integrate with Prometheus, Grafana, and distributed tracing to monitor deployment health and user experience during canary phases.

Monitoring, validation

Automated Rollback

Self-healing deployments

Automatically rollback when metrics breach thresholds, reducing MTTR (Mean Time To Recovery) and eliminating manual intervention.

Resilience, automation

                # Flagger Progressive Delivery
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: app-canary
  namespace: production
spec:
  provider: istio
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app
  progressDeadlineSeconds: 60
  service:
    port: 80
    targetPort: 8080
  analysis:
    interval: 30s
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
    - name: request-success-rate
      thresholdRange:
        min: 99
      interval: 1m
    - name: request-duration
      thresholdRange:
        max: 500
      interval: 1m
    webhooks:
    - name: acceptance-test
      url: http://flagger-loadtester.test/acceptance
      timeout: 5s
    - name: load-test
      url: http://flagger-loadtester.test/load
      timeout: 5s
  rollback:
    retries: 3
    wait: 30s
            

Frequently Asked Questions

What is the difference between Canary and A/B Testing?

Canary deployments are about gradually rolling out a new version to a subset of users to validate stability and performance. A/B testing is about routing users to different versions based on attributes to measure user behavior and business metrics. Canary focuses on technical validation; A/B focuses on business validation.

When should I use Blue-Green vs. Rolling Update?

Use Blue-Green when you need instant rollback and can afford double infrastructure. Use Rolling Update for standard deployments where gradual rollout is acceptable. Blue-Green is preferred for critical applications where downtime is unacceptable.

What tools support advanced deployment strategies?

Argo Rollouts (canary, blue-green), Flagger (canary, A/B), Istio (traffic management), Linkerd (canary), NGINX Ingress (canary), and service meshes provide traffic control for advanced strategies.

How do I handle database migrations with deployments?

Use backward-compatible database changes. Apply migrations before deploying new versions. Consider using feature flags to enable database features gradually. For blue-green, ensure both versions can work with the same database schema.

What is progressive delivery?

Progressive delivery is the evolution of continuous delivery that combines canary releases, A/B testing, and feature flags with automated analysis. It reduces risk by gradually rolling out changes with real-time monitoring and automated rollback capabilities.

How do I implement canary deployment without a service mesh?

Use NGINX Ingress canary annotations, or use tools like Argo Rollouts with the "replica" traffic routing method. You can also use a simple approach with two deployments and a service selector, but this lacks fine-grained traffic control.

What's the difference between feature flags and deployment strategies?

Deployment strategies control how new versions of code are rolled out. Feature flags control which features are enabled for which users, independent of deployments. They can be used together: deploy code with flags, then gradually enable features.

How do I test canary deployments in production?

Use automated testing tools to validate canary versions. Implement metric thresholds for automatic rollback. Use real production traffic (canary should be small). Monitor both technical metrics (latency, errors) and business metrics (conversion).

Previous: Hybrid Cloud Architecture Next: GitOps with ArgoCD

Choosing the right deployment strategy is critical for maintaining reliability and reducing risk. Start with rolling updates, then adopt canary and progressive delivery as your maturity grows.