containerd Architecture
Deep dive into containerd's modular architecture. Understand how core components work together to provide a robust, extensible container runtime used by Docker and Kubernetes.
containerd follows a modular, pluggable architecture designed for flexibility and extensibility. It consists of a core daemon with gRPC API, surrounded by various components that can be swapped or extended based on requirements.
Content Store
Content-addressable storage for OCI images. Manages blobs, digests, and content distribution.
Metadata Store
BoltDB-based storage for namespaces, labels, and image references.
Image Service
Manages OCI images, pulling, pushing, and unpacking.
Snapshot Service
Manages filesystem snapshots using various drivers (overlayfs, native, etc.).
Task Service
Manages container execution and lifecycle operations.
Events Service
Publishes container lifecycle events for monitoring and debugging.
containerd exposes all functionality through a gRPC API, enabling programmatic access and embedding. The API is versioned and includes services for runtime, image, content, snapshot, and task management.
# containerd gRPC API services
service Runtime {
rpc Create(CreateRequest) returns (CreateResponse);
rpc Start(StartRequest) returns (StartResponse);
rpc Delete(DeleteRequest) returns (DeleteResponse);
}
service Images {
rpc Get(GetImageRequest) returns (GetImageResponse);
rpc List(ListImagesRequest) returns (ListImagesResponse);
rpc Pull(PullImageRequest) returns (PullImageResponse);
}
service Content {
rpc Info(InfoRequest) returns (InfoResponse);
rpc List(ListContentRequest) returns (ListContentResponse);
rpc Delete(DeleteContentRequest) returns (DeleteContentResponse);
}
# Test gRPC API with grpcurl
grpcurl -plaintext localhost:10000 containerd.services.images.v1.Images/List
Snapshotters manage container filesystem layers using copy-on-write (CoW) technologies. containerd supports multiple snapshot drivers:
overlayfs
Default driver, best performance, requires kernel 4.0+ with d_type support.
native
Simple copy-based driver, useful for testing or unsupported filesystems.
devicemapper
Thin provisioning, deprecated but still available for legacy systems.
zfs
Native ZFS snapshot support for ZFS filesystems.
btrfs
Native Btrfs snapshot support.
overlay
Legacy overlay driver (pre-overlayfs).
# List available snapshotters
ctr snapshot ls
# Use specific snapshotter
ctr image pull --snapshotter=zfs docker.io/library/nginx:alpine
# Configure default snapshotter in config.toml
[plugins."io.containerd.snapshotter.v1.overlayfs"]
root_path = "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs"
containerd supports multiple OCI-compliant runtimes through its plugin architecture. The default is runc, but you can also use Kata Containers, gVisor, or Firecracker.
# Configure different runtimes in config.toml
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
runtime_type = "io.containerd.kata.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.gvisor]
runtime_type = "io.containerd.runsc.v1"
# Use specific runtime
ctr run --runtime io.containerd.runc.v2 docker.io/library/alpine:latest test
Namespaces isolate resources (containers, images, snapshots) within a single containerd daemon. This enables multi-tenancy without running multiple daemons.
# List namespaces
ctr namespace ls
# Create namespace
ctr namespace create mytenant
# Run container in specific namespace
ctr -n mytenant run --rm docker.io/library/alpine:latest test echo "Hello"
# List containers in namespace
ctr -n mytenant container ls
# Kubernetes uses 'k8s.io' namespace
ctr -n k8s.io container ls
# Docker uses 'moby' namespace
ctr -n moby container ls
When you pull an image, containerd goes through a well-defined flow:
# Image pull flow
1. Client calls ImageService.Pull()
2. Content service fetches manifest from registry
3. Content service downloads blobs (layers)
4. Unpacker extracts layers to snapshotter
5. Image is registered in metadata store
# Under the hood
ctr image pull docker.io/library/nginx:alpine
# Equivalent steps
ctr content fetch docker.io/library/nginx:alpine
ctr image import nginx.tar
ctr snapshot prepare nginx-rootfs
# Container lifecycle stages
1. Create - Prepare container (no process running)
2. Start - Execute the container process
3. Pause - Freeze container processes
4. Resume - Unpause container
5. Stop - Terminate the container process
6. Delete - Remove container
# Commands
ctr container create docker.io/library/nginx:alpine nginx
ctr task start nginx
ctr task pause nginx
ctr task resume nginx
ctr task kill nginx
ctr container delete nginx
containerd publishes events for all state changes, enabling monitoring and automation.
# Subscribe to events
ctr events
# Event types
# - /containers/create
# - /containers/delete
# - /tasks/start
# - /tasks/exit
# - /images/create
# - /images/delete
# Subscribe to specific event types
ctr events --filter 'topic==/tasks/exit'
containerd exposes Prometheus metrics for monitoring container operations, resource usage, and daemon health.
# Enable metrics in config.toml
[metrics]
address = "0.0.0.0:1338"
# View metrics
curl http://localhost:1338/metrics
# Key metrics
# - container_runtime_operations_total
# - container_runtime_operations_errors_total
# - container_image_pull_duration_seconds
# - container_memory_usage_bytes
# - container_cpu_usage_seconds_total
# Prometheus scrape configuration
scrape_configs:
- job_name: 'containerd'
static_configs:
- targets: ['localhost:1338']
The CRI (Container Runtime Interface) plugin implements the Kubernetes container runtime API, allowing containerd to be used as the runtime for Kubernetes nodes.
# CRI plugin configuration in config.toml
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.k8s.io/pause:3.9"
[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "runc"
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
# Test CRI with crictl
crictl pull nginx:alpine
crictl images
crictl ps -a
Understanding containerd's architecture helps you leverage its full potential as a container runtime. Its modular design makes it both powerful and extensible.