Systemd inside Containers: Complete Guide to Running Systemd in Container Environments

Master running systemd inside containers with this comprehensive guide covering container init systems, PID 1 management, systemd-nspawn, Docker/Podman integration, and production-ready configurations for systemd-based containers.

Systemd Container Architecture Host System (systemd 252+) systemd (PID 1) systemd-nspawn machinectl Docker/Podman cgroups v2 Container with systemd (PID 1) systemd (containerized) Journal, cgroups, udev, logging Service Units nginx, sshd, cron Socket Units Activation, ports Timer Units Scheduled jobs Mount Units Filesystem mounts Journal Centralized logging cgroups Resource control Communication & Integration DBus Communication Host ↔ Container Journal Forwarding Log aggregation Socket Activation On-demand service start
Complete systemd container architecture showing systemd running as PID 1 inside container

Why Systemd in Containers?

Running systemd inside containers provides a full init system experience, enabling service management, logging, and process supervision within container environments.

  • Service Management: Proper init system for multi-service containers
  • Process Supervision: Automatic restart of failed services
  • Logging: Centralized journal logging with rotation
  • Dependency Management: Service dependencies and ordering
  • Socket Activation: On-demand service startup
  • Timer Services: Scheduled job execution
  • Resource Control: cgroups integration for resource limits
  • Security: SELinux/AppArmor integration

1. Systemd Container Fundamentals

🚀
Systemd as PID 1
/usr/lib/systemd/systemd
Full init system running as process ID 1 in container. Init Services
📦
systemd-nspawn
systemd-nspawn -D /container
Lightweight container runtime using systemd. Container
🔧
machinectl
machinectl start container
Manage systemd-nspawn containers and VMs. Management
📝
Journald
journalctl --machine=container
Centralized logging for container services. Logging
🔌
DBus
dbus-daemon --system
IPC system for service communication. IPC
🛡️
Security
systemd-analyze security
SELinux, capabilities, namespaces. Security

Container Init System Comparison

Init System PID 1 Service Management Logging Container Support Complexity
Systemd Full init ✅ Advanced ✅ Journal ✅ Excellent High
Supervisor Process manager ✅ Good ⚠️ Basic ✅ Good Medium
s6 Process supervision ✅ Good ⚠️ Basic ✅ Good Medium
runit Process supervision ✅ Good ⚠️ Basic ✅ Good Low
tini Minimal init ❌ None ❌ None ✅ Excellent Very Low
dumb-init Signal forwarding ❌ None ❌ None ✅ Good Very Low
Direct exec No init ❌ None ❌ None ⚠️ Limited None
Container Start
Systemd (PID 1)
Service Units
Host Integration

2. Running Systemd in Docker Containers

Basic Docker Systemd Configuration

# Dockerfile for systemd-based container
FROM ubuntu:22.04
# Install systemd and required packages
RUN apt-get update && apt-get install -y \
systemd \
systemd-sysv \
dbus \
sudo \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Create necessary directories for systemd
RUN mkdir -p /etc/systemd/system /run/systemd/system
# Mask unnecessary units
RUN systemctl mask \
dev-hugepages.mount \
sys-fs-fuse-connections.mount \
sys-kernel-config.mount \
sys-kernel-debug.mount \
display-manager.service \
getty@.service \
systemd-logind.service \
systemd-remount-fs.service \
udev.service \
-.mount
# Create systemd as entrypoint script
RUN echo '#!/bin/bash' > /usr/local/bin/startup.sh && \
echo 'exec /usr/lib/systemd/systemd' >> /usr/local/bin/startup.sh && \
chmod +x /usr/local/bin/startup.sh
# Expose port for services
EXPOSE 80 443
# Set systemd as entrypoint
ENTRYPOINT ["/usr/local/bin/startup.sh"]
# Build and run the container
docker build -t systemd-container .
docker run -d --name systemd-test \
--tmpfs /tmp \
--tmpfs /run \
--tmpfs /run/lock \
-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
--cap-add SYS_ADMIN \
systemd-container
# Execute commands inside the container
docker exec systemd-test systemctl status
docker exec systemd-test journalctl --no-pager
# Run with Docker Compose
version: '3.8'
services:
systemd-app:
build: .
container_name: systemd-app
tmpfs:
- /tmp
- /run
- /run/lock
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
cap_add:
- SYS_ADMIN
command: /usr/lib/systemd/systemd
restart: unless-stopped

Advanced Systemd Docker Configuration

advanced-systemd-docker.sh - Complete Production Setup
#!/bin/bash
# advanced-systemd-docker.sh - Production-ready systemd in Docker setup

set -euo pipefail

# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'

log() {
    echo -e "${GREEN}[+]${NC} $1"
}

error() {
    echo -e "${RED}[!]${NC} $1" >&2
    exit 1
}

create_dockerfile() {
    log "Creating advanced Dockerfile for systemd..."
    
    cat > Dockerfile << 'EOF'
# Production systemd container
FROM ubuntu:22.04

# Set environment variables
ENV container=docker \
    DEBIAN_FRONTEND=noninteractive \
    LANG=C.UTF-8 \
    LC_ALL=C.UTF-8

# Install systemd and required packages
RUN apt-get update && apt-get install -y \
    systemd \
    systemd-sysv \
    dbus \
    dbus-user-session \
    sudo \
    curl \
    wget \
    gnupg \
    lsb-release \
    ca-certificates \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Create necessary directories
RUN mkdir -p /etc/systemd/system /run/systemd/system \
    && mkdir -p /var/lib/systemd/coredump \
    && mkdir -p /etc/systemd/system.conf.d

# Configure systemd for container
RUN echo 'ReadWritePaths=-/var/log/journal' > /etc/systemd/journald.conf.d/container.conf \
    && echo 'Storage=volatile' >> /etc/systemd/journald.conf.d/container.conf \
    && echo 'ForwardToConsole=yes' >> /etc/systemd/journald.conf.d/container.conf

# Create custom systemd configuration
RUN cat > /etc/systemd/system.conf.d/container.conf << 'CONF'
[Manager]
DefaultStandardOutput=journal
DefaultStandardError=journal
LogLevel=info
LogTarget=journal
ShowStatus=yes
CONF

# Mask unnecessary services
RUN systemctl mask \
    systemd-udev-trigger.service \
    systemd-udevd.service \
    systemd-hwdb-update.service \
    systemd-tmpfiles-setup-dev.service \
    systemd-tmpfiles-clean.timer \
    systemd-update-utmp.service \
    systemd-random-seed.service \
    systemd-machine-id-commit.service \
    systemd-journal-flush.service \
    systemd-logind.service \
    getty@.service \
    display-manager.service \
    -.mount \
    dev-hugepages.mount \
    sys-fs-fuse-connections.mount \
    sys-kernel-config.mount \
    sys-kernel-debug.mount

# Create startup script
RUN cat > /usr/local/bin/startup.sh << 'SCRIPT'
#!/bin/bash
set -euo pipefail

# Initialize systemd
if [ ! -f /etc/machine-id ]; then
    systemd-machine-id-setup
fi

# Start dbus
if [ ! -d /run/dbus ]; then
    mkdir -p /run/dbus
fi
dbus-daemon --system --fork

# Start systemd
exec /usr/lib/systemd/systemd --log-target=journal --log-level=info
SCRIPT

RUN chmod +x /usr/local/bin/startup.sh

# Add custom service
RUN cat > /etc/systemd/system/custom-app.service << 'SERVICE'
[Unit]
Description=Custom Application
After=network.target
Requires=network.target

[Service]
Type=simple
ExecStart=/usr/bin/sleep infinity
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
SERVICE

RUN systemctl enable custom-app.service

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD systemctl is-system-running --quiet || exit 1

# Expose ports
EXPOSE 80 443

# Set entrypoint
ENTRYPOINT ["/usr/local/bin/startup.sh"]
CMD []
EOF
    
    log "Dockerfile created"
}

create_docker_compose() {
    log "Creating Docker Compose configuration..."
    
    cat > docker-compose.yml << 'EOF'
version: '3.8'

services:
  systemd-app:
    build: .
    container_name: systemd-app
    hostname: systemd-container
    domainname: local
    tmpfs:
      - /tmp
      - /run
      - /run/lock
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:ro
      - ./journal:/var/log/journal:rw
      - ./config:/etc/systemd/system.conf.d/custom.conf:ro
    cap_add:
      - SYS_ADMIN
      - SYS_RESOURCE
      - NET_ADMIN
    security_opt:
      - seccomp=unconfined
      - apparmor=unconfined
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0
      - kernel.msgmax=65536
    ulimits:
      nofile:
        soft: 65536
        hard: 65536
    restart: unless-stopped
    networks:
      - systemd-net

  systemd-nginx:
    image: nginx:alpine
    container_name: nginx-systemd
    depends_on:
      - systemd-app
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "8080:80"
    restart: unless-stopped
    networks:
      - systemd-net

networks:
  systemd-net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
EOF
    
    log "Docker Compose configuration created"
}

create_systemd_services() {
    log "Creating example systemd services..."
    
    mkdir -p services
    
    # Web service
    cat > services/webapp.service << 'EOF'
[Unit]
Description=Web Application
After=network.target
Requires=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/html
ExecStart=/usr/bin/python3 -m http.server 8080
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
Environment="PORT=8080"
Environment="HOST=0.0.0.0"

# Security
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadOnlyPaths=/
ReadWritePaths=/var/www/html

[Install]
WantedBy=multi-user.target
EOF

    # Database service
    cat > services/database.service << 'EOF'
[Unit]
Description=Database Service
After=network.target
Requires=network.target

[Service]
Type=forking
User=postgres
Group=postgres
ExecStart=/usr/lib/postgresql/bin/postgres -D /var/lib/postgresql/data
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
Restart=always
RestartSec=10
TimeoutSec=300

# Resource limits
LimitNOFILE=65536
LimitNPROC=256

# Security
PrivateTmp=yes
ProtectSystem=strict
NoNewPrivileges=yes

[Install]
WantedBy=multi-user.target
EOF

    # Timer service
    cat > services/cleanup.timer << 'EOF'
[Unit]
Description=Cleanup Timer

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target
EOF

    cat > services/cleanup.service << 'EOF'
[Unit]
Description=Cleanup Service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/cleanup.sh

[Install]
WantedBy=multi-user.target
EOF
    
    log "Example systemd services created"
}

build_and_run() {
    log "Building and running the container..."
    
    docker build -t production-systemd .
    
    docker run -d \
        --name production-systemd-container \
        --tmpfs /tmp \
        --tmpfs /run \
        --tmpfs /run/lock \
        -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
        -v $(pwd)/services:/etc/systemd/system:ro \
        -v $(pwd)/journal:/var/log/journal:rw \
        --cap-add SYS_ADMIN \
        --cap-add SYS_RESOURCE \
        --security-opt seccomp=unconfined \
        production-systemd
    
    log "Container started"
    log "Check status: docker exec production-systemd-container systemctl status"
    log "View logs: docker exec production-systemd-container journalctl -f"
}

verify_container() {
    log "Verifying container functionality..."
    
    echo "=== Systemd Status ==="
    docker exec production-systemd-container systemctl is-system-running
    
    echo -e "\n=== Active Services ==="
    docker exec production-systemd-container systemctl list-units --type=service --state=running
    
    echo -e "\n=== Journal Status ==="
    docker exec production-systemd-container journalctl --disk-usage
    
    echo -e "\n=== System Info ==="
    docker exec production-systemd-container hostnamectl
    
    echo -e "\n=== Security Analysis ==="
    docker exec production-systemd-container systemd-analyze security || true
}

main() {
    log "Setting up production systemd in Docker..."
    
    create_dockerfile
    create_docker_compose
    create_systemd_services
    build_and_run
    verify_container
    
    log "Setup completed!"
    log ""
    log "Next steps:"
    log "  1. Customize services in ./services/"
    log "  2. Configure networking as needed"
    log "  3. Set up logging aggregation"
    log "  4. Implement health checks"
    log "  5. Deploy to production environment"
}

main "$@"

3. Systemd-nspawn Container Management

Creating and Managing Systemd-nspawn Containers

# Systemd-nspawn Container Management
# Create a container directory
mkdir -p /var/lib/machines/ubuntu-container
# Debootstrap a minimal system
apt-get install debootstrap
debootstrap jammy /var/lib/machines/ubuntu-container http://archive.ubuntu.com/ubuntu/
# Basic container start
systemd-nspawn -D /var/lib/machines/ubuntu-container
# Start with network
systemd-nspawn -D /var/lib/machines/ubuntu-container --boot --network-veth
# Start with specific network bridge
systemd-nspawn -D /var/lib/machines/ubuntu-container --boot --network-bridge=br0
# Bind mount directories
systemd-nspawn -D /var/lib/machines/ubuntu-container --bind=/home/user/data:/data
# Start with capabilities
systemd-nspawn -D /var/lib/machines/ubuntu-container --capability=all
# Start with resource limits
systemd-nspawn -D /var/lib/machines/ubuntu-container --rlimit=CPU=1000:2000
# Using machinectl for management
machinectl start ubuntu-container
machinectl status ubuntu-container
machinectl login ubuntu-container
machinectl shell ubuntu-container /bin/bash
machinectl stop ubuntu-container
machinectl poweroff ubuntu-container
machinectl reboot ubuntu-container
machinectl list
machinectl show ubuntu-container
# Clone a container
machinectl clone ubuntu-container ubuntu-clone
# Export/Import containers
machinectl export-tar ubuntu-container ubuntu-container.tar
machinectl import-tar ubuntu-container.tar ubuntu-imported
# Create container from Docker image
machinectl pull-tar https://example.com/ubuntu.tar.gz ubuntu-from-tar
machinectl pull-raw https://example.com/ubuntu.raw ubuntu-from-raw
# Container configuration files
cat > /etc/systemd/nspawn/ubuntu-container.nspawn << EOF
[Exec]
Boot=yes
Capability=CAP_NET_ADMIN CAP_NET_RAW
[Network]
Bridge=br0
MACAddress=02:de:ad:be:ef:00
[Files]
Bind=/home/user/data:/data
EOF
# Journal integration
journalctl -M ubuntu-container
journalctl -M ubuntu-container -u nginx
# Automatic start on boot
systemctl enable systemd-nspawn@ubuntu-container
# Resource control with systemd
systemctl set-property systemd-nspawn@ubuntu-container MemoryLimit=512M
systemctl set-property systemd-nspawn@ubuntu-container CPUQuota=50%

Systemd-nspawn vs Docker Comparison

Feature Systemd-nspawn Docker Podman LXC/LXD
Init System ✅ Native systemd ⚠️ Custom/None ⚠️ Custom/None ✅ Custom
Journal Integration ✅ Full ⚠️ Limited ⚠️ Limited ❌ None
cgroups v2 ✅ Native ⚠️ Supported ✅ Good ✅ Good
Security ✅ SELinux/AppArmor ✅ Good ✅ Excellent ✅ Good
Networking ✅ systemd-networkd ✅ Advanced ✅ Good ✅ Advanced
Image Management ⚠️ Basic ✅ Excellent ✅ Excellent ✅ Good
Orchestration ❌ None ✅ Swarm/K8s ✅ Podman/K8s ⚠️ Limited
Portability ⚠️ Linux only ✅ Cross-platform ✅ Linux focus ✅ Linux focus

4. Podman with Systemd Integration

Podman Systemd Service Files

podman-systemd-integration.sh - Complete Podman Systemd Setup
#!/bin/bash
# podman-systemd-integration.sh - Podman containers managed by systemd

set -euo pipefail

log() {
    echo "[+] $1"
}

create_podman_systemd_service() {
    local container_name="$1"
    local image="$2"
    local port="$3"
    
    log "Creating systemd service for Podman container: $container_name"
    
    cat > /etc/systemd/system/podman-${container_name}.service << EOF
[Unit]
Description=Podman Container - ${container_name}
After=network.target
Requires=network.target
Wants=podman.socket

[Service]
Type=notify
NotifyAccess=all
TimeoutStartSec=300
ExecStartPre=/bin/rm -f /%t/%n-pid /%t/%n-cid
ExecStart=/usr/bin/podman run \
    --conmon-pidfile /%t/%n-pid \
    --cidfile /%t/%n-cid \
    --cgroups=split \
    --name=${container_name} \
    --hostname=${container_name} \
    --replace \
    --network=slirp4netns \
    --publish ${port}:${port} \
    --security-opt=no-new-privileges \
    --security-opt=label=disable \
    --volume /var/lib/${container_name}/data:/data:Z \
    ${image}
ExecStop=/usr/bin/podman stop --ignore --cidfile /%t/%n-cid -t 10
ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile /%t/%n-cid
PIDFile=/%t/%n-pid
KillMode=mixed
Delegate=yes
Restart=always
RestartSec=30

[Install]
WantedBy=multi-user.target default.target
EOF
    
    # Create data directory
    mkdir -p /var/lib/${container_name}/data
    chmod 755 /var/lib/${container_name}/data
    
    log "Systemd service created: podman-${container_name}.service"
}

create_podman_socket_service() {
    log "Creating Podman socket service..."
    
    cat > /etc/systemd/system/podman-api.service << EOF
[Unit]
Description=Podman API Service
Documentation=man:podman-system-service(1)
Wants=podman.socket
After=podman.socket

[Service]
Type=exec
Environment=PODMAN_SOCKET=%t/podman/podman.sock
ExecStart=/usr/bin/podman system service --time=0 \${PODMAN_SOCKET}
TimeoutSec=0

[Install]
WantedBy=multi-user.target
EOF

    cat > /etc/systemd/system/podman.socket << EOF
[Unit]
Description=Podman API Socket
Documentation=man:podman-system-service(1)

[Socket]
ListenStream=%t/podman/podman.sock
SocketMode=0660

[Install]
WantedBy=sockets.target
EOF
    
    systemctl daemon-reload
    systemctl enable --now podman.socket
    systemctl enable --now podman-api.service
    
    log "Podman API socket service configured"
}

create_rootless_podman_service() {
    local user="$1"
    local container_name="$2"
    local image="$3"
    
    log "Creating rootless Podman service for user: $user"
    
    # Create user-specific systemd directory
    mkdir -p /home/${user}/.config/systemd/user/
    
    cat > /home/${user}/.config/systemd/user/podman-${container_name}.service << EOF
[Unit]
Description=Rootless Podman Container - ${container_name}
After=network.target
Requires=network.target

[Service]
Type=simple
TimeoutStartSec=300
ExecStart=/usr/bin/podman run \
    --name=${container_name} \
    --hostname=${container_name} \
    --replace \
    --network=slirp4netns \
    --publish 8080:80 \
    --security-opt=no-new-privileges \
    --volume /home/${user}/data:/data:Z \
    ${image}
ExecStop=/usr/bin/podman stop --ignore ${container_name} -t 10
ExecStopPost=/usr/bin/podman rm --ignore -f ${container_name}
Restart=always
RestartSec=30

[Install]
WantedBy=default.target
EOF
    
    # Enable lingering for user
    loginctl enable-linger ${user}
    
    # Reload user systemd
    sudo -u ${user} systemctl --user daemon-reload
    sudo -u ${user} systemctl --user enable podman-${container_name}.service
    sudo -u ${user} systemctl --user start podman-${container_name}.service
    
    log "Rootless Podman service created for user: $user"
}

create_podman_pod_service() {
    local pod_name="$1"
    
    log "Creating systemd service for Podman pod: $pod_name"
    
    cat > /etc/systemd/system/podman-pod-${pod_name}.service << EOF
[Unit]
Description=Podman Pod - ${pod_name}
After=network.target
Requires=network.target
Wants=podman.socket

[Service]
Type=forking
TimeoutStartSec=300
PIDFile=/run/podman-pod-${pod_name}.pid
ExecStart=/usr/bin/podman pod create --name ${pod_name} --infra
ExecStartPost=/usr/bin/podman pod start ${pod_name}
ExecStop=/usr/bin/podman pod stop ${pod_name} -t 10
ExecStopPost=/usr/bin/podman pod rm ${pod_name} -f
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
EOF
    
    # Create containers within the pod
    cat > /etc/systemd/system/podman-${pod_name}-web.service << EOF
[Unit]
Description=Web Container in Pod ${pod_name}
After=podman-pod-${pod_name}.service
Requires=podman-pod-${pod_name}.service
BindsTo=podman-pod-${pod_name}.service

[Service]
Type=simple
TimeoutStartSec=300
ExecStart=/usr/bin/podman run \
    --pod ${pod_name} \
    --name ${pod_name}-web \
    --replace \
    nginx:alpine
ExecStop=/usr/bin/podman stop --ignore ${pod_name}-web -t 10
ExecStopPost=/usr/bin/podman rm --ignore -f ${pod_name}-web
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
EOF
    
    log "Podman pod service created: $pod_name"
}

setup_podman_systemd_integration() {
    log "Setting up Podman systemd integration..."
    
    # Install Podman
    if ! command -v podman &> /dev/null; then
        apt-get update
        apt-get install -y podman
    fi
    
    # Enable Podman socket
    systemctl enable --now podman.socket
    
    # Create example services
    create_podman_systemd_service "nginx" "nginx:alpine" "80"
    create_podman_systemd_service "postgres" "postgres:15" "5432"
    create_podman_systemd_service "redis" "redis:7" "6379"
    
    # Create pod
    create_podman_pod_service "app-pod"
    
    # Enable and start services
    systemctl daemon-reload
    systemctl enable podman-nginx.service
    systemctl enable podman-postgres.service
    systemctl enable podman-redis.service
    systemctl enable podman-pod-app-pod.service
    systemctl enable podman-app-pod-web.service
    
    systemctl start podman-nginx.service
    systemctl start podman-postgres.service
    systemctl start podman-redis.service
    systemctl start podman-pod-app-pod.service
    systemctl start podman-app-pod-web.service
    
    log "Podman systemd integration completed"
}

main() {
    log "Setting up Podman with systemd integration..."
    
    setup_podman_systemd_integration
    
    log "Podman systemd setup completed!"
    log ""
    log "Services created:"
    log "  • podman-nginx.service - Nginx web server"
    log "  • podman-postgres.service - PostgreSQL database"
    log "  • podman-redis.service - Redis cache"
    log "  • podman-pod-app-pod.service - Application pod"
    log "  • podman-app-pod-web.service - Web container in pod"
    log ""
    log "Management commands:"
    log "  systemctl status podman-nginx"
    log "  journalctl -u podman-nginx -f"
    log "  podman ps"
    log "  podman pod ps"
}

main "$@"

5. Kubernetes with Systemd Containers

Systemd-based Kubernetes Pods

# Kubernetes Pod with systemd as PID 1
apiVersion: v1
kind: Pod
metadata:
name: systemd-pod
spec:
hostPID: false
shareProcessNamespace: true
containers:
- name: systemd-container
image: ubuntu:22.04
command: ["/usr/lib/systemd/systemd"]
args: ["--log-target=journal"]
securityContext:
privileged: false
capabilities:
add: ["SYS_ADMIN", "SYS_RESOURCE"]
volumeMounts:
- name: cgroup
mountPath: /sys/fs/cgroup
readOnly: true
- name: tmp
mountPath: /tmp
- name: run
mountPath: /run
- name: run-lock
mountPath: /run/lock
volumes:
- name: cgroup
hostPath:
path: /sys/fs/cgroup
type: Directory
- name: tmp
emptyDir: {}
- name: run
emptyDir: {}
- name: run-lock
emptyDir: {}
# Kubernetes Deployment with systemd
apiVersion: apps/v1
kind: Deployment
metadata:
name: systemd-deployment
spec:
replicas: 3
selector:
matchLabels:
app: systemd-app
template:
metadata:
labels:
app: systemd-app
spec:
containers:
- name: main
image: custom-systemd:latest
command: ["/usr/lib/systemd/systemd"]
args: ["--log-level=info"]
securityContext:
capabilities:
add: ["SYS_ADMIN"]
volumeMounts:
- name: journal
mountPath: /var/log/journal
- name: cgroup
mountPath: /sys/fs/cgroup
readOnly: true
resources:
requests:
memory: "512Mi"
cpu: "500m"
limits:
memory: "1Gi"
cpu: "1"
livenessProbe:
exec:
command:
- systemctl
- is-system-running
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
exec:
command:
- systemctl
- is-system-running
- --quiet
initialDelaySeconds: 5
periodSeconds: 5
volumes:
- name: journal
emptyDir: {}
- name: cgroup
hostPath:
path: /sys/fs/cgroup
type: Directory
# Kubernetes Job with systemd
apiVersion: batch/v1
kind: Job
metadata:
name: systemd-job
spec:
template:
spec:
containers:
- name: job-runner
image: ubuntu:22.04
command: ["/bin/bash"]
args:
- -c
- |
apt-get update && apt-get install -y systemd
/usr/lib/systemd/systemd --system &
sleep 2
systemctl start custom-job.service
wait
securityContext:
capabilities:
add: ["SYS_ADMIN"]
volumeMounts:
- name: cgroup
mountPath: /sys/fs/cgroup
readOnly: true
volumes:
- name: cgroup
hostPath:
path: /sys/fs/cgroup
type: Directory
restartPolicy: Never
backoffLimit: 3
# Kubernetes DaemonSet with systemd
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: systemd-daemonset
spec:
selector:
matchLabels:
name: systemd-daemon
template:
metadata:
labels:
name: systemd-daemon
spec:
hostNetwork: true
hostPID: true
containers:
- name: systemd-container
image: custom-systemd:latest
command: ["/usr/lib/systemd/systemd"]
securityContext:
privileged: true
volumeMounts:
- name: cgroup
mountPath: /sys/fs/cgroup
- name: var-log
mountPath: /var/log
- name: run
mountPath: /run
volumes:
- name: cgroup
hostPath:
path: /sys/fs/cgroup
- name: var-log
hostPath:
path: /var/log
- name: run
hostPath:
path: /run
# Apply Kubernetes manifests
kubectl apply -f systemd-pod.yaml
kubectl apply -f systemd-deployment.yaml
kubectl apply -f systemd-job.yaml
kubectl apply -f systemd-daemonset.yaml

6. Production Systemd Container Patterns

Systemd Container Best Practices:
1. Minimal systemd: Install only required systemd components
2. Mask unnecessary services: Reduce attack surface
3. Use tmpfs: Mount /tmp, /run as tmpfs for performance
4. Journal configuration: Configure journal for containers
5. Security hardening: Drop capabilities, use namespaces
6. Resource limits: Set cgroup limits for resources
7. Health checks: Implement systemd-based health checks
8. Log aggregation: Forward logs to central system
9. Update strategy: Plan for systemd updates in containers
10. Monitoring: Monitor systemd service states and journals

Production Systemd Container Configuration

production-systemd-container.sh - Enterprise Systemd Container Setup
#!/bin/bash
# production-systemd-container.sh - Enterprise-grade systemd container configuration

set -euo pipefail

# Configuration
CONTAINER_NAME="enterprise-systemd"
IMAGE_BASE="ubuntu:22.04"
SYSTEMD_VERSION="252"
JOURNAL_MAX_SIZE="100M"
MEMORY_LIMIT="1G"
CPU_LIMIT="1.0"

log() {
    echo "[+] $1"
}

create_enterprise_dockerfile() {
    log "Creating enterprise Dockerfile..."
    
    cat > Dockerfile.enterprise << 'EOF'
# Enterprise systemd container
FROM ubuntu:22.04 AS builder

# Install build dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    git \
    meson \
    ninja-build \
    pkg-config \
    && rm -rf /var/lib/apt/lists/*

# Build custom systemd (optional)
# RUN git clone https://github.com/systemd/systemd.git \
#     && cd systemd \
#     && meson build \
#     && ninja -C build \
#     && ninja -C build install

FROM ubuntu:22.04

# Metadata
LABEL org.label-schema.name="Enterprise Systemd Container" \
      org.label-schema.description="Production-ready systemd container" \
      org.label-schema.vendor="Enterprise Inc." \
      org.label-schema.version="1.0" \
      org.label-schema.schema-version="1.0"

# Environment
ENV container=docker \
    DEBIAN_FRONTEND=noninteractive \
    SYSTEMD_OPTS="--log-target=journal --log-level=info" \
    JOURNAL_OPTS="--storage=volatile --max-size=100M"

# Install systemd and enterprise packages
RUN apt-get update && apt-get install -y \
    systemd \
    systemd-sysv \
    systemd-timesyncd \
    dbus \
    dbus-user-session \
    sudo \
    curl \
    wget \
    gnupg \
    ca-certificates \
    apt-transport-https \
    software-properties-common \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Create enterprise directory structure
RUN mkdir -p \
    /etc/enterprise \
    /var/log/enterprise \
    /var/lib/enterprise \
    /run/enterprise \
    && chmod 755 /etc/enterprise /var/log/enterprise /var/lib/enterprise /run/enterprise

# Systemd configuration
RUN mkdir -p /etc/systemd/{system,system.conf.d,journald.conf.d,logind.conf.d}

# Journal configuration
RUN cat > /etc/systemd/journald.conf.d/enterprise.conf << 'JOURNAL'
[Journal]
Storage=volatile
Compress=yes
Seal=yes
SplitMode=uid
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
SystemMaxUse=100M
RuntimeMaxUse=100M
MaxRetentionSec=1month
MaxFileSec=1day
ForwardToConsole=no
ForwardToSyslog=no
ForwardToWall=no
TTYPath=/dev/console
MaxLevelStore=debug
MaxLevelSyslog=debug
MaxLevelKMsg=notice
MaxLevelConsole=info
MaxLevelWall=emerg
JOURNAL

# Systemd security hardening
RUN cat > /etc/systemd/system.conf.d/security.conf << 'SECURITY'
[Manager]
DefaultTimeoutStartSec=90s
DefaultTimeoutStopSec=90s
DefaultRestartSec=100ms
DefaultStartLimitIntervalSec=10s
DefaultStartLimitBurst=5
RuntimeWatchdogSec=10min
RebootWatchdogSec=10min
KExecWatchdogSec=10min
WatchdogDevice=/dev/watchdog
CapabilityBoundingSet=CAP_AUDIT_WRITE CAP_KILL CAP_NET_BIND_SERVICE CAP_SYS_TIME CAP_SYSLOG
NoNewPrivileges=yes
LockPersonality=yes
PrivateDevices=yes
PrivateMounts=yes
PrivateTmp=yes
PrivateUsers=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=no
ProtectSystem=strict
ReadWritePaths=/var/log/journal /var/log/enterprise /var/lib/enterprise
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
UMask=0027
SECURITY

# Create enterprise services
RUN cat > /etc/systemd/system/enterprise-health.service << 'SERVICE'
[Unit]
Description=Enterprise Health Check Service
After=network.target
Wants=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/health-check.sh
ExecReload=/usr/local/bin/health-check.sh
TimeoutStartSec=30
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target
SERVICE

RUN cat > /etc/systemd/system/enterprise-health.timer << 'TIMER'
[Unit]
Description=Enterprise Health Check Timer

[Timer]
OnBootSec=1min
OnUnitActiveSec=5min
AccuracySec=1s
RandomizedDelaySec=30s

[Install]
WantedBy=timers.target
TIMER

# Create health check script
RUN cat > /usr/local/bin/health-check.sh << 'HEALTH'
#!/bin/bash
set -euo pipefail

# Health check script for enterprise container
STATUS_FILE="/run/enterprise/health.status"
TIMESTAMP=$(date +%s)

# Check systemd status
if systemctl is-system-running --quiet; then
    echo "OK: System is running" > "$STATUS_FILE"
    echo "$TIMESTAMP:healthy" >> "/var/log/enterprise/health.log"
    exit 0
else
    echo "ERROR: System not running properly" > "$STATUS_FILE"
    echo "$TIMESTAMP:unhealthy" >> "/var/log/enterprise/health.log"
    exit 1
fi
HEALTH

RUN chmod +x /usr/local/bin/health-check.sh

# Create startup script
RUN cat > /usr/local/bin/enterprise-start.sh << 'STARTUP'
#!/bin/bash
set -euo pipefail

# Initialize container
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a /var/log/enterprise/startup.log
}

log "Starting enterprise systemd container..."

# Initialize machine ID
if [ ! -f /etc/machine-id ] || [ ! -s /etc/machine-id ]; then
    log "Initializing machine ID..."
    systemd-machine-id-setup
fi

# Start dbus
if [ ! -d /run/dbus ]; then
    mkdir -p /run/dbus
fi
log "Starting D-Bus..."
dbus-daemon --system --fork

# Set environment for systemd
export SYSTEMD_PAGER=cat
export SYSTEMD_COLORS=never

# Start systemd
log "Starting systemd..."
exec /usr/lib/systemd/systemd $SYSTEMD_OPTS
STARTUP

RUN chmod +x /usr/local/bin/enterprise-start.sh

# Enable services
RUN systemctl enable enterprise-health.service \
    && systemctl enable enterprise-health.timer

# Cleanup
RUN systemd-tmpfiles --clean \
    && systemd-tmpfiles --remove

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
    CMD /usr/local/bin/health-check.sh

# Ports
EXPOSE 80 443 8080

# Entrypoint
ENTRYPOINT ["/usr/local/bin/enterprise-start.sh"]
CMD []
EOF
    
    log "Enterprise Dockerfile created"
}

create_kubernetes_manifests() {
    log "Creating Kubernetes manifests for enterprise systemd..."
    
    cat > kubernetes/deployment.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: enterprise-systemd
  namespace: enterprise
  labels:
    app: enterprise-systemd
    component: systemd-container
spec:
  replicas: 3
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: enterprise-systemd
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: enterprise-systemd
        component: systemd-container
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9100"
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: enterprise-systemd
        image: enterprise-systemd:latest
        imagePullPolicy: IfNotPresent
        command: ["/usr/local/bin/enterprise-start.sh"]
        securityContext:
          privileged: false
          allowPrivilegeEscalation: false
          capabilities:
            drop:
              - ALL
            add:
              - SYS_ADMIN
              - SYS_RESOURCE
          readOnlyRootFilesystem: true
        ports:
        - name: http
          containerPort: 80
          protocol: TCP
        - name: https
          containerPort: 443
          protocol: TCP
        - name: metrics
          containerPort: 9100
          protocol: TCP
        env:
        - name: SYSTEMD_OPTS
          value: "--log-target=journal --log-level=info"
        - name: CONTAINER_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1"
        volumeMounts:
        - name: cgroup
          mountPath: /sys/fs/cgroup
          readOnly: true
        - name: journal
          mountPath: /var/log/journal
        - name: enterprise-data
          mountPath: /var/lib/enterprise
        - name: tmp
          mountPath: /tmp
        - name: run
          mountPath: /run
        - name: run-lock
          mountPath: /run/lock
        livenessProbe:
          exec:
            command:
            - /usr/local/bin/health-check.sh
          initialDelaySeconds: 60
          periodSeconds: 30
          timeoutSeconds: 10
          failureThreshold: 3
        readinessProbe:
          exec:
            command:
            - systemctl
            - is-system-running
            - --quiet
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
        startupProbe:
          exec:
            command:
            - systemctl
            - is-system-running
            - --quiet
          initialDelaySeconds: 10
          periodSeconds: 5
          failureThreshold: 30
        lifecycle:
          postStart:
            exec:
              command:
              - /bin/sh
              - -c
              - |
                echo "Container started at $(date)" >> /var/log/enterprise/lifecycle.log
          preStop:
            exec:
              command:
              - /bin/sh
              - -c
              - |
                echo "Container stopping at $(date)" >> /var/log/enterprise/lifecycle.log
                systemctl stop --all
      volumes:
      - name: cgroup
        hostPath:
          path: /sys/fs/cgroup
          type: Directory
      - name: journal
        emptyDir:
          sizeLimit: 100Mi
      - name: enterprise-data
        emptyDir:
          sizeLimit: 1Gi
      - name: tmp
        emptyDir:
          sizeLimit: 100Mi
      - name: run
        emptyDir:
          sizeLimit: 50Mi
      - name: run-lock
        emptyDir:
          sizeLimit: 10Mi
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - enterprise-systemd
              topologyKey: kubernetes.io/hostname
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - amd64
                - arm64
      tolerations:
      - key: "node-role.kubernetes.io/master"
        operator: "Exists"
        effect: "NoSchedule"
---
apiVersion: v1
kind: Service
metadata:
  name: enterprise-systemd
  namespace: enterprise
  labels:
    app: enterprise-systemd
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
  selector:
    app: enterprise-systemd
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: enterprise-systemd
  namespace: enterprise
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: enterprise-systemd
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
EOF
    
    log "Kubernetes manifests created"
}

create_monitoring_config() {
    log "Creating monitoring configuration..."
    
    cat > monitoring/prometheus.yml << 'EOF'
scrape_configs:
  - job_name: 'enterprise-systemd'
    scrape_interval: 30s
    scrape_timeout: 10s
    metrics_path: /metrics
    static_configs:
      - targets: ['enterprise-systemd.enterprise.svc.cluster.local:9100']
    relabel_configs:
      - source_labels: [__address__]
        target_label: instance
      - source_labels: [__meta_kubernetes_pod_name]
        target_label: pod
EOF

    cat > monitoring/grafana-dashboard.json << 'EOF'
{
  "dashboard": {
    "title": "Enterprise Systemd Dashboard",
    "panels": [
      {
        "title": "Systemd Services Status",
        "type": "stat",
        "targets": [
          {
            "expr": "systemd_units_active{name=~\"enterprise-.*\"}",
            "legendFormat": "{{name}}"
          }
        ]
      }
    ]
  }
}
EOF
    
    log "Monitoring configuration created"
}

main() {
    log "Setting up enterprise systemd container solution..."
    
    # Create directory structure
    mkdir -p kubernetes monitoring
    
    # Create configurations
    create_enterprise_dockerfile
    create_kubernetes_manifests
    create_monitoring_config
    
    log "Enterprise systemd container setup completed!"
    log ""
    log "Next steps:"
    log "  1. Build the container: docker build -f Dockerfile.enterprise -t enterprise-systemd ."
    log "  2. Push to registry: docker push registry/enterprise-systemd:latest"
    log "  3. Deploy to Kubernetes: kubectl apply -f kubernetes/"
    log "  4. Set up monitoring: kubectl apply -f monitoring/"
    log "  5. Configure CI/CD pipeline"
    log ""
    log "Features included:"
    log "  • Production-ready systemd configuration"
    log "  • Security hardening"
    log "  • Health checks and monitoring"
    log "  • Kubernetes manifests"
    log "  • Resource management"
    log "  • Logging and journal configuration"
}

main "$@"

7. Troubleshooting Systemd in Containers

Common Issues & Solutions

Issue Symptoms Root Cause Solution
Systemd fails to start Container exits immediately Missing cgroups mount, capabilities Mount /sys/fs/cgroup, add SYS_ADMIN capability
Journal not working No logs, journalctl fails Missing /run/log/journal Create directory, configure journal storage
DBus errors Service startup failures DBus daemon not running Start dbus-daemon before systemd
Socket activation fails Services not starting on connection Missing socket units or permissions Check socket unit files and permissions
Timer services not running Scheduled jobs not executing Systemd timers not enabled Enable and start timer units
Resource limits ignored Container exceeds limits cgroups v2 not properly mounted Mount cgroups with proper permissions
Security policy violations SELinux/AppArmor denials Container not configured for security modules Adjust security policies or run unconfined
Network services fail Can't bind to ports NET_BIND_SERVICE capability missing Add capability or run as root

Systemd Container Debugging Toolkit

# Systemd Container Debugging Commands
# Basic systemd status
systemctl is-system-running
systemctl status
systemctl list-units --type=service
systemctl list-units --failed
systemctl list-unit-files
# Journal debugging
journalctl --no-pager
journalctl -f
journalctl --since "1 hour ago"
journalctl -u service-name
journalctl --disk-usage
journalctl --verify
# Systemd analysis
systemd-analyze time
systemd-analyze blame
systemd-analyze critical-chain
systemd-analyze plot > boot.svg
systemd-analyze security
systemd-analyze verify /etc/systemd/system/*.service
# Service debugging
systemctl status service-name
systemctl cat service-name
systemctl show service-name
systemd-run --unit=debug-service /bin/bash
systemctl daemon-reload
systemctl reset-failed
# Container-specific debugging
machinectl list
machinectl status container-name
machinectl shell container-name
journalctl -M container-name
systemd-cgls
systemd-cgtop
# cgroups inspection
cat /proc/1/cgroup
ls -la /sys/fs/cgroup/
systemd-cgls
systemd-cgtop
# DBus debugging
dbus-send --system --print-reply --dest=org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager.ListUnits
busctl list
busctl status org.freedesktop.systemd1
# Security debugging
systemd-analyze security service-name
capsh --print
getpcaps 1
ausearch -m avc -ts recent
# Network debugging
networkctl list
networkctl status
resolvectl status

Master Systemd in Containers

Running systemd inside containers provides a powerful init system for managing services, logging, and resource control within containerized environments. By understanding systemd-nspawn, Docker/Podman integration, and production configurations, you can build robust container platforms.

Key Takeaways: Systemd provides full init system capabilities in containers. Use systemd-nspawn for lightweight systemd-native containers. Integrate systemd with Docker using proper volume mounts and capabilities. Leverage Podman's native systemd integration. Implement production patterns with security hardening and monitoring. Use systemd's journal for centralized logging. Implement health checks and service management.

Next Steps: Experiment with systemd-nspawn containers. Build Docker images with systemd as PID 1. Implement Podman systemd services. Deploy systemd-based containers to Kubernetes. Set up monitoring and logging for systemd containers. Implement security hardening practices. Explore advanced systemd features like socket activation and timers in containers.