Master running systemd inside containers with this comprehensive guide covering container init systems, PID 1 management, systemd-nspawn, Docker/Podman integration, and production-ready configurations for systemd-based containers.
Why Systemd in Containers?
Running systemd inside containers provides a full init system experience, enabling service management, logging, and process supervision within container environments.
- Service Management: Proper init system for multi-service containers
- Process Supervision: Automatic restart of failed services
- Logging: Centralized journal logging with rotation
- Dependency Management: Service dependencies and ordering
- Socket Activation: On-demand service startup
- Timer Services: Scheduled job execution
- Resource Control: cgroups integration for resource limits
- Security: SELinux/AppArmor integration
1. Systemd Container Fundamentals
Container Init System Comparison
| Init System | PID 1 | Service Management | Logging | Container Support | Complexity |
|---|---|---|---|---|---|
| Systemd | Full init | ✅ Advanced | ✅ Journal | ✅ Excellent | High |
| Supervisor | Process manager | ✅ Good | ⚠️ Basic | ✅ Good | Medium |
| s6 | Process supervision | ✅ Good | ⚠️ Basic | ✅ Good | Medium |
| runit | Process supervision | ✅ Good | ⚠️ Basic | ✅ Good | Low |
| tini | Minimal init | ❌ None | ❌ None | ✅ Excellent | Very Low |
| dumb-init | Signal forwarding | ❌ None | ❌ None | ✅ Good | Very Low |
| Direct exec | No init | ❌ None | ❌ None | ⚠️ Limited | None |
2. Running Systemd in Docker Containers
Basic Docker Systemd Configuration
Advanced Systemd Docker Configuration
#!/bin/bash
# advanced-systemd-docker.sh - Production-ready systemd in Docker setup
set -euo pipefail
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'
log() {
echo -e "${GREEN}[+]${NC} $1"
}
error() {
echo -e "${RED}[!]${NC} $1" >&2
exit 1
}
create_dockerfile() {
log "Creating advanced Dockerfile for systemd..."
cat > Dockerfile << 'EOF'
# Production systemd container
FROM ubuntu:22.04
# Set environment variables
ENV container=docker \
DEBIAN_FRONTEND=noninteractive \
LANG=C.UTF-8 \
LC_ALL=C.UTF-8
# Install systemd and required packages
RUN apt-get update && apt-get install -y \
systemd \
systemd-sysv \
dbus \
dbus-user-session \
sudo \
curl \
wget \
gnupg \
lsb-release \
ca-certificates \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Create necessary directories
RUN mkdir -p /etc/systemd/system /run/systemd/system \
&& mkdir -p /var/lib/systemd/coredump \
&& mkdir -p /etc/systemd/system.conf.d
# Configure systemd for container
RUN echo 'ReadWritePaths=-/var/log/journal' > /etc/systemd/journald.conf.d/container.conf \
&& echo 'Storage=volatile' >> /etc/systemd/journald.conf.d/container.conf \
&& echo 'ForwardToConsole=yes' >> /etc/systemd/journald.conf.d/container.conf
# Create custom systemd configuration
RUN cat > /etc/systemd/system.conf.d/container.conf << 'CONF'
[Manager]
DefaultStandardOutput=journal
DefaultStandardError=journal
LogLevel=info
LogTarget=journal
ShowStatus=yes
CONF
# Mask unnecessary services
RUN systemctl mask \
systemd-udev-trigger.service \
systemd-udevd.service \
systemd-hwdb-update.service \
systemd-tmpfiles-setup-dev.service \
systemd-tmpfiles-clean.timer \
systemd-update-utmp.service \
systemd-random-seed.service \
systemd-machine-id-commit.service \
systemd-journal-flush.service \
systemd-logind.service \
getty@.service \
display-manager.service \
-.mount \
dev-hugepages.mount \
sys-fs-fuse-connections.mount \
sys-kernel-config.mount \
sys-kernel-debug.mount
# Create startup script
RUN cat > /usr/local/bin/startup.sh << 'SCRIPT'
#!/bin/bash
set -euo pipefail
# Initialize systemd
if [ ! -f /etc/machine-id ]; then
systemd-machine-id-setup
fi
# Start dbus
if [ ! -d /run/dbus ]; then
mkdir -p /run/dbus
fi
dbus-daemon --system --fork
# Start systemd
exec /usr/lib/systemd/systemd --log-target=journal --log-level=info
SCRIPT
RUN chmod +x /usr/local/bin/startup.sh
# Add custom service
RUN cat > /etc/systemd/system/custom-app.service << 'SERVICE'
[Unit]
Description=Custom Application
After=network.target
Requires=network.target
[Service]
Type=simple
ExecStart=/usr/bin/sleep infinity
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
SERVICE
RUN systemctl enable custom-app.service
# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD systemctl is-system-running --quiet || exit 1
# Expose ports
EXPOSE 80 443
# Set entrypoint
ENTRYPOINT ["/usr/local/bin/startup.sh"]
CMD []
EOF
log "Dockerfile created"
}
create_docker_compose() {
log "Creating Docker Compose configuration..."
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
systemd-app:
build: .
container_name: systemd-app
hostname: systemd-container
domainname: local
tmpfs:
- /tmp
- /run
- /run/lock
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
- ./journal:/var/log/journal:rw
- ./config:/etc/systemd/system.conf.d/custom.conf:ro
cap_add:
- SYS_ADMIN
- SYS_RESOURCE
- NET_ADMIN
security_opt:
- seccomp=unconfined
- apparmor=unconfined
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
- kernel.msgmax=65536
ulimits:
nofile:
soft: 65536
hard: 65536
restart: unless-stopped
networks:
- systemd-net
systemd-nginx:
image: nginx:alpine
container_name: nginx-systemd
depends_on:
- systemd-app
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "8080:80"
restart: unless-stopped
networks:
- systemd-net
networks:
systemd-net:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/16
EOF
log "Docker Compose configuration created"
}
create_systemd_services() {
log "Creating example systemd services..."
mkdir -p services
# Web service
cat > services/webapp.service << 'EOF'
[Unit]
Description=Web Application
After=network.target
Requires=network.target
[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/html
ExecStart=/usr/bin/python3 -m http.server 8080
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
Environment="PORT=8080"
Environment="HOST=0.0.0.0"
# Security
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadOnlyPaths=/
ReadWritePaths=/var/www/html
[Install]
WantedBy=multi-user.target
EOF
# Database service
cat > services/database.service << 'EOF'
[Unit]
Description=Database Service
After=network.target
Requires=network.target
[Service]
Type=forking
User=postgres
Group=postgres
ExecStart=/usr/lib/postgresql/bin/postgres -D /var/lib/postgresql/data
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
Restart=always
RestartSec=10
TimeoutSec=300
# Resource limits
LimitNOFILE=65536
LimitNPROC=256
# Security
PrivateTmp=yes
ProtectSystem=strict
NoNewPrivileges=yes
[Install]
WantedBy=multi-user.target
EOF
# Timer service
cat > services/cleanup.timer << 'EOF'
[Unit]
Description=Cleanup Timer
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
EOF
cat > services/cleanup.service << 'EOF'
[Unit]
Description=Cleanup Service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/cleanup.sh
[Install]
WantedBy=multi-user.target
EOF
log "Example systemd services created"
}
build_and_run() {
log "Building and running the container..."
docker build -t production-systemd .
docker run -d \
--name production-systemd-container \
--tmpfs /tmp \
--tmpfs /run \
--tmpfs /run/lock \
-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
-v $(pwd)/services:/etc/systemd/system:ro \
-v $(pwd)/journal:/var/log/journal:rw \
--cap-add SYS_ADMIN \
--cap-add SYS_RESOURCE \
--security-opt seccomp=unconfined \
production-systemd
log "Container started"
log "Check status: docker exec production-systemd-container systemctl status"
log "View logs: docker exec production-systemd-container journalctl -f"
}
verify_container() {
log "Verifying container functionality..."
echo "=== Systemd Status ==="
docker exec production-systemd-container systemctl is-system-running
echo -e "\n=== Active Services ==="
docker exec production-systemd-container systemctl list-units --type=service --state=running
echo -e "\n=== Journal Status ==="
docker exec production-systemd-container journalctl --disk-usage
echo -e "\n=== System Info ==="
docker exec production-systemd-container hostnamectl
echo -e "\n=== Security Analysis ==="
docker exec production-systemd-container systemd-analyze security || true
}
main() {
log "Setting up production systemd in Docker..."
create_dockerfile
create_docker_compose
create_systemd_services
build_and_run
verify_container
log "Setup completed!"
log ""
log "Next steps:"
log " 1. Customize services in ./services/"
log " 2. Configure networking as needed"
log " 3. Set up logging aggregation"
log " 4. Implement health checks"
log " 5. Deploy to production environment"
}
main "$@"
3. Systemd-nspawn Container Management
Creating and Managing Systemd-nspawn Containers
Systemd-nspawn vs Docker Comparison
| Feature | Systemd-nspawn | Docker | Podman | LXC/LXD |
|---|---|---|---|---|
| Init System | ✅ Native systemd | ⚠️ Custom/None | ⚠️ Custom/None | ✅ Custom |
| Journal Integration | ✅ Full | ⚠️ Limited | ⚠️ Limited | ❌ None |
| cgroups v2 | ✅ Native | ⚠️ Supported | ✅ Good | ✅ Good |
| Security | ✅ SELinux/AppArmor | ✅ Good | ✅ Excellent | ✅ Good |
| Networking | ✅ systemd-networkd | ✅ Advanced | ✅ Good | ✅ Advanced |
| Image Management | ⚠️ Basic | ✅ Excellent | ✅ Excellent | ✅ Good |
| Orchestration | ❌ None | ✅ Swarm/K8s | ✅ Podman/K8s | ⚠️ Limited |
| Portability | ⚠️ Linux only | ✅ Cross-platform | ✅ Linux focus | ✅ Linux focus |
4. Podman with Systemd Integration
Podman Systemd Service Files
#!/bin/bash
# podman-systemd-integration.sh - Podman containers managed by systemd
set -euo pipefail
log() {
echo "[+] $1"
}
create_podman_systemd_service() {
local container_name="$1"
local image="$2"
local port="$3"
log "Creating systemd service for Podman container: $container_name"
cat > /etc/systemd/system/podman-${container_name}.service << EOF
[Unit]
Description=Podman Container - ${container_name}
After=network.target
Requires=network.target
Wants=podman.socket
[Service]
Type=notify
NotifyAccess=all
TimeoutStartSec=300
ExecStartPre=/bin/rm -f /%t/%n-pid /%t/%n-cid
ExecStart=/usr/bin/podman run \
--conmon-pidfile /%t/%n-pid \
--cidfile /%t/%n-cid \
--cgroups=split \
--name=${container_name} \
--hostname=${container_name} \
--replace \
--network=slirp4netns \
--publish ${port}:${port} \
--security-opt=no-new-privileges \
--security-opt=label=disable \
--volume /var/lib/${container_name}/data:/data:Z \
${image}
ExecStop=/usr/bin/podman stop --ignore --cidfile /%t/%n-cid -t 10
ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile /%t/%n-cid
PIDFile=/%t/%n-pid
KillMode=mixed
Delegate=yes
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target default.target
EOF
# Create data directory
mkdir -p /var/lib/${container_name}/data
chmod 755 /var/lib/${container_name}/data
log "Systemd service created: podman-${container_name}.service"
}
create_podman_socket_service() {
log "Creating Podman socket service..."
cat > /etc/systemd/system/podman-api.service << EOF
[Unit]
Description=Podman API Service
Documentation=man:podman-system-service(1)
Wants=podman.socket
After=podman.socket
[Service]
Type=exec
Environment=PODMAN_SOCKET=%t/podman/podman.sock
ExecStart=/usr/bin/podman system service --time=0 \${PODMAN_SOCKET}
TimeoutSec=0
[Install]
WantedBy=multi-user.target
EOF
cat > /etc/systemd/system/podman.socket << EOF
[Unit]
Description=Podman API Socket
Documentation=man:podman-system-service(1)
[Socket]
ListenStream=%t/podman/podman.sock
SocketMode=0660
[Install]
WantedBy=sockets.target
EOF
systemctl daemon-reload
systemctl enable --now podman.socket
systemctl enable --now podman-api.service
log "Podman API socket service configured"
}
create_rootless_podman_service() {
local user="$1"
local container_name="$2"
local image="$3"
log "Creating rootless Podman service for user: $user"
# Create user-specific systemd directory
mkdir -p /home/${user}/.config/systemd/user/
cat > /home/${user}/.config/systemd/user/podman-${container_name}.service << EOF
[Unit]
Description=Rootless Podman Container - ${container_name}
After=network.target
Requires=network.target
[Service]
Type=simple
TimeoutStartSec=300
ExecStart=/usr/bin/podman run \
--name=${container_name} \
--hostname=${container_name} \
--replace \
--network=slirp4netns \
--publish 8080:80 \
--security-opt=no-new-privileges \
--volume /home/${user}/data:/data:Z \
${image}
ExecStop=/usr/bin/podman stop --ignore ${container_name} -t 10
ExecStopPost=/usr/bin/podman rm --ignore -f ${container_name}
Restart=always
RestartSec=30
[Install]
WantedBy=default.target
EOF
# Enable lingering for user
loginctl enable-linger ${user}
# Reload user systemd
sudo -u ${user} systemctl --user daemon-reload
sudo -u ${user} systemctl --user enable podman-${container_name}.service
sudo -u ${user} systemctl --user start podman-${container_name}.service
log "Rootless Podman service created for user: $user"
}
create_podman_pod_service() {
local pod_name="$1"
log "Creating systemd service for Podman pod: $pod_name"
cat > /etc/systemd/system/podman-pod-${pod_name}.service << EOF
[Unit]
Description=Podman Pod - ${pod_name}
After=network.target
Requires=network.target
Wants=podman.socket
[Service]
Type=forking
TimeoutStartSec=300
PIDFile=/run/podman-pod-${pod_name}.pid
ExecStart=/usr/bin/podman pod create --name ${pod_name} --infra
ExecStartPost=/usr/bin/podman pod start ${pod_name}
ExecStop=/usr/bin/podman pod stop ${pod_name} -t 10
ExecStopPost=/usr/bin/podman pod rm ${pod_name} -f
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
EOF
# Create containers within the pod
cat > /etc/systemd/system/podman-${pod_name}-web.service << EOF
[Unit]
Description=Web Container in Pod ${pod_name}
After=podman-pod-${pod_name}.service
Requires=podman-pod-${pod_name}.service
BindsTo=podman-pod-${pod_name}.service
[Service]
Type=simple
TimeoutStartSec=300
ExecStart=/usr/bin/podman run \
--pod ${pod_name} \
--name ${pod_name}-web \
--replace \
nginx:alpine
ExecStop=/usr/bin/podman stop --ignore ${pod_name}-web -t 10
ExecStopPost=/usr/bin/podman rm --ignore -f ${pod_name}-web
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
EOF
log "Podman pod service created: $pod_name"
}
setup_podman_systemd_integration() {
log "Setting up Podman systemd integration..."
# Install Podman
if ! command -v podman &> /dev/null; then
apt-get update
apt-get install -y podman
fi
# Enable Podman socket
systemctl enable --now podman.socket
# Create example services
create_podman_systemd_service "nginx" "nginx:alpine" "80"
create_podman_systemd_service "postgres" "postgres:15" "5432"
create_podman_systemd_service "redis" "redis:7" "6379"
# Create pod
create_podman_pod_service "app-pod"
# Enable and start services
systemctl daemon-reload
systemctl enable podman-nginx.service
systemctl enable podman-postgres.service
systemctl enable podman-redis.service
systemctl enable podman-pod-app-pod.service
systemctl enable podman-app-pod-web.service
systemctl start podman-nginx.service
systemctl start podman-postgres.service
systemctl start podman-redis.service
systemctl start podman-pod-app-pod.service
systemctl start podman-app-pod-web.service
log "Podman systemd integration completed"
}
main() {
log "Setting up Podman with systemd integration..."
setup_podman_systemd_integration
log "Podman systemd setup completed!"
log ""
log "Services created:"
log " • podman-nginx.service - Nginx web server"
log " • podman-postgres.service - PostgreSQL database"
log " • podman-redis.service - Redis cache"
log " • podman-pod-app-pod.service - Application pod"
log " • podman-app-pod-web.service - Web container in pod"
log ""
log "Management commands:"
log " systemctl status podman-nginx"
log " journalctl -u podman-nginx -f"
log " podman ps"
log " podman pod ps"
}
main "$@"
5. Kubernetes with Systemd Containers
Systemd-based Kubernetes Pods
6. Production Systemd Container Patterns
1. Minimal systemd: Install only required systemd components
2. Mask unnecessary services: Reduce attack surface
3. Use tmpfs: Mount /tmp, /run as tmpfs for performance
4. Journal configuration: Configure journal for containers
5. Security hardening: Drop capabilities, use namespaces
6. Resource limits: Set cgroup limits for resources
7. Health checks: Implement systemd-based health checks
8. Log aggregation: Forward logs to central system
9. Update strategy: Plan for systemd updates in containers
10. Monitoring: Monitor systemd service states and journals
Production Systemd Container Configuration
#!/bin/bash
# production-systemd-container.sh - Enterprise-grade systemd container configuration
set -euo pipefail
# Configuration
CONTAINER_NAME="enterprise-systemd"
IMAGE_BASE="ubuntu:22.04"
SYSTEMD_VERSION="252"
JOURNAL_MAX_SIZE="100M"
MEMORY_LIMIT="1G"
CPU_LIMIT="1.0"
log() {
echo "[+] $1"
}
create_enterprise_dockerfile() {
log "Creating enterprise Dockerfile..."
cat > Dockerfile.enterprise << 'EOF'
# Enterprise systemd container
FROM ubuntu:22.04 AS builder
# Install build dependencies
RUN apt-get update && apt-get install -y \
build-essential \
git \
meson \
ninja-build \
pkg-config \
&& rm -rf /var/lib/apt/lists/*
# Build custom systemd (optional)
# RUN git clone https://github.com/systemd/systemd.git \
# && cd systemd \
# && meson build \
# && ninja -C build \
# && ninja -C build install
FROM ubuntu:22.04
# Metadata
LABEL org.label-schema.name="Enterprise Systemd Container" \
org.label-schema.description="Production-ready systemd container" \
org.label-schema.vendor="Enterprise Inc." \
org.label-schema.version="1.0" \
org.label-schema.schema-version="1.0"
# Environment
ENV container=docker \
DEBIAN_FRONTEND=noninteractive \
SYSTEMD_OPTS="--log-target=journal --log-level=info" \
JOURNAL_OPTS="--storage=volatile --max-size=100M"
# Install systemd and enterprise packages
RUN apt-get update && apt-get install -y \
systemd \
systemd-sysv \
systemd-timesyncd \
dbus \
dbus-user-session \
sudo \
curl \
wget \
gnupg \
ca-certificates \
apt-transport-https \
software-properties-common \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Create enterprise directory structure
RUN mkdir -p \
/etc/enterprise \
/var/log/enterprise \
/var/lib/enterprise \
/run/enterprise \
&& chmod 755 /etc/enterprise /var/log/enterprise /var/lib/enterprise /run/enterprise
# Systemd configuration
RUN mkdir -p /etc/systemd/{system,system.conf.d,journald.conf.d,logind.conf.d}
# Journal configuration
RUN cat > /etc/systemd/journald.conf.d/enterprise.conf << 'JOURNAL'
[Journal]
Storage=volatile
Compress=yes
Seal=yes
SplitMode=uid
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
SystemMaxUse=100M
RuntimeMaxUse=100M
MaxRetentionSec=1month
MaxFileSec=1day
ForwardToConsole=no
ForwardToSyslog=no
ForwardToWall=no
TTYPath=/dev/console
MaxLevelStore=debug
MaxLevelSyslog=debug
MaxLevelKMsg=notice
MaxLevelConsole=info
MaxLevelWall=emerg
JOURNAL
# Systemd security hardening
RUN cat > /etc/systemd/system.conf.d/security.conf << 'SECURITY'
[Manager]
DefaultTimeoutStartSec=90s
DefaultTimeoutStopSec=90s
DefaultRestartSec=100ms
DefaultStartLimitIntervalSec=10s
DefaultStartLimitBurst=5
RuntimeWatchdogSec=10min
RebootWatchdogSec=10min
KExecWatchdogSec=10min
WatchdogDevice=/dev/watchdog
CapabilityBoundingSet=CAP_AUDIT_WRITE CAP_KILL CAP_NET_BIND_SERVICE CAP_SYS_TIME CAP_SYSLOG
NoNewPrivileges=yes
LockPersonality=yes
PrivateDevices=yes
PrivateMounts=yes
PrivateTmp=yes
PrivateUsers=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=no
ProtectSystem=strict
ReadWritePaths=/var/log/journal /var/log/enterprise /var/lib/enterprise
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
UMask=0027
SECURITY
# Create enterprise services
RUN cat > /etc/systemd/system/enterprise-health.service << 'SERVICE'
[Unit]
Description=Enterprise Health Check Service
After=network.target
Wants=network.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/health-check.sh
ExecReload=/usr/local/bin/health-check.sh
TimeoutStartSec=30
TimeoutStopSec=30
[Install]
WantedBy=multi-user.target
SERVICE
RUN cat > /etc/systemd/system/enterprise-health.timer << 'TIMER'
[Unit]
Description=Enterprise Health Check Timer
[Timer]
OnBootSec=1min
OnUnitActiveSec=5min
AccuracySec=1s
RandomizedDelaySec=30s
[Install]
WantedBy=timers.target
TIMER
# Create health check script
RUN cat > /usr/local/bin/health-check.sh << 'HEALTH'
#!/bin/bash
set -euo pipefail
# Health check script for enterprise container
STATUS_FILE="/run/enterprise/health.status"
TIMESTAMP=$(date +%s)
# Check systemd status
if systemctl is-system-running --quiet; then
echo "OK: System is running" > "$STATUS_FILE"
echo "$TIMESTAMP:healthy" >> "/var/log/enterprise/health.log"
exit 0
else
echo "ERROR: System not running properly" > "$STATUS_FILE"
echo "$TIMESTAMP:unhealthy" >> "/var/log/enterprise/health.log"
exit 1
fi
HEALTH
RUN chmod +x /usr/local/bin/health-check.sh
# Create startup script
RUN cat > /usr/local/bin/enterprise-start.sh << 'STARTUP'
#!/bin/bash
set -euo pipefail
# Initialize container
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a /var/log/enterprise/startup.log
}
log "Starting enterprise systemd container..."
# Initialize machine ID
if [ ! -f /etc/machine-id ] || [ ! -s /etc/machine-id ]; then
log "Initializing machine ID..."
systemd-machine-id-setup
fi
# Start dbus
if [ ! -d /run/dbus ]; then
mkdir -p /run/dbus
fi
log "Starting D-Bus..."
dbus-daemon --system --fork
# Set environment for systemd
export SYSTEMD_PAGER=cat
export SYSTEMD_COLORS=never
# Start systemd
log "Starting systemd..."
exec /usr/lib/systemd/systemd $SYSTEMD_OPTS
STARTUP
RUN chmod +x /usr/local/bin/enterprise-start.sh
# Enable services
RUN systemctl enable enterprise-health.service \
&& systemctl enable enterprise-health.timer
# Cleanup
RUN systemd-tmpfiles --clean \
&& systemd-tmpfiles --remove
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
CMD /usr/local/bin/health-check.sh
# Ports
EXPOSE 80 443 8080
# Entrypoint
ENTRYPOINT ["/usr/local/bin/enterprise-start.sh"]
CMD []
EOF
log "Enterprise Dockerfile created"
}
create_kubernetes_manifests() {
log "Creating Kubernetes manifests for enterprise systemd..."
cat > kubernetes/deployment.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: enterprise-systemd
namespace: enterprise
labels:
app: enterprise-systemd
component: systemd-container
spec:
replicas: 3
revisionHistoryLimit: 3
selector:
matchLabels:
app: enterprise-systemd
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
labels:
app: enterprise-systemd
component: systemd-container
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9100"
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: enterprise-systemd
image: enterprise-systemd:latest
imagePullPolicy: IfNotPresent
command: ["/usr/local/bin/enterprise-start.sh"]
securityContext:
privileged: false
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
add:
- SYS_ADMIN
- SYS_RESOURCE
readOnlyRootFilesystem: true
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
- name: metrics
containerPort: 9100
protocol: TCP
env:
- name: SYSTEMD_OPTS
value: "--log-target=journal --log-level=info"
- name: CONTAINER_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
resources:
requests:
memory: "512Mi"
cpu: "500m"
limits:
memory: "1Gi"
cpu: "1"
volumeMounts:
- name: cgroup
mountPath: /sys/fs/cgroup
readOnly: true
- name: journal
mountPath: /var/log/journal
- name: enterprise-data
mountPath: /var/lib/enterprise
- name: tmp
mountPath: /tmp
- name: run
mountPath: /run
- name: run-lock
mountPath: /run/lock
livenessProbe:
exec:
command:
- /usr/local/bin/health-check.sh
initialDelaySeconds: 60
periodSeconds: 30
timeoutSeconds: 10
failureThreshold: 3
readinessProbe:
exec:
command:
- systemctl
- is-system-running
- --quiet
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
startupProbe:
exec:
command:
- systemctl
- is-system-running
- --quiet
initialDelaySeconds: 10
periodSeconds: 5
failureThreshold: 30
lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- |
echo "Container started at $(date)" >> /var/log/enterprise/lifecycle.log
preStop:
exec:
command:
- /bin/sh
- -c
- |
echo "Container stopping at $(date)" >> /var/log/enterprise/lifecycle.log
systemctl stop --all
volumes:
- name: cgroup
hostPath:
path: /sys/fs/cgroup
type: Directory
- name: journal
emptyDir:
sizeLimit: 100Mi
- name: enterprise-data
emptyDir:
sizeLimit: 1Gi
- name: tmp
emptyDir:
sizeLimit: 100Mi
- name: run
emptyDir:
sizeLimit: 50Mi
- name: run-lock
emptyDir:
sizeLimit: 10Mi
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- enterprise-systemd
topologyKey: kubernetes.io/hostname
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/arch
operator: In
values:
- amd64
- arm64
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
---
apiVersion: v1
kind: Service
metadata:
name: enterprise-systemd
namespace: enterprise
labels:
app: enterprise-systemd
spec:
type: ClusterIP
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
- name: https
port: 443
targetPort: 443
protocol: TCP
selector:
app: enterprise-systemd
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: enterprise-systemd
namespace: enterprise
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: enterprise-systemd
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
EOF
log "Kubernetes manifests created"
}
create_monitoring_config() {
log "Creating monitoring configuration..."
cat > monitoring/prometheus.yml << 'EOF'
scrape_configs:
- job_name: 'enterprise-systemd'
scrape_interval: 30s
scrape_timeout: 10s
metrics_path: /metrics
static_configs:
- targets: ['enterprise-systemd.enterprise.svc.cluster.local:9100']
relabel_configs:
- source_labels: [__address__]
target_label: instance
- source_labels: [__meta_kubernetes_pod_name]
target_label: pod
EOF
cat > monitoring/grafana-dashboard.json << 'EOF'
{
"dashboard": {
"title": "Enterprise Systemd Dashboard",
"panels": [
{
"title": "Systemd Services Status",
"type": "stat",
"targets": [
{
"expr": "systemd_units_active{name=~\"enterprise-.*\"}",
"legendFormat": "{{name}}"
}
]
}
]
}
}
EOF
log "Monitoring configuration created"
}
main() {
log "Setting up enterprise systemd container solution..."
# Create directory structure
mkdir -p kubernetes monitoring
# Create configurations
create_enterprise_dockerfile
create_kubernetes_manifests
create_monitoring_config
log "Enterprise systemd container setup completed!"
log ""
log "Next steps:"
log " 1. Build the container: docker build -f Dockerfile.enterprise -t enterprise-systemd ."
log " 2. Push to registry: docker push registry/enterprise-systemd:latest"
log " 3. Deploy to Kubernetes: kubectl apply -f kubernetes/"
log " 4. Set up monitoring: kubectl apply -f monitoring/"
log " 5. Configure CI/CD pipeline"
log ""
log "Features included:"
log " • Production-ready systemd configuration"
log " • Security hardening"
log " • Health checks and monitoring"
log " • Kubernetes manifests"
log " • Resource management"
log " • Logging and journal configuration"
}
main "$@"
7. Troubleshooting Systemd in Containers
Common Issues & Solutions
| Issue | Symptoms | Root Cause | Solution |
|---|---|---|---|
| Systemd fails to start | Container exits immediately | Missing cgroups mount, capabilities | Mount /sys/fs/cgroup, add SYS_ADMIN capability |
| Journal not working | No logs, journalctl fails | Missing /run/log/journal | Create directory, configure journal storage |
| DBus errors | Service startup failures | DBus daemon not running | Start dbus-daemon before systemd |
| Socket activation fails | Services not starting on connection | Missing socket units or permissions | Check socket unit files and permissions |
| Timer services not running | Scheduled jobs not executing | Systemd timers not enabled | Enable and start timer units |
| Resource limits ignored | Container exceeds limits | cgroups v2 not properly mounted | Mount cgroups with proper permissions |
| Security policy violations | SELinux/AppArmor denials | Container not configured for security modules | Adjust security policies or run unconfined |
| Network services fail | Can't bind to ports | NET_BIND_SERVICE capability missing | Add capability or run as root |
Systemd Container Debugging Toolkit
Master Systemd in Containers
Running systemd inside containers provides a powerful init system for managing services, logging, and resource control within containerized environments. By understanding systemd-nspawn, Docker/Podman integration, and production configurations, you can build robust container platforms.
Key Takeaways: Systemd provides full init system capabilities in containers. Use systemd-nspawn for lightweight systemd-native containers. Integrate systemd with Docker using proper volume mounts and capabilities. Leverage Podman's native systemd integration. Implement production patterns with security hardening and monitoring. Use systemd's journal for centralized logging. Implement health checks and service management.
Next Steps: Experiment with systemd-nspawn containers. Build Docker images with systemd as PID 1. Implement Podman systemd services. Deploy systemd-based containers to Kubernetes. Set up monitoring and logging for systemd containers. Implement security hardening practices. Explore advanced systemd features like socket activation and timers in containers.