Service Lifecycle Management: Complete Guide to Managing systemd Services

Master the complete lifecycle of systemd services. This comprehensive guide covers everything from basic service operations to advanced management techniques, troubleshooting, and automation.

Understanding Service States

1. Basic Service Operations

2. Essential Commands Reference

3. Service State Management

Detailed Status Example

# systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2025-12-07 10:30:45 UTC; 2h ago
       Docs: man:nginx(8)
    Process: 1234 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 1235 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 1236 (nginx)
      Tasks: 3 (limit: 1137)
     Memory: 10.2M
        CPU: 1.234s
     CGroup: /system.slice/nginx.service
             ├─1236 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             ├─1237 nginx: worker process
             └─1238 nginx: worker process

Dec 07 10:30:45 server systemd[1]: Starting A high performance web server and a reverse proxy server...
Dec 07 10:30:45 server nginx[1235]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Dec 07 10:30:45 server nginx[1235]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Dec 07 10:30:45 server systemd[1]: Started A high performance web server and a reverse proxy server.

4. Service Dependency Management

Understanding Dependencies

# View service dependencies
systemctl list-dependencies nginx.service          # Show all dependencies
systemctl list-dependencies nginx.service --reverse # Show reverse dependencies
systemctl list-dependencies nginx.service --before # Show what runs before
systemctl list-dependencies nginx.service --after  # Show what runs after

# Dependency types in unit files
[Unit]
Requires=postgresql.service      # Strong dependency - fails if dependency fails
Wants=redis.service             # Weak dependency - continues even if dependency fails
After=network.target            # Ordering - start after
Before=multi-user.target        # Ordering - start before
Conflicts=old-service.service   # Cannot run concurrently
PartOf=web-stack.target         # Part of a target unit
BindsTo=storage.service         # Stronger than Requires + restarts together

Dependency Management Commands

5. Advanced Service Operations

Service Reload Strategies

systemctl show nginx.service -p ExecReload
# ExecReload=/bin/kill -HUP $MAINPID

nginx -t  # For nginx
apachectl configtest  # For Apache
systemctl reload --dry-run nginx.service

systemctl reload nginx.service      # Send SIGHUP
systemctl reload-or-restart nginx.service  # Reload or restart
systemctl try-reload-or-restart nginx.service  # Try reload first

Service Isolation and Sandboxing

# Run service in temporary scope
systemd-run --unit=temporary-service --service-type=simple /path/to/command
systemd-run --user --unit=user-service --service-type=simple /path/to/command

# Run service with specific properties
systemd-run --unit=test-service \
  --property=User=testuser \
  --property=WorkingDirectory=/tmp \
  --property=Environment="PATH=/usr/bin" \
  /path/to/command

# Create transient service
systemd-run --unit=transient --property=RuntimeMaxSec=300 /path/to/script

# Control service execution
systemctl kill nginx.service --signal=SIGTERM      # Send specific signal
systemctl kill nginx.service --kill-who=main --signal=SIGKILL
systemctl clean nginx.service                      # Clean runtime directories

6. Troubleshooting Common Issues

Troubleshooting Workflow

systemctl status service.service -l --no-pager

journalctl -u service.service --since "1 hour ago" -f
journalctl -u service.service -p err..alert  # Only errors
journalctl -u service.service --boot         # Current boot only

sudo -u serviceuser /path/to/command  # Run as service user
strace -f /path/to/command           # Trace system calls
ltrace /path/to/command              # Trace library calls

systemctl show service.service -p Limit*
ulimit -a                            # Current shell limits
df -h /var/log                      # Disk space
free -h                             # Memory

systemd-analyze verify service.service
test -f /path/to/config && echo "Config exists"
cat /path/to/config | head -20      # Check config syntax

7. Automation & Scripting

Bash Script Examples

#!/bin/bash
# service-manager.sh - Manage services with error handling

SERVICE_NAME="$1"
ACTION="$2"

# Function to check if service exists
service_exists() {
    systemctl list-unit-files | grep -q "^$SERVICE_NAME.service"
    return $?
}

# Function to check service status
service_status() {
    if service_exists; then
        systemctl is-active "$SERVICE_NAME.service"
        return $?
    else
        echo "Service $SERVICE_NAME does not exist"
        return 3
    fi
}

# Function to start service
start_service() {
    echo "Starting $SERVICE_NAME..."
    systemctl start "$SERVICE_NAME.service"
    local ret=$?
    
    if [ $ret -eq 0 ]; then
        echo "$SERVICE_NAME started successfully"
        return 0
    else
        echo "Failed to start $SERVICE_NAME (exit code: $ret)"
        journalctl -u "$SERVICE_NAME.service" --since "1 minute ago" | tail -20
        return $ret
    fi
}

# Function to stop service
stop_service() {
    echo "Stopping $SERVICE_NAME..."
    systemctl stop "$SERVICE_NAME.service"
    sleep 2  # Give time for graceful shutdown
    
    if systemctl is-active "$SERVICE_NAME.service" &>/dev/null; then
        echo "Service still running, forcing stop..."
        systemctl kill "$SERVICE_NAME.service" --signal=SIGKILL
    fi
    
    systemctl is-active "$SERVICE_NAME.service" &>/dev/null
    if [ $? -ne 0 ]; then
        echo "$SERVICE_NAME stopped successfully"
        return 0
    else
        echo "Failed to stop $SERVICE_NAME"
        return 1
    fi
}

# Main execution
case "$ACTION" in
    start)
        start_service
        ;;
    stop)
        stop_service
        ;;
    restart)
        stop_service
        sleep 1
        start_service
        ;;
    status)
        service_status
        ;;
    *)
        echo "Usage: $0  {start|stop|restart|status}"
        exit 1
        ;;
esac

Ansible Playbook Example

# service-management.yml
- name: Manage systemd services
  hosts: all
  become: yes
  tasks:
    - name: Ensure nginx is installed
      package:
        name: nginx
        state: present
    
    - name: Enable and start nginx service
      systemd:
        name: nginx
        state: started
        enabled: yes
        daemon_reload: yes
    
    - name: Configure nginx service limits
      copy:
        dest: /etc/systemd/system/nginx.service.d/limits.conf
        content: |
          [Service]
          LimitNOFILE=65536
          LimitNPROC=4096
        owner: root
        group: root
        mode: '0644'
      notify: reload systemd
    
    - name: Monitor service health
      command: systemctl is-active nginx
      register: service_status
      changed_when: false
      check_mode: no
    
    - name: Alert if service is down
      debug:
        msg: "WARNING: nginx service is not running!"
      when: service_status.rc != 0
  
  handlers:
    - name: reload systemd
      systemd:
        daemon_reload: yes

8. Best Practices & Pro Tips

Pro Tips for Power Users