Managing Services with systemd: Complete Guide

Master the modern Linux service management with systemd. This comprehensive guide covers essential systemctl commands, service control, monitoring, troubleshooting, and creating custom systemd units for efficient daemon management.

Systemd Architecture Overview Kernel systemd Services systemctl journalctl logind networkd timedatectl Kernel Interface Core systemd Managed Services Components
Systemd Architecture: Core components and their relationships

What is systemd?

systemd is a system and service manager for Linux operating systems. It replaces the traditional SysV init system and provides:

  • Parallel startup of system services
  • On-demand activation of daemons
  • Dependency-based service control
  • Snapshot and restore capabilities
  • Integrated logging with journald
  • Socket activation for better resource management

1. Essential systemctl Commands

Start Services
systemctl start service-name
Start a service immediately. The service must be enabled or manually started.
Stop Services
systemctl stop service-name
Stop a running service. Use with caution for critical services.
Restart Services
systemctl restart service-name
Stop and then start a service. Useful for applying configuration changes.
Check Status
systemctl status service-name
View detailed service status including recent logs and process information.
Enable Services
systemctl enable service-name
Configure service to start automatically at boot. Creates symlinks in systemd directories.
Disable Services
systemctl disable service-name
Prevent service from starting automatically at boot. Does not stop currently running service.
View Logs
journalctl -u service-name
View logs for a specific service. Add -f to follow logs in real-time.
List Services
systemctl list-units --type=service
List all services with their current status. Add --all to show inactive services too.
$ systemctl status nginx โ— nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2025-12-01 10:30:45 UTC; 6 days ago Docs: man:nginx(8) Main PID: 1234 (nginx) Tasks: 2 (limit: 4915) Memory: 10.2M CGroup: /system.slice/nginx.service โ”œโ”€1234 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; โ””โ”€1235 nginx: worker process Dec 01 10:30:45 server systemd[1]: Starting A high performance web server... Dec 01 10:30:45 server systemd[1]: Started A high performance web server...
Example of systemctl status command output showing service information

Service Status Commands in Detail

# Basic service control
systemctl start nginx
systemctl stop apache2
systemctl restart mysql
systemctl reload ssh # Reload without restart
# Enable/disable at boot
systemctl enable docker
systemctl disable postfix
systemctl is-enabled sshd # Check if enabled
systemctl mask service # Prevent all starts
systemctl unmask service # Remove mask
# Status information
systemctl status nginx -l # Full logs
systemctl is-active nginx # Active/inactive
systemctl is-failed nginx # Check if failed
systemctl show nginx # Show all properties
# List and filter services
systemctl list-units --type=service --state=running
systemctl list-unit-files --type=service
systemctl list-dependencies nginx.service
systemctl list-timers --all

2. Service Management Scenarios

Scenario Commands Explanation Start service now and at boot systemctl enable --now nginx Enable for boot and start immediately Check why service failed systemctl status nginx
journalctl -u nginx -xe
View status and detailed logs with explanations Reload config without downtime systemctl reload nginx Reload configuration files without restarting service Prevent service from starting systemctl mask nginx Create symlink to /dev/null to prevent all starts Reset failed service systemctl reset-failed nginx Clear failed state after fixing issues Edit service configuration systemctl edit nginx Create override configuration in /etc

Journalctl: Mastering System Logs

# View service-specific logs
journalctl -u nginx.service # Nginx logs
journalctl -u nginx --since "1 hour ago"
journalctl -u nginx -f # Follow logs
journalctl -u nginx -n 50 # Last 50 lines
# Filter by priority
journalctl -p err # Error messages only
journalctl -p warning # Warning messages
journalctl -p info # Info messages
# Time-based filtering
journalctl --since "2025-12-01" --until "2025-12-07"
journalctl --since yesterday
journalctl --since "09:00" --until "17:00"
# Boot-specific logs
journalctl -b # Current boot
journalctl -b -1 # Previous boot
journalctl --list-boots # List all boots
# Advanced filtering
journalctl _PID=1234 # By process ID
journalctl _UID=1000 # By user ID
journalctl _COMM=sshd # By command name
journalctl --grep="error\|fail" # Search patterns
# Export and maintenance
journalctl --output=json # JSON output
journalctl --disk-usage # Check log size
journalctl --vacuum-size=500M # Reduce to 500MB
journalctl --vacuum-time=2weeks # Keep 2 weeks

3. Creating Custom systemd Services

Basic Service Unit File

# /etc/systemd/system/myapp.service
[Unit]
Description=My Custom Application
After=network.target
Wants=network.target

[Service]
Type=simple
User=myappuser
Group=myappgroup
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/python3 /opt/myapp/app.py
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal

# Security hardening (optional)
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/var/lib/myapp

[Install]
WantedBy=multi-user.target

Service Types Explained

Type Description Use Case simple Default. systemd considers service started immediately. Most applications, scripts forking Service forks a child process and exits parent. Traditional daemons like nginx, apache oneshot Service runs once and exits. Scripts, maintenance tasks dbus Service acquires a D-Bus name before starting. D-Bus services notify Service sends notification when ready. Services that need to signal readiness idle Service starts after all jobs are dispatched. Low priority services

Installation and Management Script

#!/bin/bash
# install-service.sh
# Creates and manages a custom systemd service

SERVICE_NAME="myapp"
SERVICE_FILE="/etc/systemd/system/${SERVICE_NAME}.service"
APP_PATH="/opt/${SERVICE_NAME}"
APP_USER="${SERVICE_NAME}"

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

echo -e "${GREEN}=== Custom Service Manager ===${NC}"

case "$1" in
    install)
        echo -e "${YELLOW}Installing ${SERVICE_NAME} service...${NC}"
        
        # Check if running as root
        if [ "$EUID" -ne 0 ]; then 
            echo -e "${RED}Please run as root${NC}"
            exit 1
        fi
        
        # Create service file
        cat > "$SERVICE_FILE" << EOF
[Unit]
Description=My Custom Application
After=network.target

[Service]
Type=simple
User=$APP_USER
Group=$APP_USER
WorkingDirectory=$APP_PATH
ExecStart=/usr/bin/python3 $APP_PATH/app.py
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
EOF
        
        echo -e "${GREEN}Service file created: $SERVICE_FILE${NC}"
        
        # Create application user if doesn't exist
        if ! id "$APP_USER" &>/dev/null; then
            useradd -r -s /bin/false "$APP_USER"
            echo -e "${GREEN}Created user: $APP_USER${NC}"
        fi
        
        # Reload systemd
        systemctl daemon-reload
        echo -e "${GREEN}Systemd daemon reloaded${NC}"
        
        # Enable and start service
        systemctl enable "$SERVICE_NAME"
        systemctl start "$SERVICE_NAME"
        
        echo -e "${GREEN}Service installed and started${NC}"
        systemctl status "$SERVICE_NAME"
        ;;
    
    uninstall)
        echo -e "${YELLOW}Uninstalling ${SERVICE_NAME} service...${NC}"
        
        systemctl stop "$SERVICE_NAME"
        systemctl disable "$SERVICE_NAME"
        rm -f "$SERVICE_FILE"
        systemctl daemon-reload
        echo -e "${GREEN}Service uninstalled${NC}"
        ;;
    
    status)
        systemctl status "$SERVICE_NAME"
        ;;
    
    logs)
        journalctl -u "$SERVICE_NAME" -f
        ;;
    
    restart)
        systemctl restart "$SERVICE_NAME"
        echo -e "${GREEN}Service restarted${NC}"
        ;;
    
    *)
        echo "Usage: $0 {install|uninstall|status|logs|restart}"
        echo "  install   - Install and start the service"
        echo "  uninstall - Stop and remove the service"
        echo "  status    - Check service status"
        echo "  logs      - Follow service logs"
        echo "  restart   - Restart the service"
        exit 1
        ;;
esac

4. Advanced systemd Features

Timers: Systemd's Cron Replacement

# /etc/systemd/system/backup.timer
[Unit]
Description=Daily backup timer

[Timer]
OnCalendar=daily
Persistent=true
RandomizedDelaySec=1h

[Install]
WantedBy=timers.target

# /etc/systemd/system/backup.service
[Unit]
Description=Database backup service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh
User=backup

Socket Activation

# /etc/systemd/system/myservice.socket
[Unit]
Description=My Service Socket

[Socket]
ListenStream=8080
Accept=true

[Install]
WantedBy=sockets.target

# /etc/systemd/system/myservice@.service
[Unit]
Description=My Service Instance

[Service]
ExecStart=/usr/bin/myservice
StandardInput=socket

Troubleshooting Service Issues

# Diagnostic commands for service issues
systemctl status service -l --no-pager # Full output
systemctl cat service # View unit file
systemctl show service # All properties
systemd-analyze verify service.service # Validate syntax
# Check dependencies
systemctl list-dependencies service
systemctl list-dependencies service --reverse
# Test service manually
systemd-run --unit=test-service --service-type=simple /path/to/command
systemd-analyze plot > boot.svg # Boot timeline
# Reset failed units
systemctl reset-failed # All failed
systemctl reset-failed service # Specific
Common Service Issues and Solutions:
1. Service fails to start: Check journalctl -xe and verify paths/permissions
2. Service starts but exits immediately: Check service Type and ExecStart command
3. Can't enable service: Check if service file is in correct location (/etc/systemd/system/)
4. Changes not taking effect: Run systemctl daemon-reload after editing service files
5. Permission denied: Check User/Group settings and file permissions
6. Dependency issues: Verify After= and Requires= directives
7. Resource limits: Check if service hits memory or CPU limits
8. Socket activation not working: Verify Accept=true and correct socket unit

5. systemd Cheat Sheet

Quick Start
systemctl enable --now service
Enable at boot and start immediately
Debug Service
systemd-analyze verify service
Validate service file syntax
Boot Analysis
systemd-analyze blame
Find services slowing boot
Override Config
systemctl edit service
Create configuration override
Clean Failed
systemctl reset-failed
Clear failed service state
Reload Config
systemctl daemon-reload
Reload after editing service files
List Timers
systemctl list-timers
View all systemd timers
Run Test
systemd-run command
Run command as transient service

Best Practices for systemd Services

  • Always use systemctl daemon-reload after editing service files
  • Set appropriate User and Group for security
  • Use Restart=on-failure for resilient services
  • Specify WorkingDirectory for consistent file paths
  • Use journalctl -u service -f to monitor logs in real-time
  • Test with systemd-analyze verify before deploying
  • Use systemctl edit instead of modifying original files
  • Set resource limits with MemoryLimit and CPUShares
  • Use Type=notify for services that need readiness notification
  • Regularly check systemctl list-units --failed

Getting Started with systemd

Follow this learning path to master systemd:

  1. Learn basic commands: Start with start, stop, restart, status
  2. Master journalctl: Learn to filter and search logs effectively
  3. Understand unit files: Learn the structure of .service files
  4. Create custom services: Practice by creating simple service units
  5. Explore timers: Replace cron jobs with systemd timers
  6. Learn dependencies: Master After=, Requires=, Wants=
  7. Practice troubleshooting: Use the diagnostic commands regularly
  8. Study advanced features: Learn socket activation, resource control
  9. Create templates: Use service templates for multiple instances
  10. Stay updated: Follow systemd releases and new features

Master Modern Linux Service Management

systemd has revolutionized Linux service management with its powerful features and unified approach. By mastering systemctl commands and understanding service unit files, you gain complete control over your system's services.

Remember: systemd is not just an init system but a complete suite of system management tools. Take advantage of its logging, dependency management, and resource control features to build robust and maintainable systems.

Next Steps: Practice creating custom services for your applications. Convert existing init scripts to systemd units. Experiment with timers and socket activation. The more you work with systemd, the more you'll appreciate its power and flexibility.