Go beyond traditional Linux permissions with mandatory access control (MAC). Learn how SELinux and AppArmor add an extra security layer to protect your systems from compromised applications and zero-day attacks.
What is Mandatory Access Control (MAC)?
Traditional Linux uses DAC (Discretionary Access Control) - users decide who can access their files. MAC adds an extra layer where the system decides what processes can do, regardless of user permissions.
DAC (Traditional)
User controls permissions
Files: 644, 755, etc.
MAC (Extra Layer)
System controls access
Policies define allowed actions
Result
Even if app is hacked,
damage is limited
SELinux vs AppArmor: Which One?
Origin: NSA (National Security Agency)
Used By: RHEL, CentOS, Fedora, Android
Approach: Fine-grained, label-based
Complexity: High (steep learning curve)
Origin: Immunix (now Canonical)
Used By: Ubuntu, Debian, SUSE, openSUSE
Approach: Path-based, simpler
Complexity: Medium (easier to learn)
Comparison Table for DevOps
| Feature | SELinux | AppArmor | DevOps Recommendation |
|---|---|---|---|
| Learning Curve | Steep | Gentle | Start with AppArmor if new to MAC |
| Configuration | Labels & Contexts | Path-based Rules | AppArmor easier for custom apps |
| Default Policies | Comprehensive | Good for common apps | SELinux better out-of-box |
| Debugging | Complex (audit2allow) | Simpler logs | AppArmor easier to troubleshoot |
| Performance | Slower (more checks) | Faster (path-based) | AppArmor for high-performance needs |
| Community | Enterprise-focused | Community-focused | Choose based on your distro |
Part 1: SELinux Basics for DevOps
SELinux Modes
# Check SELinux status
sudo sestatus
# Output example:
# SELinux status: enabled
# SELinuxfs mount: /sys/fs/selinux
# SELinux root directory: /etc/selinux
# Loaded policy name: targeted
# Current mode: enforcing
# Mode from config file: enforcing
# Policy MLS status: enabled
# Policy deny_unknown status: allowed
# Memory protection checking: actual (secure)
# Max kernel policy version: 33
# Check current mode
getenforce
# Returns: Enforcing, Permissive, or Disabled
# Temporary change mode (reboot resets)
sudo setenforce 0 # Permissive mode (logs but doesn't block)
sudo setenforce 1 # Enforcing mode (blocks violations)
# Permanent change (edit config file)
sudo nano /etc/selinux/config
# Change: SELINUX=enforcing
# Options: enforcing, permissive, disabled
# Check SELinux booleans (on/off switches)
getsebool -a
sudo semanage boolean -l
# Toggle a boolean
sudo setsebool -P httpd_can_network_connect on
sudo setsebool -P httpd_can_sendmail off
Understanding SELinux Contexts
# View file contexts
ls -Z /var/www/html
# Output: -rw-r--r--. root root unconfined_u:object_r:httpd_sys_content_t:s0 index.html
# View process contexts
ps -Z
ps auxZ | grep nginx
# Breakdown of context:
# unconfined_u:object_r:httpd_sys_content_t:s0
# ├── unconfined_u User (identity)
# ├── object_r Role
# ├── httpd_sys_content_t Type (most important)
# └── s0 Level (MLS/MCS)
# Change file context
sudo chcon -t httpd_sys_content_t /var/www/html/index.html
sudo chcon -R -t httpd_sys_content_t /var/www/html/
# Restore default context
sudo restorecon -v /var/www/html/index.html
sudo restorecon -Rv /var/www/html/
Common SELinux Contexts:
httpd_sys_content_t - Web content readable by Apache
httpd_sys_rw_content_t - Web content writable by Apache
mysql_db_t - MySQL database files
var_log_t - Log files in /var/log
user_home_t - User home directory files
bin_t - System binaries
Troubleshooting SELinux
# 1. Check SELinux denials (most important!)
sudo ausearch -m avc -ts recent
# OR
sudo sealert -a /var/log/audit/audit.log
# 2. Common problem: Web server can't access files
# Error in /var/log/audit/audit.log:
# type=AVC msg=audit(1234567890.123:456): avc: denied { getattr } for pid=1234 comm="httpd"...
# Solution: Allow access
sudo semanage fcontext -a -t httpd_sys_content_t "/var/www/html(/.*)?"
sudo restorecon -Rv /var/www/html
# 3. Generate custom policy module from denial
sudo grep "avc:.*denied" /var/log/audit/audit.log | audit2allow -M mypolicy
sudo semodule -i mypolicy.pp
# 4. Temporarily allow while developing
sudo setenforce 0 # Switch to permissive
# Test your application
sudo setenforce 1 # Switch back to enforcing
# Then fix properly with audit2allow
# 5. Check if issue is SELinux related
# Temporarily disable SELinux and test:
sudo setenforce 0
# If problem disappears → SELinux issue
# If problem persists → Not SELinux issue
sudo setenforce 1 # Re-enable immediately!
Part 2: AppArmor Basics for DevOps
AppArmor Status & Modes
# Check AppArmor status
sudo aa-status
# Output example:
# apparmor module is loaded.
# 43 profiles are loaded.
# 43 profiles are in enforce mode.
# 0 profiles are in complain mode.
# 0 processes have profiles defined.
# 0 processes are in enforce mode.
# 0 processes are in complain mode.
# 0 processes are unconfined but have a profile defined.
# List all profiles
sudo aa-status | grep -A 100 "profiles are loaded"
# Check specific profile
sudo apparmor_status | grep nginx
sudo apparmor_status | grep apache2
# Profile modes:
# enforce - Blocks violations (like SELinux enforcing)
# complain - Logs violations but doesn't block (like SELinux permissive)
# disable - Profile is disabled
Managing AppArmor Profiles
# List profiles in /etc/apparmor.d/
ls /etc/apparmor.d/
# Common profile locations:
# /etc/apparmor.d/usr.bin.nginx # Nginx profile
# /etc/apparmor.d/usr.sbin.mysqld # MySQL profile
# /etc/apparmor.d/usr.lib.snapd.snap-confine # Snap profiles
# Change profile mode
sudo aa-complain /etc/apparmor.d/usr.sbin.mysqld # Switch to complain mode
sudo aa-enforce /etc/apparmor.d/usr.sbin.mysqld # Switch to enforce mode
# Disable a profile
sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld
# Enable a profile
sudo rm /etc/apparmor.d/disable/usr.sbin.mysqld
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.mysqld
# Reload all profiles
sudo systemctl reload apparmor
# OR
sudo /etc/init.d/apparmor reload
Understanding AppArmor Profiles
# View a profile
sudo cat /etc/apparmor.d/usr.sbin.mysqld
# Example profile structure:
#include
profile mysqld /usr/sbin/mysqld {
#include
#include
# Capabilities
capability dac_override,
capability sys_resource,
capability setgid,
capability setuid,
# Network access
network inet tcp,
network inet6 tcp,
# Filesystem access
/etc/mysql/** r,
/var/lib/mysql/** rwk,
/var/log/mysql.log rw,
/var/log/mysql.err rw,
/var/run/mysqld/mysqld.pid w,
/var/run/mysqld/mysqld.sock w,
# Libraries
/usr/lib/mysql/plugin/** mr,
# Deny everything else (implicit)
deny /** w,
}
# Key syntax:
# r - read
# w - write
# k - lock
# l - link
# x - execute
# m - memory map as executable
# mr - memory map as readable
# mrw - memory map as read/write
Troubleshooting AppArmor
# 1. Check AppArmor denials
sudo dmesg | grep apparmor
sudo journalctl | grep apparmor
sudo cat /var/log/kern.log | grep apparmor
# 2. Common problem: Application blocked
# Error in logs: "apparmor=\"DENIED\""
# 3. Use aa-logprof to create/modify profiles
# First, put profile in complain mode
sudo aa-complain /etc/apparmor.d/usr.sbin.mysqld
# Run your application to generate logs
# Then analyze logs and update profile
sudo aa-logprof
# 4. Generate profile from scratch
sudo aa-genprof /usr/sbin/nginx
# This will:
# 1. Create a profile in complain mode
# 2. Ask you to run your application
# 3. Scan logs for denials
# 4. Interactively update profile
# 5. Move profile to enforce mode
# 5. Manual profile debugging
# Check what's happening in real-time
sudo tail -f /var/log/kern.log | grep apparmor
# 6. Temporarily disable for testing
sudo aa-complain /path/to/profile
# OR disable completely (not recommended)
sudo systemctl stop apparmor
sudo systemctl disable apparmor
Practical DevOps Examples
Example 1: Web Server Configuration
SELinux for Apache/Nginx
# Allow web server to connect to network
sudo setsebool -P httpd_can_network_connect on
# Allow web server to send email
sudo setsebool -P httpd_can_sendmail on
# Allow PHP to connect to MySQL
sudo setsebool -P httpd_can_network_connect_db on
# Allow web server to write to upload directory
sudo chcon -R -t httpd_sys_rw_content_t /var/www/html/uploads/
sudo semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/uploads(/.*)?"
# Allow access to specific port (e.g., Node.js on 3000)
sudo semanage port -a -t http_port_t -p tcp 3000
AppArmor for Apache/Nginx
# Edit web server profile
sudo nano /etc/apparmor.d/usr.sbin.apache2
# Add these lines to allow specific directories:
/var/www/html/** r,
/var/www/html/uploads/** rw,
/tmp/** rw,
/run/php/php7.4-fpm.sock rw,
# Reload profile
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.apache2
Example 2: Database Server
SELinux for MySQL/MariaDB
# Allow database to access data directory
sudo chcon -R -t mysqld_db_t /var/lib/mysql/
# Allow connections from network
sudo setsebool -P mysqld_connect_any on
# Allow to write to log directory
sudo chcon -R -t mysqld_log_t /var/log/mysql/
# Fix common permission issues
sudo restorecon -Rv /var/lib/mysql
sudo restorecon -Rv /var/log/mysql
AppArmor for MySQL/MariaDB
# Check existing profile
sudo cat /etc/apparmor.d/usr.sbin.mysqld
# Common additions needed:
/var/lib/mysql/** rwk,
/var/log/mysql/** rw,
/run/mysqld/mysqld.pid w,
/run/mysqld/mysqld.sock w,
/usr/share/mysql/** r,
# If using custom data directory:
/data/mysql/** rwk,
Example 3: Custom Application
# For SELinux:
# 1. Run in permissive mode first
sudo setenforce 0
# 2. Run your application
./myapp
# 3. Check audit logs
sudo ausearch -m avc -ts recent
# 4. Generate policy
sudo grep "avc:.*denied" /var/log/audit/audit.log | audit2allow -M myapp
sudo semodule -i myapp.pp
# 5. Test in enforcing mode
sudo setenforce 1
./myapp
# For AppArmor:
# 1. Generate profile
sudo aa-genprof /path/to/myapp
# 2. Follow interactive prompts
# 3. Profile will be created in /etc/apparmor.d/
# 4. Manually edit if needed
sudo nano /etc/apparmor.d/path.to.myapp
DevOps Automation Scripts
SELinux Automation Script
#!/bin/bash
# selinux-setup.sh - Automated SELinux configuration for DevOps
echo "=== SELinux Setup for DevOps ==="
# 1. Check status
echo "1. Current SELinux status:"
sestatus
# 2. Ensure SELinux is enabled
if ! sestatus | grep -q "enabled"; then
echo "Enabling SELinux..."
sudo sed -i 's/SELINUX=disabled/SELINUX=enforcing/' /etc/selinux/config
echo "SELinux will be enabled on next reboot"
fi
# 3. Set common booleans for web servers
echo "Setting common booleans..."
sudo setsebool -P httpd_can_network_connect on
sudo setsebool -P httpd_can_sendmail on
sudo setsebool -P httpd_can_network_connect_db on
sudo setsebool -P mysqld_connect_any on
# 4. Set common contexts
echo "Setting common contexts..."
sudo chcon -R -t httpd_sys_content_t /var/www/html 2>/dev/null || true
sudo chcon -R -t var_log_t /var/log 2>/dev/null || true
# 5. Install troubleshooting tools
echo "Installing troubleshooting tools..."
sudo yum install -y setroubleshoot setools 2>/dev/null || \
sudo apt install -y setools policycoreutils 2>/dev/null || true
# 6. Create audit log monitoring
echo "Creating audit log monitoring..."
cat > /usr/local/bin/check-selinux-denials << 'EOF'
#!/bin/bash
echo "Recent SELinux denials:"
sudo ausearch -m avc -ts recent --raw | audit2why
EOF
chmod +x /usr/local/bin/check-selinux-denials
echo "=== SELinux setup complete ==="
echo "Use: check-selinux-denials to see recent issues"
AppArmor Automation Script
#!/bin/bash
# apparmor-setup.sh - Automated AppArmor setup
echo "=== AppArmor Setup for DevOps ==="
# 1. Check if AppArmor is installed
if ! command -v aa-status &> /dev/null; then
echo "Installing AppArmor..."
sudo apt update
sudo apt install -y apparmor apparmor-utils apparmor-profiles
fi
# 2. Enable AppArmor
echo "Enabling AppArmor..."
sudo systemctl enable apparmor
sudo systemctl start apparmor
# 3. Ensure kernel module is loaded
sudo modprobe apparmor 2>/dev/null || true
# 4. Set common profiles to enforce mode
echo "Setting common profiles to enforce..."
for profile in usr.sbin.mysqld usr.sbin.apache2 usr.sbin.nginx; do
if [ -f "/etc/apparmor.d/$profile" ]; then
sudo aa-enforce "/etc/apparmor.d/$profile"
fi
done
# 5. Install additional profiles
echo "Installing extra profiles..."
sudo apt install -y apparmor-profiles-extra 2>/dev/null || true
# 6. Create monitoring script
echo "Creating monitoring script..."
cat > /usr/local/bin/check-apparmor-denials << 'EOF'
#!/bin/bash
echo "Recent AppArmor denials:"
sudo dmesg | grep apparmor | tail -20
echo ""
echo "Current status:"
sudo aa-status
EOF
chmod +x /usr/local/bin/check-apparmor-denials
echo "=== AppArmor setup complete ==="
echo "Use: check-apparmor-denials to monitor"
Common Problems & Solutions
chcon -t httpd_sys_rw_content_trw permission to profilerestorecon -Rv /var/lib/mysqlsetenforce 0 (then setenforce 1 after testing)aa-complain profile or stop service• Never disable SELinux/AppArmor permanently in production
• Use permissive/complain mode for troubleshooting only
• Always return to enforcing mode after fixes
• Document all policy changes for audit purposes
• Test changes in staging before production
SELinux:
• Status:
sestatus, getenforce• Mode:
setenforce 0|1• Contexts:
ls -Z, chcon, restorecon• Logs:
ausearch -m avc -ts recent• Booleans:
getsebool -a, setseboolAppArmor:
• Status:
aa-status• Modes:
aa-enforce, aa-complain• Profiles:
ls /etc/apparmor.d/• Logs:
dmesg | grep apparmor• Create:
aa-genprof, aa-logprof
DevOps MAC Strategy
SELinux and AppArmor are not enemies - they're additional security layers that make your systems more resilient. Think of them as seatbelts: you hope you never need them, but you're glad they're there when something goes wrong.
Remember: The goal isn't to disable MAC when you encounter problems, but to understand and configure it properly. A well-configured MAC system can prevent 90% of post-exploitation activities if an application gets compromised.
Next Steps: Start with complain/permissive mode for your applications, monitor the logs, gradually tighten policies, and only move to enforce mode when you're confident the policies are correct.