Effective disk usage monitoring is essential for maintaining system health and performance. This comprehensive guide covers everything from basic disk space checking with df and du to advanced monitoring techniques and automated alerting. Master the art of disk space management and prevent storage-related issues before they impact your system.
Disk Usage Tools Comparison
| Tool Category | Primary Tools | Purpose | Skill Level |
|---|---|---|---|
| Basic Monitoring | df, du | Filesystem and directory space | Beginner |
| Interactive Analysis | ncdu, gdu | Interactive disk usage navigation | Intermediate |
| Advanced Scanning | find, fdupes, rmlint | File discovery and cleanup | Intermediate |
| Real-time Monitoring | iotop, iostat, watch | Live disk activity monitoring | Advanced |
| Automated Alerting | cron, systemd, custom scripts | Proactive disk space management | Advanced |
| Visualization | duf, dfc, filelight | Graphical disk usage representation | Intermediate |
• Check filesystem space:
df -h• Check directory size:
du -sh /path• Interactive analysis:
ncdu /path• Find large files:
find / -size +100M• Monitor in real-time:
watch -n 1 'df -h'• Check inode usage:
df -i• Sort by size:
du -h --max-depth=1 | sort -hr• Set up alerts for disk space thresholds
• Regular cleanup of temp files and logs
• Monitor growth trends for capacity planning
Disk Usage Analysis Workflow
Disk Space Management Process
Essential Disk Usage Tools
| Tool | Command Examples | Purpose | Key Output |
|---|---|---|---|
| df | df -hdf -i |
Filesystem space usage | Used/available space, inodes |
| du | du -sh /homedu -h --max-depth=1 |
Directory space usage | Directory sizes, file sizes |
| ncdu | ncdu /varncdu -x / |
Interactive disk analysis | Navigable size hierarchy |
| find | find / -size +100Mfind /var -name "*.log" |
File discovery by criteria | Matching files list |
| ls | ls -lhls -laSh |
File listing with sizes | File details, sorted sizes |
| stat | stat file.txtstat -c%s file.txt |
File statistics | Size, inodes, timestamps |
Size Visualization
Basic Disk Usage Commands
Display filesystem disk space usage and availability.
Common Operations:
# Human-readable output
df -h
# Show inode usage instead of block usage
df -i
# Show specific filesystem type
df -t ext4
df -t xfs
# Exclude specific filesystem types
df -x tmpfs -x devtmpfs
# Show available space in bytes
df -B1
# Show only local filesystems
df -l
# Continuous monitoring
watch -n 5 'df -h'
# Show filesystem type
df -T
# Sort by usage percentage
df -h | sort -k5 -hr
# Show specific mount point
df -h /home
df -h /var/log
Estimate file and directory space usage.
Directory Analysis:
# Summary of directory size
du -sh /home
# Show sizes of all subdirectories
du -h --max-depth=1 /var
# Sort by size (largest first)
du -h --max-depth=1 / | sort -hr
# Show apparent sizes rather than disk usage
du -sh --apparent-size /path
# Exclude certain patterns
du -sh --exclude="*.log" /var
# One filesystem only (don't cross mount points)
du -x -h --max-depth=1 /
# Show both summary and individual files
du -ch /var/log/*
# Human readable with custom block size
du -B M /path # Show in megabytes
du -B G /path # Show in gigabytes
# Time-based analysis
du -h --time /path
Interactive disk usage analyzer with ncurses interface.
Interactive Analysis:
# Install ncdu
sudo apt install ncdu
# Scan directory interactively
ncdu /var
# Scan with exclude patterns
ncdu --exclude "*.log" /home
# Don't cross filesystem boundaries
ncdu -x /
# Export results to file
ncdu -o report.txt /var
# Import and view exported scan
ncdu -f report.txt
# Color scheme for different terminals
ncdu --color dark /path
# Extended information display
ncdu -e /path
# Keyboard shortcuts in ncdu:
# ↑↓ - Navigate
# → - Enter directory
# ← - Leave directory
# d - Delete selected
# n - Sort by name
# s - Sort by size
# r - Recalculate size
# q - Quit
Advanced Usage Analysis
Locate large files and directories efficiently.
File Discovery:
# Find files larger than 100MB
find / -type f -size +100M
# Find files larger than 1GB
find /home -type f -size +1G
# Find and display sizes
find /var -type f -size +10M -exec ls -lh {} \;
# Find and sort by size
find / -type f -size +50M -exec du -h {} \; | sort -hr
# Find empty files and directories
find /tmp -empty
find /var -type d -empty
# Find by modification time (older than 30 days)
find /var/log -type f -mtime +30
# Find by access time (not accessed in 90 days)
find /home -type f -atime +90
# Find and delete old files (careful!)
find /tmp -type f -mtime +7 -delete
# Find files with specific extensions
find / -type f -name "*.log" -size +100M
# Find and compress old logs
find /var/log -name "*.log" -mtime +30 -exec gzip {} \;
Monitor disk usage and activity in real-time.
Live Monitoring:
# Monitor disk space every 2 seconds
watch -n 2 'df -h'
# Monitor specific filesystems
watch -n 5 'df -h / /home /var'
# Monitor directory growth
watch -n 1 'du -sh /var/log'
# I/O monitoring
sudo iotop
sudo iotop -o # Only show active I/O
# Disk statistics
iostat -x 1
iostat -dx 2
# Monitor file system events
inotifywait -m -r /var/log
# Continuous disk usage with highlighting
watch -d 'df -h'
# Monitor with custom command
watch -n 10 'du -h --max-depth=1 /var | sort -hr'
# Log disk usage over time
while true; do
echo "$(date): $(df -h / | awk 'NR==2{print $5}')" >> disk.log
sleep 300
done
Specialized tools for detailed disk usage analysis.
Specialized Analysis:
# baobab - graphical disk usage analyzer
baobab /home
# gdu - fast disk usage analyzer
sudo apt install gdu
gdu /var
# duf - better df alternative
sudo apt install duf
duf
duf --only local
# dust - more intuitive du
cargo install du-dust
dust /var
# fdupes - find duplicate files
sudo apt install fdupes
fdupes -r /home
# rmlint - find lint (duplicates, empties, etc)
sudo apt install rmlint
rmlint /home
# plocate - fast file finding
sudo apt install plocate
plocate "*.log" | head -20
# filelight - KDE disk usage visualizer
sudo apt install filelight
filelight
Disk Usage Thresholds & Monitoring
Typical Disk Usage Thresholds
Advanced Monitoring Tools
| Tool Category | Tools | Primary Use | Installation |
|---|---|---|---|
| Basic Monitoring | df, du, ls | Quick space checking | Built-in |
| Interactive Analysis | ncdu, gdu, dust | Detailed exploration | sudo apt install ncdu |
| File Discovery | find, locate, fdfind | Finding specific files | Built-in |
| Real-time Monitoring | watch, iotop, inotify | Live activity tracking | Built-in |
| Duplicate Finding | fdupes, rmlint | Space reclamation | sudo apt install fdupes |
Practical Disk Usage Scenarios
Real-World Disk Usage Examples
# 1. Complete Disk Usage Report Script
#!/bin/bash
echo "=== Disk Usage Report $(date) ==="
echo -e "\n1. Filesystem Usage:"
df -h
echo -e "\n2. Inode Usage:"
df -i
echo -e "\n3. Large Directories in /:"
du -h --max-depth=1 / 2>/dev/null | sort -hr | head -10
echo -e "\n4. Large Files (>100MB):"
find / -type f -size +100M 2>/dev/null | xargs ls -lh | head -10
echo -e "\n5. Log Directory Analysis:"
du -h --max-depth=1 /var/log 2>/dev/null | sort -hr
# 2. Automated Cleanup Script
#!/bin/bash
THRESHOLD=90
CURRENT_USAGE=$(df / --output=pcent | tail -1 | tr -d '% ')
if [ $CURRENT_USAGE -gt $THRESHOLD ]; then
echo "Disk usage is ${CURRENT_USAGE}%. Starting cleanup..."
# Clean package cache
sudo apt-get autoremove -y
sudo apt-get autoclean -y
# Clean temporary files
sudo find /tmp -type f -atime +7 -delete
sudo find /var/tmp -type f -atime +7 -delete
# Clean old logs
sudo find /var/log -name "*.log" -type f -mtime +30 -delete
sudo journalctl --vacuum-time=7d
# Clean thumbnail cache
rm -rf ~/.cache/thumbnails/*
echo "Cleanup completed."
else
echo "Disk usage (${CURRENT_USAGE}%) is below threshold (${THRESHOLD}%)."
fi
# 3. Directory Size Monitor
#!/bin/bash
WATCH_DIR="/var/log"
MAX_SIZE="10G"
while true; do
CURRENT_SIZE=$(du -s $WATCH_DIR | cut -f1)
if [ $CURRENT_SIZE -gt ${MAX_SIZE%G}000000 ]; then
echo "WARNING: $WATCH_DIR size ($(du -sh $WATCH_DIR | cut -f1)) exceeds $MAX_SIZE"
# Add alert actions here
fi
sleep 300
done
# 4. Find and Archive Old Logs
#!/bin/bash
LOG_DIR="/var/log"
ARCHIVE_DIR="/archive/logs"
RETENTION_DAYS=30
echo "Archiving logs older than $RETENTION_DAYS days..."
find $LOG_DIR -name "*.log" -type f -mtime +$RETENTION_DAYS | while read file; do
archive_file="$ARCHIVE_DIR/$(basename $file).$(date +%Y%m%d).gz"
echo "Archiving $file to $archive_file"
sudo gzip -c "$file" > "$archive_file"
sudo truncate -s 0 "$file"
done
# 5. Disk Usage Alert System
#!/bin/bash
THRESHOLD=85
ALERT_EMAIL="admin@example.com"
df -H | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output; do
usage=$(echo $output | awk '{ print $1}' | cut -d'%' -f1)
partition=$(echo $output | awk '{ print $2 }')
if [ $usage -ge $THRESHOLD ]; then
echo "Alert: Partition $partition is ${usage}% full" | \
mail -s "Disk Space Alert: $partition ${usage}%" $ALERT_EMAIL
fi
done
# 6. Interactive Disk Analyzer
#!/bin/bash
echo "Interactive Disk Analyzer"
PS3="Select an option: "
options=("Quick Overview" "Large Files Search" "Directory Analysis" "Cleanup Suggestions" "Quit")
select opt in "${options[@]}"; do
case $opt in
"Quick Overview")
echo "=== Quick Overview ==="
df -h
echo -e "\nTop 5 largest directories in /home:"
du -h --max-depth=1 /home 2>/dev/null | sort -hr | head -6
;;
"Large Files Search")
read -p "Enter minimum size (e.g., 100M): " size
echo "Finding files larger than $size..."
find / -type f -size "+$size" 2>/dev/null | xargs ls -lh | head -20
;;
"Directory Analysis")
read -p "Enter directory to analyze: " dir
if [ -d "$dir" ]; then
ncdu "$dir"
else
echo "Directory not found: $dir"
fi
;;
"Cleanup Suggestions")
echo "=== Cleanup Suggestions ==="
echo "1. Package cache: $(du -sh /var/cache/apt 2>/dev/null || echo 'N/A')"
echo "2. Log files: $(du -sh /var/log 2>/dev/null || echo 'N/A')"
echo "3. Temporary files: $(du -sh /tmp 2>/dev/null || echo 'N/A')"
echo "4. User caches: $(du -sh ~/.cache 2>/dev/null || echo 'N/A')"
;;
"Quit")
break
;;
*) echo "Invalid option";;
esac
done
# 7. Duplicate File Finder
#!/bin/bash
SEARCH_DIR="/home"
RESULTS_FILE="/tmp/duplicates.txt"
echo "Searching for duplicate files in $SEARCH_DIR..."
fdupes -r "$SEARCH_DIR" > "$RESULTS_FILE"
if [ -s "$RESULTS_FILE" ]; then
echo "Found duplicates. Review $RESULTS_FILE"
echo "Total duplicates found: $(grep -c "^$" "$RESULTS_FILE")"
else
echo "No duplicates found."
rm -f "$RESULTS_FILE"
fi
# 8. Growth Trend Analysis
#!/bin/bash
LOG_FILE="/var/log/disk_usage.log"
echo "$(date),$(df / --output=used | tail -1),$(df / --output=avail | tail -1)" >> "$LOG_FILE"
# Generate weekly report
if [ $(date +%u) -eq 1 ]; then # Monday
echo "=== Weekly Disk Usage Report ==="
echo "Date,Used,Available"
tail -7 "$LOG_FILE"
fi
# 9. Docker Disk Cleanup
#!/bin/bash
echo "Cleaning up Docker disk usage..."
docker system df
echo -e "\nRemoving unused containers, networks, and images..."
docker system prune -f
echo -e "\nRemoving build cache..."
docker builder prune -f
echo -e "\nCurrent Docker disk usage:"
docker system df
# 10. MySQL/PostgreSQL Table Size Analysis
#!/bin/bash
# MySQL version
mysql -e "
SELECT table_schema as 'Database',
ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) as 'Size (MB)'
FROM information_schema.TABLES
GROUP BY table_schema
ORDER BY SUM(data_length + index_length) DESC;"
# PostgreSQL version
psql -c "
SELECT datname, pg_size_pretty(pg_database_size(datname))
FROM pg_database
ORDER BY pg_database_size(datname) DESC;"
# 11. Web Server Log Analysis
#!/bin/bash
LOG_DIR="/var/log/nginx"
echo "Web Server Log Analysis:"
echo "Total log size: $(du -sh $LOG_DIR)"
echo "Number of log files: $(find $LOG_DIR -name "*.log" | wc -l)"
echo "Largest log files:"
find $LOG_DIR -name "*.log" -exec du -h {} \; | sort -hr | head -5
# 12. User Home Directory Quotas
#!/bin/bash
echo "User Home Directory Sizes:"
for user in /home/*; do
if [ -d "$user" ]; then
size=$(du -sh "$user" 2>/dev/null | cut -f1)
echo "$(basename $user): $size"
fi
done | sort -k2 -hr
# 13. Backup Size Monitor
#!/bin/bash
BACKUP_DIR="/backup"
MAX_BACKUP_SIZE="100G"
CURRENT_SIZE=$(du -s $BACKUP_DIR | cut -f1)
MAX_SIZE_BYTES=$(echo $MAX_BACKUP_SIZE | sed 's/G/*1024*1024*1024/' | bc)
if [ $CURRENT_SIZE -gt $MAX_SIZE_BYTES ]; then
echo "Backup size ($(du -sh $BACKUP_DIR | cut -f1)) exceeds limit ($MAX_BACKUP_SIZE)"
echo "Rotating backups..."
# Add backup rotation logic here
fi
# 14. Container Storage Analysis
#!/bin/bash
echo "Container Storage Analysis:"
echo "Docker:"
docker system df
echo -e "\nPodman:"
podman system df
echo -e "\nLXC:"
lxc storage list
echo -e "\nKubernetes PVCs:"
kubectl get pvc --all-namespaces
# 15. Automated Report Generator
#!/bin/bash
REPORT_FILE="/var/log/disk-report-$(date +%Y%m%d).txt"
{
echo "Disk Usage Report - $(date)"
echo "========================"
echo -e "\nFilesystem Overview:"
df -h
echo -e "\nInode Usage:"
df -i
echo -e "\nLarge Directories:"
du -h --max-depth=1 / 2>/dev/null | sort -hr | head -10
echo -e "\nLarge Files:"
find / -type f -size +500M 2>/dev/null | xargs ls -lh 2>/dev/null | head -10
} > $REPORT_FILE
echo "Report generated: $REPORT_FILE"
Common Disk Usage Issues & Solutions
Space Exhaustion
- Root Partition Full: Clean package cache, remove old logs, check for large core dumps
- Home Directory Full: Clean user caches, find large media files, implement quotas
- Var Partition Full: Rotate logs, clean temporary files, check database sizes
- Temp Space Full: Clear /tmp, check for stuck processes creating large temp files
Inode Exhaustion
- Too Many Small Files: Combine small files, increase inode count at filesystem creation
- Email Server Issues: Clean mail queues, implement proper mail retention policies
- Web Server Cache: Clear cache directories, implement cache size limits
- Application Logs: Implement log rotation, use binary logs where possible
Performance Issues
- Slow I/O: Monitor disk usage patterns, identify I/O-heavy processes
- Fragmentation: Defragment filesystems (ext4), consider XFS for better performance
- Cache Issues: Adjust vm.dirty_ratio, monitor memory usage
- Network Storage: Optimize NFS/CIFS mount options, check network latency
Monitoring Gaps
- No Proactive Alerts: Implement disk space monitoring with thresholds
- Missing Trends: Set up historical data collection for capacity planning
- Container Storage: Monitor Docker/Podman storage, clean unused images
- Database Growth: Monitor table sizes, implement archiving strategies
Advanced Monitoring Techniques
Automated Alerting
Set up proactive disk space monitoring and alerts.
Alerting Setup:
# Cron-based disk check
# Add to crontab: 0 * * * * /usr/local/bin/disk-check.sh
# Systemd service for disk monitoring
# /etc/systemd/system/disk-monitor.service
[Unit]
Description=Disk Space Monitor
[Service]
Type=oneshot
ExecStart=/usr/local/bin/disk-alert.sh
# Systemd timer
# /etc/systemd/system/disk-monitor.timer
[Unit]
Description=Disk Space Monitor Timer
[Timer]
OnCalendar=*-*-* *:0:0
Persistent=true
[Install]
WantedBy=timers.target
# Prometheus node_exporter metrics
# Monitor with: node_filesystem_avail_bytes
# Zabbix disk monitoring
# Use: vfs.fs.size[/,pfree]
Capacity Planning
Analyze growth trends and plan for future storage needs.
Capacity Analysis:
# Collect historical data
#!/bin/bash
LOG_FILE="/var/log/disk-growth.log"
echo "$(date +%s),$(df / --output=used | tail -1)" >> $LOG_FILE
# Analyze growth rate
awk -F, '{print $2}' disk-growth.log | \
awk 'NR==1{first=$1} END{print (($1-first)/NR/86400)" KB/day"}'
# Predict exhaustion date
CURRENT=$(df / --output=used | tail -1)
TOTAL=$(df / --output=size | tail -1)
GROWTH_RATE=1000000 # 1MB/day growth
DAYS_LEFT=$(( (TOTAL - CURRENT) / GROWTH_RATE ))
echo "Estimated days until full: $DAYS_LEFT"
# Generate growth chart (requires gnuplot)
gnuplot << EOF
set terminal png
set output "disk_growth.png"
set xdata time
set timefmt "%s"
set format x "%Y-%m-%d"
plot "disk-growth.log" using 1:2 with lines
EOF
Container Storage
Monitor and manage disk usage in container environments.
Container Storage Management:
# Docker disk usage
docker system df
docker system df -v
# Docker cleanup
docker system prune -a -f
docker volume prune -f
# Podman disk usage
podman system df
podman system prune -a -f
# Kubernetes persistent volumes
kubectl get pvc --all-namespaces
kubectl describe pvc
# Container storage drivers
docker info | grep -A10 "Storage Driver"
# Thin pool monitoring (LVM)
lvs docker-thinpool
lvs docker-thinpool -o+metadata_percent
# Build cache management
docker builder prune -a -f
docker buildx prune -a -f
• Always have at least 10-15% free space for filesystem health and performance
• Monitor inode usage separately from disk space usage
• Set up alerts before reaching critical levels (85%+ usage)
• Regularly clean temporary files and application caches
• Implement log rotation and retention policies
• Monitor container storage separately from host storage
• Keep historical data for capacity planning and trend analysis
• Test cleanup scripts in non-production environments first
• Have a rollback plan for any major cleanup operations
• Use
ncdu for interactive analysis of complex directory structures• Set up
cron jobs for regular disk space checks and cleanups• Use
find -mtime to locate and archive old files• Monitor
/proc/sys/fs/file-nr for file handle usage• Use
lsof | grep deleted to find processes holding deleted files• Implement disk quotas for multi-user systems
• Use
zfs or btrfs for built-in compression and deduplication• Set up centralized logging to reduce local disk usage
• Use
tmpfs for frequently accessed temporary data
Key Takeaways
Effective disk usage monitoring requires a combination of basic tools for quick checks, interactive tools for detailed analysis, and automated systems for proactive management. By understanding the different aspects of disk usage - from basic space monitoring to inode management and growth trend analysis - you can prevent storage-related issues and maintain optimal system performance.
Next Step: Explore advanced storage topics like distributed filesystems, storage performance tuning, automated storage management with tools like Ansible, and cloud storage optimization techniques.