# Essential logs to watch For an Ubuntu server administrator, I recommend monitoring logs in **six major areas: kernel/hardware, storage, system services, security, networking, and applications**. ## 1. System and Kernel Logs — highest priority ### Kernel messages ``` journalctl -k dmesg -T ```

Watch for: - `I/O error` - `EXT4-fs error` - `Buffer I/O error` - `reset` - `disconnect` - `USB error` - `Out of memory` - `OOM killer` - `segfault` - `hardware error` - `thermal` - `watchdog` For live monitoring: ``` journalctl -kf ```

--- ## 2. Storage and Disk Health ⚠️ Especially important for servers running databases, Docker, Nextcloud, Immich, etc. ### Filesystem errors ``` journalctl -k | grep -iE "error|ext4|xfs|btrfs|i/o" ```

Critical messages include: ``` EXT4-fs error Buffer I/O error blk_update_request I/O error device offline reset ```

### Disk SMART health ``` sudo smartctl -a /dev/sda ```

Monitor:

SMART Attribute	Concern
Reallocated\_Sector\_Ct	Growing
Current\_Pending\_Sector	> 0
Offline\_Uncorrectable	> 0
UDMA\_CRC\_Error\_Count	Increasing
Temperature	High
Power\_On\_Hours	Aging disks

Run long tests periodically: ``` sudo smartctl -t long /dev/sda ```

--- ## 3. Memory and OOM Events ### OOM killer ``` journalctl | grep -iE "out of memory|oom|killed process" ```

Live: ``` journalctl -f | grep -iE "oom|memory" ```

Check memory: ``` free -h ```

Swap: ``` swapon --show ```

For servers, unexpected OOM events are **high priority** because they may silently kill: - MariaDB - PHP-FPM - Redis - Docker containers - Nextcloud background processes --- # 4. Failed System Services One of the most useful administrator commands: ``` systemctl --failed ```

Check system errors: ``` journalctl -p err..alert ```

Recent boot errors: ``` journalctl -b -p warning ```

Useful services to watch: ``` systemctl status nginx systemctl status php8.3-fpm systemctl status mariadb systemctl status redis-server systemctl status docker ```

Depending on your PHP version: ``` systemctl list-units --type=service ```

--- # 5. Nginx Web Server For web servers: ``` tail -f /var/log/nginx/error.log ```

Access log: ``` tail -f /var/log/nginx/access.log ```

Important errors: ``` 502 Bad Gateway 504 Gateway Timeout connect() failed upstream timed out permission denied too many open files ```

For error frequency: ``` grep -i "error" /var/log/nginx/error.log ```

--- # 6. PHP-FPM Check: ``` journalctl -u php8.3-fpm ```

Live: ``` journalctl -fu php8.3-fpm ```

Watch for: - PHP worker crashes - `server reached pm.max_children` - Out of memory - Slow requests - Permission errors - Session errors A particularly important message: ``` server reached pm.max_children ```

This usually means PHP-FPM is saturated. --- # 7. MariaDB / MySQL Database Check logs: ``` journalctl -u mariadb ```

Live: ``` journalctl -fu mariadb ```

Watch for: - InnoDB corruption - Disk full - Connection exhaustion - Crashes - Table corruption Useful checks: ``` mysqladmin status ```

Connections: ``` mysqladmin processlist ```

--- # 8. Docker For your type of server setup, Docker should be monitored continuously. ### Docker daemon ``` journalctl -u docker ```

Live: ``` journalctl -fu docker ```

### Container status ``` docker ps ```

Stopped containers: ``` docker ps -a ```

Container logs: ``` docker logs CONTAINER_NAME ```

Live: ``` docker logs -f CONTAINER_NAME ```

Check Docker disk usage: ``` docker system df ```

This is important because Docker logs and images can unexpectedly fill the filesystem. --- # 9. Disk Space 🚨 Run: ``` df -h ```

Also check inode exhaustion: ``` df -ih ```

Administrators should generally investigate:

Usage	Status
< 70%	Normal
70–85%	Monitor
85–95%	Warning
>95%	Critical

Find large directories: ``` sudo du -xh / --max-depth=1 2>/dev/null | sort -h ```

For logs: ``` journalctl --disk-usage ```

Reduce old journals if necessary: ``` sudo journalctl --vacuum-time=14d ```

--- # 10. Network Problems Check networking services: ``` journalctl -u NetworkManager ```

For Ubuntu Server using systemd-networkd: ``` journalctl -u systemd-networkd ```

Current network connections: ``` ss -tulpn ```

Network errors: ``` ip -s link ```

Watch for: - RX errors - TX errors - Dropped packets - Interface resets --- # 11. SSH and Security 🔐 Ubuntu authentication log: ``` sudo tail -f /var/log/auth.log ```

Failed SSH attempts: ``` grep "Failed password" /var/log/auth.log ```

Successful logins: ``` grep "Accepted" /var/log/auth.log ```

Current users: ``` who ```

Recent logins: ``` last ```

Failed login database: ``` lastb ```

--- # 12. System Reboots and Unexpected Shutdowns Check reboot history: ``` last reboot ```

Previous boot: ``` journalctl -b -1 ```

Check for unexpected shutdown: ``` journalctl | grep -iE "shutdown|reboot|panic|crash" ```

Kernel panic: ``` journalctl -k | grep -i panic ```

--- # 13. Cron and Scheduled Jobs Depending on Ubuntu configuration: ``` journalctl -u cron ```

Or: ``` grep CRON /var/log/syslog ```

Important for: - Nextcloud cron - Backups - Database dumps - SSL certificate renewal - Monitoring scripts --- # 14. Nextcloud-specific Monitoring For your Nextcloud servers, I would specifically watch: ``` sudo -u www-data php /var/www/nextcloud/occ status ```

Cron: ``` sudo -u www-data php /var/www/nextcloud/occ background:cron ```

Logs: ``` tail -f /path/to/nextcloud/data/nextcloud.log ```

Watch for: - Database errors - Redis connection failures - File locking problems - PHP memory exhaustion - Background job failures - WebDAV errors - WOPI/Collabora errors --- # ⭐ My Recommended Administrator Dashboard For a self-hosted Ubuntu server, these are the **most important things to continuously monitor**: ``` ┌───────────────────────────────────────┐ │ Ubuntu Server Monitoring │ ├───────────────────────────────────────┤ │ 🔴 Disk I/O / Filesystem errors │ │ 🔴 SMART disk health │ │ 🔴 Disk space / inode usage │ │ 🔴 OOM / memory exhaustion │ │ 🔴 Failed systemd services │ │ 🔴 Docker container failures │ │ 🟠 Nginx 502/504 errors │ │ 🟠 PHP-FPM saturation │ │ 🟠 MariaDB errors │ │ 🟠 Network interface errors │ │ 🟠 SSH/security attacks │ │ 🟡 Backup failures │ │ 🟡 SSL certificate expiration │ │ 🟡 Temperature / hardware issues │ └───────────────────────────────────────┘ ```

## 🚀 My recommended daily commands ``` # Critical system errors journalctl -p err..alert --since today # Failed services systemctl --failed # Disk usage df -h # Memory free -h # Docker containers docker ps -a # Kernel/hardware errors journalctl -k -p err..alert # Recent authentication failures grep "Failed password" /var/log/auth.log | tail ```

### For your Ubuntu servers specifically Given that you operate **multiple Ubuntu servers with Nextcloud, Nginx, PHP-FPM, MariaDB, Redis, Docker, Immich, and other self-hosted services**, I would prioritize: **1. Disk I/O errors → 2. SMART health → 3. OOM events → 4. disk capacity → 5. failed services → 6. Docker container health → 7. Nginx/PHP/MariaDB errors.**