7 Linux Commands That Make Log Troubleshooting Faster
In this article, you'll use journalctl, tail, grep, less, awk, zgrep, and dmesg to find errors in Linux logs faster and trace their root cause.
When something breaks on a Linux server, the logs usually already know what happened, but the hard part is finding the right five lines among the hundreds of thousands written since the last reboot, especially when users are waiting and the website is returning errors.
Many admins start by opening a log file in a text editor and keep scrolling that works on a quiet test box, but on a busy production server the file may be several gigabytes, and it keeps growing while you read it, and the error you need may be sitting in yesterday's rotated and compressed copy.
The good news is that Linux already ships with a small set of commands that turn log hunting into a quick, repeatable process.
In this guide, we will walk through seven of them, starting with the systemd journal and ending with the kernel's own messages, and then combine them to trace a real 502 error back to its cause.
Prerequisites
Before you start, make sure you have the following:
- A Linux server running a systemd-based distribution such as Ubuntu 24.04+, Debian 12, Rocky Linux 9/10, RHEL 9/10, or AlmaLinux 9/10.
- A user with sudo privileges, because most log files under
/var/logare readable only by root or theadmgroup (on Ubuntu and Debian), so regular users often get a "Permission denied" error.
Log file locations differ between distribution families, and that difference catches a lot of people out.
- Ubuntu and Debian write general system messages to
/var/log/syslogand authentication events to/var/log/auth.log. - Rocky Linux, RHEL, and AlmaLinux use
/var/log/messagesand/var/log/securefor the same purposes.
Debian also does not install rsyslog by default, which means /var/log/syslog may not exist at all. On such systems, the systemd journal is the only place those messages live, which is one more reason to get comfortable with journalctl.
Log file locations differ between distribution families, and that difference catches a lot of people out. Ubuntu and Debian write general system messages to /var/log/syslog and authentication events to /var/log/auth.log, while Rocky Linux, RHEL, and AlmaLinux use /var/log/messages and /var/log/secure for the same purposes.
Debian 12 also does not install rsyslog by default, which means /var/log/syslog may not exist at all. On such systems, the systemd journal is the only place those messages live, which is one more reason to get comfortable with journalctl.