Log Analysis: Spotting Unusual Activity

TL;DR

We’ll look at how to check if your log entries are normal by using basic tools and techniques. This guide focuses on identifying unexpected patterns, failed logins, and unusual system activity. It’s about finding things that *shouldn’t* be happening.

Checking Your Logs

1. Understand Your Log Sources: First, know where your logs come from. Common sources include:
   • System Logs (Linux): Usually in /var/log/syslog or /var/log/auth.log
   • Event Logs (Windows): Accessed via Event Viewer.
   • Application Logs: Specific to the software running on your systems – check documentation for locations.
   • Firewall Logs: Records of network traffic, often stored by your firewall appliance or software.
2. Basic Log Viewing: Use simple tools to get started.
   • Linux: tail -f /var/log/syslog (shows the log as it updates). grep 'failed password' /var/log/auth.log (searches for failed login attempts)
   • Windows: Open Event Viewer (search in Start Menu), navigate to Windows Logs > Security.
3. Look for Failed Login Attempts: A high number of failed logins is a red flag.
   1. Linux Example:

      grep 'Failed password' /var/log/auth.log | wc -l

      This counts the number of failed login attempts.

   2. Windows Example: Filter Event ID 4625 in Event Viewer (failed login). Look for patterns – same username repeatedly failing, logins from unusual IP addresses.
4. Identify Unusual System Activity: Check for processes starting unexpectedly or at odd times.
   1. Linux Example:

      journalctl -xe

      (shows system logs with explanations). Look for errors or warnings about unknown programs.

   2. Windows Example: Filter Event ID 4688 in Event Viewer (new process created). Investigate processes you don’t recognise.
5. Check for Network Connections: Look for connections to strange IP addresses or ports.
   1. Linux Example:

      netstat -tulnp

      (shows listening ports and associated processes). ss -tulnp is a modern alternative.

   2. Windows Example: Use Resource Monitor > Network tab to see active connections. Use tcpview for more detailed information.
6. Time-Based Analysis: Look at logs over specific periods.
   • If you suspect an incident, focus on the time around when it occurred.
   • Use tools like grep with date/time filters (e.g., grep '2024-10-27' /var/log/syslog).
7. IP Address Reputation: Check suspicious IP addresses against known threat lists.
   • Use websites like AbuseIPDB or VirusTotal to see if an IP address has been reported for malicious activity.
8. Correlation: Combine information from multiple log sources.
   • If you see a failed login attempt followed by unusual network activity, it’s more concerning than either event in isolation.

What’s Normal?

Determining what is ‘normal’ for *your* systems takes time and observation. Keep a baseline of typical log entries so you can easily spot deviations.