Linux Log Forensics: auth.log, Journal, auditd and wtmp

TL;DR. A Linux host records the same intrusion in up to four independent places: the text logs written by rsyslog (auth.log, secure, syslog, messages), the binary systemd journal, the audit log written by auditd, and the binary login records (wtmp, btmp, utmp, lastlog, or their SQLite successors). Each one answers a different question, each one can be missing for a benign reason, and an attacker rarely cleans all four. The investigation is the comparison.

Four sources, four questions

The systemd journal and rsyslog usually hold overlapping copies of the same line, written and rotated independently. auditd is fed by the kernel, not by syslog. The wtmp and btmp files are written by login, sshd and PAM directly. That independence is what makes a merged timeline more than a convenience: it is how deletion and editing become visible.

What one SSH login looks like everywhere

Take a single password login by svc_backup from 203.0.113.45:

• auth.log: sshd[3071]: Accepted password for svc_backup from 203.0.113.45 port 51544 ssh2, then pam_unix(sshd:session): session opened, then systemd-logind[845]: New session 7 of user svc_backup.
• Journal: the same three messages, each with _PID, _UID, _SYSTEMD_UNIT=ssh.service and a microsecond UTC timestamp.
• audit.log: USER_AUTH, USER_ACCT, LOGIN (which sets auid=1001 ses=114), USER_START and USER_LOGIN with addr=203.0.113.45 res=success.
• wtmp: a USER_PROCESS record on pts/1 with host 203.0.113.45; at logout, a DEAD_PROCESS record on the same line.
• lastlog: the slot for UID 1001 overwritten with the time, pts/1 and 203.0.113.45.

Three session identifiers appear: the sshd PID (3071), the systemd-logind session number (7) and the audit session (ses=114). They are different numbers for the same login. Rebuilding a session means linking them: the PAM session opened and session closed lines share the sshd PID, the logind session is announced within milliseconds of it, and every audit record in between carries ses=114, including commands run later in a root shell.

Where the sources disagree, and why

Disagreement is normal. Before calling it tampering, rule out the benign explanations:

• No auth.log at all. Debian 12 and recent Fedora installs may not run rsyslog. The journal is then the only system log.
• Journal starts later than auth.log. The journal is size-limited (10% of the file system, capped at 4G by default) and may be volatile (/run/log/journal) if persistent storage is not enabled.
• SSH command with no wtmp record. ssh host command, scp and sftp do not allocate a terminal and often leave no wtmp entry.
• No execve records. No audit rule was loaded for execve. Read /etc/audit/rules.d/ first.
• Different times. Traditional syslog lines are in local time without a year; the journal and audit log are UTC. A one-hour offset is usually a time zone, not an attacker.

What remains after these checks is interesting: auth lines present in the journal but missing from auth.log, gaps in the journal sequence numbers, a wtmp record that is all zeros, a lastlog time with no matching wtmp session. The tampering guide walks through each of these.

Normalising time first

A merged timeline is only as good as its clock handling:

• RFC 3164 syslog (Sep 14 10:02:11) has no year and no zone. The zone comes from /etc/localtime or /etc/timezone on the image; the year is counted back from the file's modification time, watching for a year rollover inside the file. See RFC 3164 syslog.
• RFC 3339 high-precision lines (2026-09-14T10:02:11.482113+00:00), the default on Ubuntu 24.04 and Debian 12+, carry their offset and microseconds.
• RFC 5424 lines from forwarders carry a full timestamp as well.
• Journal times are microseconds since the epoch, UTC. journalctl shows _SOURCE_REALTIME_TIMESTAMP (when the message was produced) when the entry has one.
• audit.log stamps are msg=audit(1789380131.486:181296): epoch seconds, milliseconds, then a serial.
• wtmp/btmp/lastlog hold epoch seconds (plus microseconds in utmp records).

Convert everything to UTC and keep a note of how each time was obtained (explicit offset, zone file, or guessed year).

Deduplicate, then correlate

On a host running both rsyslog and journald, every sshd line exists twice. Counting both inflates brute-force numbers and clutters the timeline. Keep the journal copy (it has trusted fields and microseconds) and mark the text line as its duplicate, but keep the text line available: if the journal copy is missing, that difference is itself a finding.

Then correlate by session: sshd PID pairs for PAM open and close, logind session numbers, audit ses, and wtmp login and logout records. A session that appears in three sources but not in wtmp is exactly what a log cleaner leaves behind.

Doing it in the browser

Linux Log Parser does these steps on the files you drop: it reads text logs (RFC 3164 with year inference and zone conversion, RFC 3339, RFC 5424, rotated and .gz generations), binary journal files including compact mode and compressed fields, journalctl -o export and -o json, audit.log in RAW or ENRICHED format, wtmp/btmp/utmp for x86_64 and aarch64 layouts, lastlog, wtmp.db and lastlog2.db. It builds one UTC timeline, marks auth.log lines that the journal also holds, rebuilds sessions and lists findings such as password guessing followed by a success, root shells, cron and systemd persistence, journal gaps and blanked wtmp records. Everything runs locally in WebAssembly; nothing is uploaded.

Where to go next

• Collect the logs from a live host, a triage tool or a disk image.
• Read auth.log and secure for SSH, sudo and account changes.
• Journal forensics: file format, sequence numbers and .journal~ files.
• auditd EXECVE forensics: auid, ses and hex-encoded arguments.
• wtmp, btmp and lastlog, including wtmpdb and lastlog2.
• A full investigation walkthrough on the tool's synthetic sample.

For per-artifact reference tables, the cheat sheets on linuxforensics.app cover the systemd journal, auth.log and syslog, auditd, wtmp, btmp and lastlog, sudo logs and SSH artifacts.