Physical Address
Metro Manila, Philippines
Physical Address
Metro Manila, Philippines
When investigating a compromised Linux server, finding malicious processes or suspicious files is only part of the job.
You also need to determine how accounts on the server were accessed.
During our investigation of a compromised Zimbra mail server running on Ubuntu, we examined the zimbra account’s SSH configuration and discovered entries in:
/opt/zimbra/.ssh/authorized_keys
that required investigation.
Among them were additional SSH public keys and an entry containing a forced command.
We then examined:
/var/log/auth.log
and found evidence of successful SSH public-key authentication to the zimbra account from an internal IP address during the incident period.
That raised several important questions:
This article explains how we approached those questions and how Linux administrators can investigate suspicious SSH access without immediately destroying useful evidence.
Customer information, domains, actual IP addresses, SSH public keys, key fingerprints, hostnames, credentials, and other identifying information have been removed.
Important: Finding an unknown SSH key or an
Accepted publickeylog entry does not by itself prove how a server was originally compromised. Treat each finding as evidence that must be correlated with other evidence.
Before discussing general SSH investigation techniques, it is important to distinguish our confirmed findings from the additional checks recommended later in this guide.
During our Zimbra incident, we inspected:
/opt/zimbra/.ssh/authorized_keys
The file contained SSH public-key entries that required investigation.
One entry included a forced-command configuration. There were also additional keys whose legitimacy needed to be verified.
We then examined:
/var/log/auth.log
and found a successful public-key authentication event involving the zimbra account from an internal IP address during the incident period.
A sanitized example of the type of event we investigated looks like:
<DATE> <HOST> sshd[<PID>]: Accepted publickey for zimbra from <INTERNAL_IP> port <SOURCE_PORT> ssh2
Depending on the OpenSSH version and logging configuration, an authentication record may also include the key type and fingerprint.
These findings were significant because we were already investigating suspicious processes and files running under the same zimbra account.
However, we did not establish from this evidence alone that SSH was the original attack vector.
That distinction is important.
/opt/zimbra/.ssh/authorized_keys MattersOpenSSH normally uses an authorized_keys file to determine which public keys are allowed to authenticate as a particular user.
The OpenSSH sshd documentation describes the authorized-keys format as containing, for each key:
options key-type base64-key comment
The options field is optional.
For a normal Linux user, the file is commonly found under:
~/.ssh/authorized_keys
In our case, the zimbra account’s SSH files were under:
/opt/zimbra/.ssh/
because that was the account’s environment on the affected server.
You can verify an account’s configured home directory with:
getent passwd zimbra
A typical result may resemble:
zimbra:x:<UID>:<GID>:Zimbra:/opt/zimbra:/bin/bash
Do not assume that every server uses the same home directory. Verify it.
You should also verify which authorized-key files OpenSSH is actually configured to use.
A useful command is:
sshd -T | grep -i authorizedkeysfile
You may see something similar to:
authorizedkeysfile .ssh/authorized_keys .ssh/authorized_keys2
The effective configuration is more useful than assuming the default.
OpenSSH documents the AuthorizedKeysFile setting in sshd_config.
If you suspect an SSH key is unauthorized, do not immediately delete the line.
First preserve the file.
For example:
mkdir -p /root/incident-evidence
chmod 700 /root/incident-evidence
Copy the file while preserving metadata:
cp -a /opt/zimbra/.ssh/authorized_keys \
/root/incident-evidence/authorized_keys.zimbra
Record its hash:
sha256sum /root/incident-evidence/authorized_keys.zimbra \
> /root/incident-evidence/authorized_keys.zimbra.sha256
Record metadata from the original:
stat /opt/zimbra/.ssh/authorized_keys \
> /root/incident-evidence/authorized_keys.zimbra.stat
You can also record the collection time:
date -Is \
> /root/incident-evidence/collection-time.txt
This creates a simple record of what existed before you modify the account.
For incidents involving legal, regulatory, or disciplinary investigation, follow your organization’s forensic evidence procedures instead of relying only on an operational copy.
Start by examining the SSH directory and key file:
ls -ld /opt/zimbra/.ssh
ls -l /opt/zimbra/.ssh/authorized_keys
For more metadata:
stat /opt/zimbra/.ssh
stat /opt/zimbra/.ssh/authorized_keys
Pay attention to:
owner
group
permissions
modification time
metadata-change time
file size
For example, unexpected ownership would deserve investigation:
root:root
when you expected:
zimbra:zimbra
Likewise, a recent timestamp during the known incident window may help build your timeline.
A timestamp alone does not prove when an attacker added a key.
Files may be copied, restored, edited, or deliberately timestamped. Treat filesystem timestamps as evidence to correlate with other information.
authorized_keys CarefullyTo view the file locally:
cat /opt/zimbra/.ssh/authorized_keys
For easier review with line numbers:
nl -ba /opt/zimbra/.ssh/authorized_keys
A normal key may resemble:
ssh-ed25519 AAAA...REDACTED... administrator@example
An entry containing options may look more complex:
from="<TRUSTED_NETWORK>",restrict ssh-ed25519 AAAA...REDACTED... backup-key
Or:
restrict,command="<FORCED_COMMAND>" ssh-ed25519 AAAA...REDACTED... automation-key
Do not publish the actual key contents when documenting an incident.
The long base64-encoded portion is the public key. Public keys are not secret in the same way private keys are, but publishing production authorization data unnecessarily provides useful information about the environment.
One of the entries we found during our incident contained a forced command.
OpenSSH supports this syntax legitimately:
command="<COMMAND>" ssh-ed25519 AAAA...
When that key successfully authenticates, OpenSSH executes the specified command instead of the command requested by the SSH client.
Forced commands have legitimate uses.
Examples include:
Therefore:
command="..."
does not automatically mean malware.
The correct question is:
Should this key and this forced command exist on this account?
In our case, the forced-command entry required investigation because it appeared on an account already associated with suspicious activity and its legitimacy was not immediately established.
authorized_keys Options to Look ForOpenSSH allows several options before a public key.
Some common ones include:
command="..."
from="..."
restrict
no-pty
no-port-forwarding
no-agent-forwarding
no-X11-forwarding
no-user-rc
permitopen="host:port"
permitlisten="host:port"
environment="NAME=value"
OpenSSH documents these options as part of the authorized-keys format.
Several deserve special attention during an investigation.
command="..."Forces a particular command whenever the key authenticates.
Investigate unexpected commands.
from="..."Restricts where the key may authenticate from.
For example:
from="192.0.2.0/24" ssh-ed25519 AAAA...
A from= restriction can be a legitimate security measure.
OpenSSH supports hostnames, addresses, and CIDR notation for this option.
permitopen=Limits SSH local port forwarding.
For example:
permitopen="127.0.0.1:3306"
This may be completely legitimate for a tunnel account.
But an unexpected forwarding configuration on a compromised account deserves investigation.
environment=Can associate environment variables with the key when permitted by the SSH server configuration.
Again, context matters.
A quick way to identify key lines containing options is:
grep -nE \
'command=|from=|restrict|no-pty|no-port-forwarding|no-agent-forwarding|no-X11-forwarding|permitopen=|permitlisten=|environment=' \
/opt/zimbra/.ssh/authorized_keys
The -n option displays line numbers.
Do not paste this output into public forums because the command prints the complete matching key lines.
For incident documentation, redact the key material.
For example:
Line 3:
restrict,command="<REDACTED_COMMAND>" ssh-ed25519 <REDACTED_KEY> <COMMENT>
This provides enough information for your report without exposing the actual authorization material.
SSH key fingerprints provide a convenient way to identify keys without repeatedly handling the full public-key text.
OpenSSH’s ssh-keygen -l option displays the fingerprint of a public key.
For the entire authorized-keys file, you can try:
ssh-keygen -E sha256 -lf \
/opt/zimbra/.ssh/authorized_keys
The result may resemble:
256 SHA256:<FINGERPRINT_1> <COMMENT> (ED25519)
3072 SHA256:<FINGERPRINT_2> <COMMENT> (RSA)
The fingerprint is much easier to compare than a long public-key string.
You can use fingerprints to answer questions such as:
Does this key belong to one of our administrators?
Does it match a key in our configuration-management repository?
Was this key deployed to another server?
Does an SSH authentication log identify the same fingerprint?
Is this key present on multiple compromised hosts?
Do not rely on the key’s comment.
A comment such as:
admin@laptop
is simply text attached to the public key. OpenSSH does not use the comment to authenticate the user.
An attacker could choose a convincing comment.
The fingerprint identifies the actual key.
For an incident investigation, create a simple inventory.
For example:
| Line | Key Type | Fingerprint | Comment | Known Owner | Status |
|---|---|---|---|---|---|
| 1 | ED25519 | SHA256:<REDACTED> | admin-key | Administrator A | Verified |
| 2 | RSA | SHA256:<REDACTED> | backup | Unknown | Investigate |
| 3 | ED25519 | SHA256:<REDACTED> | automation | Unknown | Investigate |
Do not include full keys in a public incident report.
Then ask the administrators individually whether they recognize each fingerprint.
Do not ask:
Is this comment yours?
Ask:
Does this SHA-256 SSH key fingerprint belong to you?
That is a much stronger verification method.
/var/log/auth.logUbuntu documents:
/var/log/auth.log
as containing authorization-related activity, including remote login activity.
During our incident, this log provided an important clue.
We found successful public-key authentication involving the zimbra account.
To search for successful SSH authentication:
grep -E 'sshd.*Accepted' /var/log/auth.log
To focus on the zimbra account:
grep -E 'sshd.*Accepted.*zimbra' \
/var/log/auth.log
For public-key authentication specifically:
grep -E 'Accepted publickey for zimbra' \
/var/log/auth.log
A sanitized record may resemble:
<DATE> <HOST> sshd[<PID>]: Accepted publickey for zimbra from <SOURCE_IP> port <SOURCE_PORT> ssh2
On systems that log the key fingerprint, it may look similar to:
<DATE> <HOST> sshd[<PID>]: Accepted publickey for zimbra from <SOURCE_IP> port <SOURCE_PORT> ssh2: ED25519 SHA256:<FINGERPRINT>
That gives you several useful fields:
timestamp
username
source IP
source port
authentication method
key type
key fingerprint, when logged
Consider:
from <SOURCE_IP> port 54321
That port is normally the client’s source port for that SSH connection.
It does not mean your SSH server was necessarily listening on port 54321.
The server may still have been listening on the normal SSH port or whatever port was configured for sshd.
Do not confuse the remote source port with the local SSH listening port.
The activity you need may no longer be in the current:
/var/log/auth.log
Check:
/var/log/auth.log.1
as well.
For example:
grep -hE 'Accepted publickey for zimbra' \
/var/log/auth.log \
/var/log/auth.log.1 \
2>/dev/null
Older logs may be compressed.
Search them with:
zgrep -hE 'Accepted publickey for zimbra' \
/var/log/auth.log.*.gz \
2>/dev/null
This can provide a much longer authentication history.
Successful authentication is only part of the picture.
Also inspect failed attempts:
grep -E \
'Failed password|Failed publickey|Invalid user|authentication failure' \
/var/log/auth.log
For rotated logs:
grep -hE \
'Failed password|Failed publickey|Invalid user|authentication failure' \
/var/log/auth.log \
/var/log/auth.log.1 \
2>/dev/null
and:
zgrep -hE \
'Failed password|Failed publickey|Invalid user|authentication failure' \
/var/log/auth.log.*.gz \
2>/dev/null
Ubuntu’s security documentation specifically notes that auth.log can be useful for determining whether unauthorized remote authentication may have occurred and shows failed SSH authentication records as examples.
Imagine you find:
Failed publickey for zimbra from <IP_A>
Failed publickey for zimbra from <IP_A>
Failed publickey for zimbra from <IP_A>
Accepted publickey for zimbra from <IP_A>
That pattern deserves investigation.
But it still does not automatically prove malicious access.
Perhaps an administrator initially offered the wrong key and then used the correct one.
Now consider:
Accepted publickey for zimbra from <UNKNOWN_IP>
using a key fingerprint that nobody on the administration team recognizes.
That is much stronger evidence of unauthorized access.
Context changes the meaning.
In our incident, an Accepted publickey record came from an internal IP address.
An internal source does not automatically make the authentication legitimate.
You need to determine what device owned that address at the time.
Ask:
Was this an administrator workstation?
Was it another server?
Was it a VPN address?
Was it a jump host?
Was DHCP involved?
Could the address have belonged to a different machine earlier?
Was that internal system itself compromised?
An IP address tells you where the connection appeared to originate from at the network layer.
It does not automatically identify the human responsible.
For quick analysis:
grep 'Accepted publickey for zimbra' /var/log/auth.log
You can extract the source address from typical OpenSSH records with:
grep 'Accepted publickey for zimbra' /var/log/auth.log |
sed -n 's/.* from \([^ ]*\) port .*/\1/p' |
sort |
uniq -c |
sort -nr
Example output:
12 <INTERNAL_IP_A>
3 <VPN_IP_B>
1 <UNKNOWN_IP_C>
This gives you a frequency count.
Do not treat the highest count as the most suspicious.
A legitimate automation system may authenticate hundreds of times.
A single unknown successful login may be more important.
Timestamps become much more useful when compared across evidence sources.
Suppose you observe:
10:14 SSH public-key authentication
10:16 suspicious file created
10:17 suspicious process started
10:18 cron modified
That sequence deserves close investigation.
Your actual evidence sources may include:
/var/log/auth.log
/var/log/syslog
filesystem timestamps
process information
Zimbra logs
cron entries
shell history
network-device logs
During our Zimbra incident, we had already discovered suspicious processes and cron persistence.
The SSH authentication records became another piece of the same timeline.
Instead of reviewing an entire large log file, narrow the search.
For example:
grep 'Sep 2 09:' /var/log/auth.log
Or search for SSH activity:
grep -E 'Sep 2 09:.*sshd' \
/var/log/auth.log
For publication or shared incident reports, replace the actual date and address with sanitized values unless they are necessary.
A report can instead say:
At <TIME_A>, the zimbra account successfully authenticated using a public key from <INTERNAL_IP>.
On systems using the systemd journal, SSH records may also be available through journalctl.
For example:
journalctl -u ssh
For a defined period:
journalctl -u ssh \
--since "YYYY-MM-DD HH:MM:SS" \
--until "YYYY-MM-DD HH:MM:SS"
Depending on the Ubuntu release and logging configuration, useful SSH information may exist in /var/log/auth.log, the journal, or both.
Our confirmed incident finding came from /var/log/auth.log.
The journalctl checks are additional recommendations.
If your SSH authentication logs include the key fingerprint, compare it directly with:
ssh-keygen -E sha256 -lf \
/opt/zimbra/.ssh/authorized_keys
Suppose your log contains:
Accepted publickey for zimbra from <SOURCE_IP> ... ED25519 SHA256:<ABC123>
and your authorized-key inventory contains:
256 SHA256:<ABC123> admin@example (ED25519)
You have now identified which key authenticated.
Then verify who actually owns that fingerprint.
Do not assume the comment identifies the owner correctly.
Another useful investigation step is determining how sshd is configured.
Instead of reading only:
/etc/ssh/sshd_config
use:
sshd -T
This displays the effective configuration.
OpenSSH documents -T as a configuration test mode that prints the effective server configuration.
Useful settings to extract include:
sshd -T | grep -E \
'authorizedkeysfile|pubkeyauthentication|passwordauthentication|permitrootlogin|loglevel|allowusers|allowgroups'
This can help answer:
Is public-key authentication enabled?
Which authorized_keys files are used?
Is password authentication enabled?
Can root log in?
Are SSH users restricted?
What logging level is configured?
Do not modify the SSH configuration during evidence collection unless containment requires it.
Modern Ubuntu configurations may also use:
/etc/ssh/sshd_config.d/
Inspect the files:
ls -lah /etc/ssh/sshd_config.d/
Search SSH authorization-related configuration:
grep -RniE \
'AuthorizedKeysFile|AuthorizedKeysCommand|PubkeyAuthentication|PasswordAuthentication|PermitRootLogin|AllowUsers|AllowGroups|ForceCommand' \
/etc/ssh/sshd_config \
/etc/ssh/sshd_config.d \
2>/dev/null
An unexpected:
AuthorizedKeysCommand
or:
ForceCommand
could materially change how SSH authentication behaves.
This is an additional recommended check. We did not establish such a configuration change as part of the original Zimbra incident.
Once one account contains suspicious authorization data, inspect other relevant accounts.
A targeted search can locate authorized_keys files:
find /root /home /opt/zimbra \
-type f \
-name authorized_keys \
-ls 2>/dev/null
Then inspect each result.
For broader investigations, be careful with filesystem scope and mounted storage.
Zimbra’s current compromise-check guidance specifically recommends validating SSH configuration, determining which authorized-keys file is in use, and checking the contents of authorized-key files for all users.
Once you have confirmed that a key is unauthorized, disabling it may be appropriate.
But preserve the original file first.
Rather than immediately deleting evidence, you can make a preserved copy and then edit:
cp -a /opt/zimbra/.ssh/authorized_keys \
/root/incident-evidence/authorized_keys.before-removal
Then:
nano /opt/zimbra/.ssh/authorized_keys
or:
vi /opt/zimbra/.ssh/authorized_keys
Remove only the confirmed unauthorized entry.
Afterward:
ssh-keygen -E sha256 -lf \
/opt/zimbra/.ssh/authorized_keys
to inventory the remaining keys.
If the account should not accept interactive SSH access at all, the correct long-term configuration may involve stronger account or SSH restrictions. That should be planned carefully so legitimate Zimbra operations are not broken.
SSH configuration changes can disconnect your administrative path to the server.
Before changing:
sshd_config
authorized_keys
firewall rules
AllowUsers
AllowGroups
PubkeyAuthentication
PasswordAuthentication
make sure you have a tested recovery path.
If you modify SSH server configuration, validate it before restarting:
sshd -t
OpenSSH documents -t as a configuration and key sanity check.
If that returns no error, you can then plan the service reload or restart according to your environment.
Do not blindly restart SSH on a remote production server after editing its configuration.
Accepted Publickey Prove Unauthorized Access?No.
This is one of the most important points in the investigation.
Consider:
Accepted publickey for zimbra from <INTERNAL_IP>
This proves that OpenSSH recorded a successful public-key authentication for that account.
It does not, by itself, tell you:
whether the key was authorized by your organization
who physically used the corresponding private key
whether the originating computer was compromised
whether SSH was the original intrusion vector
whether the session performed malicious actions
You need additional evidence.
A stronger case emerges when several facts align.
For example:
The key is unknown to administrators.
The key appeared during the incident period.
The source IP is unexplained.
The same account owns suspicious files.
Suspicious processes run under the account.
Persistence was installed under the account.
Authentication occurred shortly before those changes.
That combination is much more meaningful than a single log entry.
Our incident included public-key authentication from an internal address.
It is tempting to conclude:
It came from inside the network, so it must have been legitimate.
That would be unsafe.
Possible explanations include:
The source address must be mapped to the system that owned it at the incident time.
Depending on your environment, useful sources may include:
DHCP logs
VPN logs
firewall logs
switch records
endpoint inventory
asset-management records
administrator records
That network investigation was beyond the specific SSH findings described here, but it is an important next step.
Another useful check is:
last -ai
This can provide historical login information from the system’s login records.
You can also focus on one account:
last -ai zimbra
Treat this as supporting evidence.
Not every type of SSH command execution produces the same interactive-login artifacts. A forced command, non-interactive session, automation task, or tampered accounting database may make login-history evidence incomplete.
Do not conclude that SSH access did not occur simply because last does not show a matching interactive session.
The SSH authentication logs are more important for determining whether authentication succeeded.
You can search authentication logs for the SSH daemon’s process ID.
Suppose the accepted login contains:
sshd[28471]
Search:
grep 'sshd\[28471\]' /var/log/auth.log
Related records may show additional activity associated with the connection.
This technique depends on the OpenSSH version and logging configuration, but it can sometimes help connect authentication to session creation and termination.
Ubuntu documentation notes that OpenSSH normally logs through the authentication logging facility and that increasing the SSH LogLevel to VERBOSE records additional authentication information.
Check the current effective setting:
sshd -T | grep loglevel
If your security policy requires more SSH authentication visibility, consider:
LogLevel VERBOSE
in the SSH server configuration.
Do not make this change in the middle of evidence collection without documenting it.
Changing logging during an incident modifies the system and may complicate timeline analysis.
This is a future hardening recommendation, not a confirmed configuration from our incident.
The following findings came directly from our Zimbra investigation:
We inspected /opt/zimbra/.ssh/authorized_keys.
The file contained multiple SSH key entries requiring investigation.
At least one entry included a forced command.
We reviewed /var/log/auth.log.
We found successful public-key authentication involving the zimbra account.
The source was an internal IP address.
The SSH activity occurred during the broader incident period.
The same zimbra account was already associated with suspicious processes, files, and cron persistence.
Those are the confirmed findings relevant to this article.
We did not establish from the SSH evidence alone:
that SSH was definitely the original compromise vector
that the internal source IP belonged to the attacker
that every unfamiliar key was malicious
that the forced-command syntax itself was malicious
that a particular person controlled the private key
that the successful login directly created every suspicious file
These distinctions are important when writing an incident report.
A good report separates:
observations
evidence
reasonable conclusions
unverified hypotheses
If I were repeating the investigation, I would perform and document the following checks systematically.
cp -a /opt/zimbra/.ssh/authorized_keys \
/root/incident-evidence/
stat /opt/zimbra/.ssh
stat /opt/zimbra/.ssh/authorized_keys
ssh-keygen -E sha256 -lf \
/opt/zimbra/.ssh/authorized_keys
sshd -T | grep authorizedkeysfile
grep -E 'sshd.*Accepted' \
/var/log/auth.log
grep -E \
'Failed password|Failed publickey|Invalid user|authentication failure' \
/var/log/auth.log
grep -hE 'sshd.*Accepted' \
/var/log/auth.log \
/var/log/auth.log.1 \
2>/dev/null
zgrep -hE 'sshd.*Accepted' \
/var/log/auth.log.*.gz \
2>/dev/null
sshd -T | grep -E \
'authorizedkeysfile|pubkeyauthentication|passwordauthentication|permitrootlogin|loglevel'
find /root /home /opt/zimbra \
-type f \
-name authorized_keys \
-ls 2>/dev/null
Compare SSH events against:
cron modifications
file timestamps
suspicious process start times
Zimbra logs
system logs
firewall logs
VPN logs
These are recommended investigation steps. They should not be interpreted as additional malicious findings from our original Zimbra incident.
If you discover questionable SSH access on a Linux server, a useful workflow is:
1. Preserve the authorized_keys file.
2. Record its ownership, permissions, and timestamps.
3. Generate SHA-256 fingerprints for every key.
4. Ask administrators to verify fingerprints, not comments.
5. Identify forced commands and unusual key restrictions.
6. Verify which AuthorizedKeysFile sshd actually uses.
7. Search auth.log for successful SSH authentication.
8. Search for failed attempts.
9. Review rotated logs.
10. Identify source IP addresses.
11. Determine which device owned each source address at that time.
12. Correlate authentication timestamps with suspicious filesystem and process activity.
13. Disable only keys confirmed to be unauthorized.
14. Rotate affected SSH keys and credentials when necessary.
15. Continue investigating the account for other persistence mechanisms.
Do not treat this as a checklist that automatically proves compromise.
Each result needs interpretation.
The suspicious SSH evidence did not exist in isolation.
By the time we investigated /opt/zimbra/.ssh/authorized_keys, we had already found suspicious activity associated with the zimbra account.
That included:
javab
idle
.khp
/home/SSL
cron persistence
Then we found questionable SSH authorization data.
Then we found successful public-key authentication in:
/var/log/auth.log
That made SSH account access an important part of the incident.
But we still avoided making the unsupported conclusion that:
The attacker definitely entered through SSH.
The evidence did not prove that.
The server may have been compromised through another mechanism first, followed by the installation of an SSH key for persistence.
Or an existing SSH key may have been used.
Or some of the authentication could have been legitimate.
A security investigation should distinguish what the evidence shows from what we think may have happened.
Zimbra’s security documentation explicitly recommends checking SSH when investigating a compromised Zimbra system.
Its current compromise-check guidance includes:
Zimbra also recommends replacing SSH keys when there is reason to believe files or authentication material may have been accessed.
That aligns closely with what we learned during our investigation.
Several lessons from this SSH investigation apply to Linux servers in general.
Do not trust SSH key comments.
A comment is descriptive text. Verify the fingerprint.
A forced command is not automatically malicious.
OpenSSH supports command="..." for legitimate restricted-access use cases. Investigate whether the command and key are authorized.
An internal source IP does not automatically mean legitimate access.
Identify the actual device that owned the address during the incident.
Successful authentication does not prove the original attack vector.
It proves authentication occurred.
Check rotated logs.
The event you need may no longer exist in the current auth.log.
Correlate timestamps.
SSH authentication becomes much more meaningful when compared with suspicious process starts, file creation, cron modifications, and other activity.
Fingerprint keys.
A SHA-256 fingerprint is a better identifier than a comment or filename.
Preserve evidence before deleting keys.
Once the authorization file is changed, part of the original system state is gone.
Investigate application accounts too.
Service accounts such as zimbra can be valuable targets because they may have access to application data, configuration, and operational processes.
Do not stop after removing the unauthorized key.
If an attacker had enough access to install an SSH key, determine what other changes may have been made.
sshd(8) ManualThe official OpenSSH documentation for SSH authentication and the authorized_keys file format, including command=, from=, restrict, forwarding controls, and other key options.
sshd_config(5) ManualDocuments AuthorizedKeysFile, AuthorizedKeysCommand, authentication configuration, and other SSH server settings.
ssh-keygen(1) ManualDocuments ssh-keygen -l for displaying public-key fingerprints and related key-management functions.
Ubuntu documentation describing /var/log/auth.log and its use for authorization activity and remote logins.
Ubuntu Viewing and Monitoring Log Files
Ubuntu security guidance covering the use of auth.log when investigating unauthorized access and SSH authentication attempts.
Ubuntu Basic Security investigation guide
Zimbra’s incident investigation guidance covering suspicious processes, SSH access, modified cron jobs, compromised accounts, and replacement of SSH keys when compromise is suspected.
Zimbra Investigating and Securing Systems
Zimbra’s compromise checklist specifically recommends checking sshd configuration, determining the authorized-keys location, and validating authorized-key contents for users.