SSH Key Management: Security Best Practices

G5 Cyber Security

2 weeks ago

TL;DR

Using a separate SSH key per host (and user) is significantly more secure than using one SSH key for all hosts. While the single-key approach is easier to manage, it creates a much larger attack surface. This guide explains why and how to implement the best practice.

Why Separate Keys are Better

Think of your SSH keys like passwords. You wouldn’t use the same password for everything, right? If one service is compromised, all your accounts using that password are at risk. The same principle applies to SSH keys.

Reduced Blast Radius: If a key is stolen or compromised, only access to one host is affected.
Improved Auditing: It’s easier to track which users have access to specific servers.
Key Rotation: Easier to rotate keys for individual hosts without disrupting other connections.
Compliance: Many security standards require strict key management practices.

Step-by-Step Guide: Separate SSH Keys

Generate a New Key Pair for Each Host/User Combination: Use the ssh-keygen command.
```
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_hostname -C "user@hostname"
```
Replace hostname with the actual hostname and adjust the path if needed. The `-t rsa` specifies RSA, `-b 4096` sets key length to 4096 bits (good security), and `-f` defines the filename.
Copy the Public Key to the Host: Use ssh-copy-id. This is the easiest method if you have password access initially.
```
ssh-copy-id -i ~/.ssh/id_rsa_hostname user@hostname
```
You’ll be prompted for the user’s password on the remote host.
Test the Connection: Try connecting using the new key.
```
ssh -i ~/.ssh/id_rsa_hostname user@hostname
```
If it connects without asking for a password, you’ve successfully added the key.
Disable Password Authentication (Highly Recommended): Once keys are set up, disable password authentication on the host to prevent brute-force attacks.
- Edit /etc/ssh/sshd_config as root.
- Find and change these lines:
```
PasswordAuthentication no
```
```
ChallengeResponseAuthentication no
```
- Restart the SSH service:
```
sudo systemctl restart sshd
```
Repeat for Each Host/User Combination: Follow steps 1-4 for every server and user you need to access.

Managing Multiple Keys

Using many keys can seem daunting, but SSH provides tools to make it manageable.

SSH Config File (~/.ssh/config): This file lets you define settings for different hosts.
```
Host hostname
```
```
  User user
```
```
  IdentityFile ~/.ssh/id_rsa_hostname
```
This way, you can simply type ssh hostname to connect using the correct key.
SSH Agent (ssh-agent): The agent stores your decrypted private keys in memory so you don’t have to enter passphrases repeatedly.
- Start the agent:
```
eval "$(ssh-agent -s)"
```
- Add your key(s) to the agent:
```
ssh-add ~/.ssh/id_rsa_hostname
```

Risks of Using a Single SSH Key

Compromised Key = Total Compromise: If your single key is stolen, an attacker gains access to all servers it’s authorized for.
Difficult Revocation: Revoking access requires changing the key on every server.
Limited Auditing: It’s hard to tell which host a connection originated from.