SSH Key Management: Security Best Practices

TL;DR

Using a separate SSH key per host (and user) is significantly more secure than using one SSH key for all hosts. While the single-key approach is easier to manage, it creates a much larger attack surface. This guide explains why and how to implement the best practice.

Why Separate Keys are Better

Think of your SSH keys like passwords. You wouldn’t use the same password for everything, right? If one service is compromised, all your accounts using that password are at risk. The same principle applies to SSH keys.

• Reduced Blast Radius: If a key is stolen or compromised, only access to one host is affected.
• Improved Auditing: It’s easier to track which users have access to specific servers.
• Key Rotation: Easier to rotate keys for individual hosts without disrupting other connections.
• Compliance: Many security standards require strict key management practices.

Step-by-Step Guide: Separate SSH Keys

1. Generate a New Key Pair for Each Host/User Combination: Use the ssh-keygen command.

   ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_hostname -C "user@hostname"

   Replace hostname with the actual hostname and adjust the path if needed. The `-t rsa` specifies RSA, `-b 4096` sets key length to 4096 bits (good security), and `-f` defines the filename.

2. Copy the Public Key to the Host: Use ssh-copy-id. This is the easiest method if you have password access initially.

   ssh-copy-id -i ~/.ssh/id_rsa_hostname user@hostname

   You’ll be prompted for the user’s password on the remote host.

3. Test the Connection: Try connecting using the new key.

   ssh -i ~/.ssh/id_rsa_hostname user@hostname

   If it connects without asking for a password, you’ve successfully added the key.

4. Disable Password Authentication (Highly Recommended): Once keys are set up, disable password authentication on the host to prevent brute-force attacks.
   • Edit /etc/ssh/sshd_config as root.
   • Find and change these lines:

     PasswordAuthentication no

     ChallengeResponseAuthentication no

   • Restart the SSH service:

     sudo systemctl restart sshd

5. Repeat for Each Host/User Combination: Follow steps 1-4 for every server and user you need to access.

Managing Multiple Keys

Using many keys can seem daunting, but SSH provides tools to make it manageable.

• SSH Config File (~/.ssh/config): This file lets you define settings for different hosts.

  Host hostname

    User user

    IdentityFile ~/.ssh/id_rsa_hostname

  This way, you can simply type ssh hostname to connect using the correct key.

• SSH Agent (ssh-agent): The agent stores your decrypted private keys in memory so you don’t have to enter passphrases repeatedly.
  • Start the agent:

    eval "$(ssh-agent -s)"

  • Add your key(s) to the agent:

    ssh-add ~/.ssh/id_rsa_hostname

Risks of Using a Single SSH Key

• Compromised Key = Total Compromise: If your single key is stolen, an attacker gains access to all servers it’s authorized for.
• Difficult Revocation: Revoking access requires changing the key on every server.
• Limited Auditing: It’s hard to tell which host a connection originated from.