How to set up SSH keys: Effective key management for secure remote connections

To access your server from a local computer with Secure Shell Protocol (SSH), you must learn how to set up SSH keys.

An SSH key pair eliminates manual password authentication, creating a faster server login process and enhanced remote server access security.

Whether you’re new to SSH key-based authentication or want to improve your key management, this comprehensive guide will explain the process and provide effective strategies.

Download ultimate SSH commands cheat sheet

Prerequisites for setting up SSH keys

Before setting up SSH keys, let’s review the requirements first. You will need access to:

• Your local device.
• Your remote computer or server and its username and IP address.
• The Terminal application to establish your remote connection.

For the last prerequisite, Mac and Linux users can use their respective built-in Terminals.

However, Windows users can’t use Terminal for SSH connection. Fortunately, there are local SSH clients for Windows. These are the top four recommendations:

• PowerShell – a built-in command-line shell and scripting language by Microsoft. It’s recommended for advanced Windows users.
• Windows Subsystem for Linux (WSL) – a program that allows Windows users to run a Linux environment on their computer. Best for beginners and users familiar with the Linux operating system.
• Git BASH – a command-line interface bundled with Git, making it ideal for those frequently working with Git repositories.
• PuTTY – a free and open-source Terminal emulator with a key generator and key manager. It’s a viable option for those using older Windows operating systems. Check out our generating PuTTY SSH Keys tutorial for more information.

With these prerequisites in place, you are ready to set up your SSH key authentication.

How to set up SSH keys

With your local device, remote server, and Terminal prepared, let’s learn how to set up SSH keys step by step.

1. Initiate SSH key generation process

To establish secure connections using SSH, you need to generate a pair of public and private keys. These work together to authenticate and encrypt your communication with remote servers.

The remote server uses public keys to verify your identity based on their SSH key fingerprints. Meanwhile, the local computer stores the private keys for authenticating your SSH connection.

When you connect to a remote computer, your local machine provides the private key. Then, the server checks it against the associated public key to grant you access.

There are two commonly used algorithms for generating authentication keys:

• RSA – an SSH RSA key is considered highly secure as it has typically larger key sizes, often 2048 or 4096 bits. It is also more compatible with older operating systems.
• Ed25519 – a more modern algorithm with a smaller standard key size of 256 bits. It is just as secure and efficient as an RSA key due to its strong cryptographic properties. The compatibility is lower, but newer operating systems support it.

This tutorial will generate private and public SSH keys using Ed25519 as an example. However, you can also follow this guide if you use RSA.

First, open your Terminal or local SSH client. Then, enter the ssh-keygen command:

ssh-keygen -t ed25519

The SSH command will display the following message:

Generating public/private ed25519 key pair.

Move on to the next step.

2. Set up the key storage and passphrase

The key generator should ask you to choose a location to store the keys:

Enter file in which to save the key (/home/local_username/.ssh/id_ed25519):

Press Enter to save them in the default location, which is the /.ssh directory in /home.

Alternatively, specify a different location by typing its path and file name. For example, you may want to save your key files in a separate directory from the /.ssh folder for better organization or an encrypted external drive for enhanced security.

Next, the key generator window will ask you to create an SSH key passphrase to access your private key.

Enter passphrase (empty for no passphrase):

You can press Enter to skip this step, but it is strongly encouraged to create a private key passphrase to enhance server connection security. If someone gains unauthorized access to your private key, they still need to know the passphrase to use it.

Choose a strong, long, and unique passphrase. Combining uppercase and lowercase letters, numbers, and special characters is ideal. Here are some examples:

• Random sentence – PurpleElephantsDanceOnFridayNights!
• Combination of words and numbers – TrickyGiraffe37$Sunshine
• Passphrase with special characters – C0ffee#Mug@3xtraShot
• Song lyrics or book quotes – ToBeOrNot2Bee?That@isTheQuestion
• Mixing multiple languages – Bonjour!Grazie!Danke!¡Hola!

Once you have created the passphrase, hit Enter. Then, fill out the same key passphrase and press Enter again.

The ssh-keygen command will now display the SSH key fingerprint and randomart image for the public key. Here’s what the output should look like from start to finish:

ssh-keygen -t ed25519

Generating public/private ed25519 key pair.

Enter file in which to save the key (/home/local_username/.ssh/id_ed25519):

Enter passphrase (empty for no passphrase):

Enter the same passphrase again:

Your identification has been saved in /home/local_username/.ssh/id_ed25519

Your public key has been saved in /home/local_username/.ssh/id_ed25519

The key fingerprint is:

SHA256:kbMwyYwHymTudFmkCxt8yX6+2ztczkUJwRWTwXdRzcU localusername@hostname

The key’s randomart image is:

+--[ED25519 256]--+

|      ..o==o    |

|    . +*+o*+    |

|   . +o.*.o+    |

|    .o. = o     |

|    . o S +     |

|     . o =      |

|      o o       |

|       + .      |

|      . o       |

+-----------------+

The ssh-keygen command should also have created a public key and a private key in the specified path and file name. The public key file should have a PUB extension, like id_rsa.pub or id_ed25519.pub.

Private RSA keys typically end with RSA in their file name. Meanwhile, the file for Ed25519 often does not have a specific extension.

Managing SSH keys

Now that you know how to set up SSH keys, let’s dive into effective key management. This makes sure your remote connections are secure and efficient.

Transferring SSH public key to a remote server

To use your SSH key pair for remote connection, you need to transfer your public key to an SSH server first.

If you’re a Hostinger VPS user, our hPanel provides a one-click button to copy your public key to your server’s SSH.

Simply follow these steps:

1. Go to the SSH key directory, where your public key is stored. Select and open the id_ed25519.pub file using a text editor. Keep this window open.
2. Log into your account and navigate to the VPS menu. Pick a server.
3. Then, choose Settings → SSH keys. Click Add SSH key.

4. A pop-up will appear asking you to give a Name for where the SSH key is saved. In the SSH public key box, copy and paste the content of the id_ed25519.pub file there.

5. Once done, click Add SSH key. If the SSH key authentication is successful, you’ll see the keys displayed like so:

Hostinger Premium and Business clients can also establish secure connections with SSH for their websites. Simply find the SSH settings page on your Dashboard → Advanced → SSH Access. Then, click Enable on the SSH status section.

From there, follow steps three to five from the previous VPS explanation to connect your local device to the server’s SSH service.

If you encounter any issues, performing SSH key troubleshooting is essential. Check for potential errors in key generation. Additionally, verify that the public key is properly configured on the remote server.

For assistance, go to our knowledge base or contact our 24/7 Customer Success team. Our VPS also includes Kodee – a built-in AI assistant that can help solve your issues.

Another way to transfer your public key is with your Terminal or SSH client using the ssh-copy-id command. Replace ‘username’ with your server’s username and ‘remote_server’ with its IP address or domain name:

ssh-copy-id username@remote_server

The SSH command will display this output and ask you for the remote server’s password login:

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: 

"/home/localusername/.ssh/id_ed25519.pub"

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

username@remote_server's password:

If the password authentication is successful, you should see this result:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'username@remote_server'"

and check to make sure that only the key(s) you wanted were added.

Test out your new SSH key user authentication using the suggested command. If it works, you will see a welcome message, which will differ by your server and operating system. Here’s an example of what a successful login may look like:

Managing multiple SSH keys

As your projects grow and diversify, you may utilize more than one SSH key for different servers, user accounts, or projects.

If this occurs, we recommend using SSH Agent. This feature runs in the background and securely stores your private keys. It lets you manage SSH key pairs across multiple connections and eliminates the need to repeatedly enter private key passphrases during a session.

To use an SSH agent, follow these steps:

1. Launch your Terminal or SSH client. Then, enter the following command to start the SSH agent:

eval "$(ssh-agent -s)"

2. Use the ssh-add command to add your private key to the SSH agent. Replace ‘private_key’ with your file name.

ssh-add ‘private_key’

If your private key is protected with a passphrase, it will prompt you to enter it.

3. Run the following command to confirm that your private key has been added. It will list the identities currently held by the agent.

ssh-add -l

With the SSH agent running and your private key added, you can connect to remote servers without entering your passphrase for each session. Simply use the ssh username@remote_server command to log in.

Here are other strategies for multiple SSH keys management:

• Follow best practices to organize your key pairs – they include using clear naming conventions, applying logical directory structures, and maintaining proper permissions. This way, each SSH key pair is easily identifiable.
• Have a unique SSH private key and public key pair for each server or account – while you can use the same keys for different servers, it is unsafe. If the private key gets lost, it will compromise the security of all its associated devices.
• Run the hostname command to display the hostname of the current system – the SSH command hostname can identify the specific host of an SSH key. It helps ensure you select the appropriate key for the chosen host.
• Use key aliases – an alias can help distinguish each key’s purpose or associated project. During the key generation process, add a comment with the -C flag to specify the alias – such as ssh-keygen -C ‘alias’.
• Adjust SSH settings for specific hosts – the SSH configuration file lets you define which SSH keys to use for certain hosts or domains. This minimizes the risk of using the wrong key during the login process. If you store the keys in the default path, you can find this file in the /.ssh directory.

Dealing with lost or compromised SSH keys

If a private SSH key gets lost or compromised, take immediate action to mitigate potential security risks.

To prevent unauthorized access, remove SSH keys from the remote servers where they were installed. Follow these steps:

1. Open your Terminal or SSH client and use the cd command to navigate to your keys’ directory. Here’s what it would look like if you use the default /.ssh path:

cd ~/.ssh

2. List the contents of the SSH directory using the ls command. You will see the file names displayed like so:

ls

id_ed25519 id_ed25519.pub known_hosts

3. Delete the private key file by running the rm command, replacing ‘private_key_file’ with the actual file name:

rm ‘private_key_file’

4. Remove the corresponding public key file by typing the same command on your SSH client window.

rm ‘public_key_file’

Hostinger clients should delete the public key on hPanel as well. Go to SSH keys for VPS users or SSH Access if you’re a web hosting customer. Then, press Delete on the SSH key.

5. Use the ls command to confirm that the files have been successfully deleted from the SSH directory. If successful, the private and public key files should no longer exist.

Besides deleting compromised keys, using the ssh-keygen command to regenerate a new key pair is highly recommended. Doing this ensures that any copies of the compromised keys will be useless.

To minimize the risk of loss or compromise, follow these SSH key management tips:

• Disable password authentication – Using private-public key authentication as your main server login ensures that only authorized users can access the server via SSH.
• Use secure key storage – Restrict access to your private keys by storing them in a protected area of your file system and setting proper permissions using the chmod command.
• Update your private and public keys regularly – This will reduce the likelihood of compromising your keys. Run the ssh-keygen command regularly to create new keys and replace old ones.
• Limit the key pair usage – Use one key pair for one purpose only, such as a specific project or user account. This makes your keys easier to manage and isolates potential security risks.

Conclusion

Learning to set up SSH keys is essential for website owners looking to establish remote server access. Let’s summarize the key steps:

1. Initiate SSH key generation – open your Terminal or SSH service to generate a pair of public and private keys using Ed25519 or RSA algorithms.
2. Set up the key storage and passphrase – choose the location to save the keys, either the default location or a specific directory. Create a strong passphrase to protect the private key from unauthorized use and exposure.

Afterward, transfer the public key to the remote server through your hosting control panel or using the ssh-copy-id command.

To effectively manage your keys, use an SSH agent to store the private keys securely. Also, consider other tips like following the best file management practices, creating unique key pairs for specific SSH servers, and utilizing key aliases.

If a private key gets lost or compromised, don’t panic. Simply remove it from your local device and run the ssh-keygen command again to generate a new key pair.

How to set up SSH keys FAQ

Here are some commonly asked questions about setting up SSH keys and their answers: