How To Set Up Multiple SSH Keys on Your Computer (2024)

Hello! I was inspired to write this article as I stumbled upon some issues when setting up multiple SSH keys.

I originally had it set up for my personal GitHub. A few days ago, I started to volunteer for one of the non-profit organizations that required another set of SSH key pairs. What I didn’t know is that I was not supposed to overwrite the old SSH key pairs with my new ones and expect them to work. I was working on one of my personal projects the other day and attempted to push to GitHub, but my terminal told me: “Permission denied (publickey).” After spending countless hours Googling, I was able to fix the problem and learned from my mistakes. I am going to walk you through this, so you can set up yours quickly and efficiently.

▹ Step 1: Double-check if you have an existing SSH key.
▹ Step 2a: If you don’t have an existing SSH key, generate a new SSH key
▹ Step 2b: Save the SSH key in the according file of your choice.
▹ Step 3: Add the SSH to the ssh-agent
▹ Step 4: Add your SSH private key to the ssh-agent
▹ Step 5 It depends: Add the SSH key to your GitHub account
▹ Step 6 Optional: To double-check if your git is connecting to your server

Step 1: Double-check if you have an existing SSH key.

★ Open Terminal (Git Bash for Windows users.)

★ Enter ls ~/.ssh (/.ssh/ is a hidden folder. You can access it via Terminal, or Mac: cmd + shift + . ; Windows: check Show Hidden Files)

Step 2a: If you don’t have an existing SSH key, generate a new SSH key

★ Enter either ssh-keygen -t ed25519 -C "your_email@example.com" (substitute the email with your associated email with GitHub) or simply ssh-keygen for other hosts.

★ You should see Generating public/private ed25519 key pair if it’s for GitHub, or Generating public/private rsa key pair.

PAUSE!!

Normally, you would simply press Enter(or Return for Mac users) to continue as the SSH key would store in the default file the prompt has assigned (id_rsa). However, since we are setting up multiple SSH keys, we would like to name the keys differently, so you can distinguish which one is which and also for later purposes.

Step 2b: Save the SSH key in the according file of your choice.

Unfortunately, you have to type the whole pathname to where you want to save it and make sure to name the keys differently.

All right, now press Enter! You should see Enter passphrase(empty for no passphrase):. It is totally up to you if you want to have a passphrase or not. Otherwise, you could press Enter and Enter when you see this: Enter same passphrase again: to skip the process of entering any passphrase.

You should see something like this 👆🏻

Step 3: Adding the SSH to the ssh-agent

Mac User: According to the GitHub documentation, if you are adding the SSH key to the agent, use the default macOS ssh-add command.

I assume that you got the config file within your ssh folder, since this is not your first time setting up the SSH key.

If you don’t or you simply want to double-check, you could:

★ Start the ssh-agent in the background eval "$(ssh-agent -s)"

Don’t worry, you and I would have a different Agent pid number.

★ Check to see if your ~/.ssh/config exists.

★ You could type ls within the ~/.ssh/

(I am sorry, Windows users. I don’t know much from this point, but according to GitHub, you can skip this part and go straight to setting up the private key.)

★ Type vim config. This will open the config file, no matter if you have that file or not. You can edit the config file using the Vim text editor.

★ Once you are on this page, press i so you can insert your ssh-agent. And hold Option (alt on Windows) on your keyboard and click wherever you want to copy and paste the following items.

For GitHub, you can copy and paste the following:

Host *
 AddKeysToAgent yes
 UseKeychain yes
 IdentityFile ~/.ssh/id_rsa_github2 <- change this to whatever you name your ssh-key, in my case, it would be id_rsa_github2

If you are using other sites, like AWS CodeCommit, you would need to find the according ssh-agent name:

Host git-codecommit.*.amazonaws.com 
 User APKAEIBAERJR2EXAMPLE <- You would find this when you upload the SSH key and it's under SSH Key ID
 IdentityFile ~/.ssh/codecommit_rsa <- change this to whatever you name your ssh-key

After vim config

★ Once you are done, press esc on your keyboard to leave the “insert” mode and type :wq and then Enter to exit the text editor.

Step 4: Add your SSH private key to the ssh-agent

ssh-add -K ~/.ssh/id_rsa_github2

Step 5 It depends: Add the SSH key to your GitHub account

Here are a detailed instructions and guidance from GitHub that you can follow.

Step 6 Optional: To double-check if your git is connecting to your server

(That’s what I went for after I set up my key.)

In the case of GitHub, type the following:

ssh -T GITHUB-USERNAME@github.com

If you got Permission denied (publickey) , go ahead and copy and paste:

ssh -T git@github.com
> Hi username! You've successfully authenticated...

If you still got Permission denied , you might want to check your SSH keys whether it’s set up properly in your host agent, your config, etc. More info here specifically for GitHub.

Step 7: Repeat the instructions if you have to (Back to Step 1)

If you have more SSH keys to set up, simply repeat the instructions above and you should be good to go 😉. I would also recommend double-checking with the instructions of the host agent that you are trying to connect with. This article is a general case if you are curious about how to set up multiple SSH keys.

NOTE: I would appreciate any comments if I have missed anything. I would love to include Windows users in this article. Let me know what else you did to make it work!