Wednesday, May 12, 2010

SSH-2 bites

We all have to deal with constant massive attacks on our servers, so we have to zip them up very tight and communicate with them using the secure shell protocol SSH-2. The first set of problem arises when we have to propagate our private keys in our zoo of machines. Submit this problem to Google and you will be served a gazillion war stories. The fundamental reason is that SSH-2 does not specify a file format and each utility has its own incompatible format; on top of this, add the different end-of-line conventions used by the various operating systems.

I have done this so many times that I thought I can do this in my sleep, but today I got bitten very badly because a GUI hid from me an error message that was revealed only when I finally used the terminal. As a pro memoria, let me recapitulate the procedure.

Password authentication is a no-no in this wild world and any server will always use a key and an IP address to authenticate.

When your machine park is a zoo, it is usually best to start on a Windows PC. Install the free PuTTY client and run the PuTTY Key Generator. After you generate your key, the top window contains the public key you send to you system administrator for adding to the authorized_keys file. Usually you must also send a list of the IP addresses from which you will log in.

Towards the bottom of the window you will see the button to save the private key. This is all you need on Windows. For your Linux and Unix PCs you will need a non-PuTTY formatted file for you key. On the top of the PuTTY Key Generator window you see a menu called Conversions, where you can save your private key in a different format. Do not worry about the public key, you can extract it on each client.

Usually on Linux and Unix PCs the SSH-2 implementation we use is OpenSSH. There is a multitude of SSH clients, but usually they are just a clever GUI on top of OpenSSH. You can use FTP to distribute your key in binary mode to all your computers.

On the -nix machines the keys go in the ~/.ssh directory. On a Mac, since this directory is hidden, open a Terminal window and type

  • cd .ssh
  • open .

This gives you a file browser in case you are afraid of the Terminal. Your SSH-related files go in this directory. You can do all your editing with TextEdit, making sure you use it in plain text mode.

In the .ssh directory you have a file called known_hosts, which has the keys of the servers you are using. Each time you start hand-shaking a session, you get the server's key. The first time, there will be no key for your server, but OpenSSH will ask you to confirm you are indeed talking to the legitimate server as you intended, and will update known_hosts.

In subsequent sessions, you will only be able to communicate with your server if it authenticates with the same key that is in known_hosts. If this is not the case, because you will usually request strict checking, you will get an error and the login attempt will fail.

My problem today was that the GUI I was using did not display the long message about an incorrect fingerprint, but just told me there is an error and closed the window. We had changed the server's key, but I still had the old key in my known_hosts file and had forgotten about it. When I tried to connect using the command line login, I got the full story and could simply delete the old key, but it took me a while to realize this was the problem and not the transfer or format of my key.

The lesson is that the first time you log into your server you should do it from your Terminal:

  • ssh -2 -i private_key

To make the login process easy, there is a number of agents you can execute in your .login file, so you type the key passwords on once when you log in. In Windows, Pageant is a popular agent, which is part of PuTTY. On the Mac, SSH Agent is popular.

When you transfer many files in a complex directory structure, you do not want to bother with using the Terminal, but want to use a GUI instead. On the Mac, MacSFTP used to be a popular commercial product, but it has not been updated for several years and is buggy (stalls, load errors, Finder crashes). I use Fetch, from Dartmouth College, which I have been using since 1989.

The reason I use it is that the transfers are very fast, which is important when you have to transfer gigabytes of data. Unfortunately, Fetch does not tell you how to set it up for SSH and there is no interface to specify your key. You either have to use an agent, or in the ~/.ssh directory you have to create a file called config, containing these two lines:

  • host
  • IdentityFile ~/.ssh/private_key

Hopefully some day we will again get an SSH protocol with a file format specification and the GUI people will hide the dirty set-up process for us. The challenge is to implement it in a way that it prevents attacks from the men in the middle.