UNIX / Linux keyboard.

How to Set Up and Use SSH
Step 1 — Administrative Work

Configure the Server and Client

Configure the service by modifying /etc/ssh/ssh_config and /etc/ssh/sshd_config on all hosts as follows:

/etc/ssh/ssh_config is used by local users connecting from this host to some remote server. Outbound connections. Put the following in that configuration file: 

# Only support the more secure SSH2 protocol.
# Recent versions of OpenSSH default to SSHv2 only,
# but there's no harm in explicitly configuring this.
Protocol 2 

# Allow the user to tunnel X applications through SSH.
# Read "man ssh_config" first.  Yes, there is a security
# issue.  However, very carefully read its description.
# It is an additional problem if the attacker is able
# to subvert Unix file system security on one of the
# hosts.  But if the attacker is able to do that,
# I have far worse problems that must be fixed NOW.
ForwardX11 yes

/etc/ssh/sshd_config controls how this server handles connection requests from remote users. Inbound connections. Put the following in that configuration file: 

# Only support the more secure SSH2 protocol.
# Recent versions of OpenSSH default to SSHv2 only,
# but there's no harm in explicitly configuring this.
Protocol 2 

# If you list users allowed to connect via SSH, then
# they will be the only users allowed to authenticate
# to SSH.  THIS IS OPTIONAL, you do NOT need to include
# an AllowUsers line!  If you do, these will be the
# only users allowed to authenticate.  If you do not
# include this line, then ALL users other than root will
# be allowed to authenticate.  Also see "AllowGroups",
# "DenyUsers", and "DenyGroups" in "man sshd_config".
AllowUsers joe jane ....

# Only accept authentication via cryptographic
# challenge-response.  The remote client must have
# cryptographic keys.
PasswordAuthentication no

# Do not allow root login -- first login as an ordinary
# user, then su to root
PermitRootLogin no

Collect host public keys

You could collect host public keys with the following command on a small network. The parameters need to list every SSH server as both simple hostname and fully-qualified domain name, as well as their IP addresses, so this could quickly lead to a lot of typing: 

# ssh-keyscan  host1  host1.example.com  10.0.0.1  fc00::2e27:d7ff:fec5:d37b \
               host2  host2.example.com  10.0.0.2  fc00::213:3bff:fe12:6fa9 \
               [.... and so on ....] \
               hostN  hostN.example.com  10.0.0.N  fc00::ba27:ebff:feac:bf18  >  /etc/ssh/ssh_known_hosts

For a solution that scales well to a typical organization, first create a file, let's say /tmp/hostlist, containing all hostnames, all fully-qualified domain names, and all IP addresses, one item per line, for all of your SSH servers. You might be able to generate such a list with the following command pipeline, if your name server allows zone transfers from your workstation: 

$ dig yourdomain.com @nameserver.yourdomain.com AXFR | \
	awk '/\<A\>|\<AAAA\>/ {print $1; print $NF}' | \
	sed 's/\.$//' > /tmp/hostlist

That list will include every host, not just the SSH servers. However, ssh-keyscan will quickly give up on hosts that don't respond.

Then run the following command: 

# ssh-keyscan -f /tmp/hostlist > /etc/ssh/ssh_known_hosts

Once you have generated /etc/ssh/ssh_known_hosts, securely copy it to every host on which SSH will be used, even the workstations that will only be SSH clients.

You will generate new (and different!) SSH keys if you reinstall the operating system. That means that clients will refuse to connect to that freshly built SSH server, because it will look like it is a different host trying to masquerade. You will need to at least update that host's key. A simple yet safe procedure would be:

  1. On some host, edit /etc/ssh/ssh_known_hosts and remove the entry for the freshly rebuilt server.
  2. Re-run the above ssh-keyscan command to create an updated ssh_known_hosts file with the public key for the newly rebuilt server.
  3. Securely copy the new /etc/ssh/ssh_known_hosts file into place. Note — since we have disabled root login over SSH, we have also disabled root use of scp. You must copy the files to all servers as a user, and then su to root on each server to install them.
  4. At this point you may wish that you had thought to save a backup copy of those key files before starting... If you did, be careful about file permissions! The public keys are in files named *key.pub and must be mode 644 or 444. The private keys are in files named *key and must be mode 600 or 400.

Protect the user accounts

Now would be a good time to disable all unneeded accounts and make sure that all accounts still in use have strong passwords. Attackers are trying constantly to guess passwords.

The patterns of logins and passwords used in these guessing attacks can be interesting. It certainly would be a very bad idea for even an unprivileged user account to have a commonly used password.

Disable unused accounts

Lock the password, meaning replace the password hash in /etc/shadow with an invalid string so that there is no such thing as a valid password for that user. On Linux:
# passwd -l username

On other Unixes (BSD, Solaris) you may need to carefully edit /etc/shadow (Solaris) or /etc/master.passwd (BSD) and replace the password hash for the user with some other string. Simply inserting LOCKED as the initial part of the string will suffice. From:
cromwell:$2a$06$5Fkvg4EFsRCk....
to:
cromwell:LOCKED$2a$06$5Fkvg4EFsRCk....

On BSD you then need to run pwd_mkdb — don't omit this crucial step!

For defense in depth, also replace that user's login shell with something that exits immediately and is not a shell. On Linux:
# chsh -s /bin/false username
On Solaris:
# chsh -u username
and then specify /bin/false
On BSD:
# chpass -s /usr/bin/false username

Set file permissions, especially in NFS situations

At some point, SSH started refusing to work if the user's SSH key storage area was not securely configured. That's a good idea, although it tends not to explain why it's failing and can be frustrating to debug.

The directory ~/.ssh should be mode 700, and the files in there containing private keys (id_dsa, (id_ecdsa, (id_ed25519, id_rsa) should be mode 600. Also, all the files must not be group-writeable. Do something like this:
# chmod -R g-w /home/*/.ssh

The SSH daemon must be able to read those files, and since it is owned by root that normally is no problem. However, root is "squashed" to nobody by default on NFS mounts. So, make sure that NFS exports of user home directories are shared with the no_root_squash option.

Start an SSH agent at login time

Start an SSH agent at login time if your display manager does not already do this. Some, like Gnome's gdm display manager, tend to do this. Others, like KDE's kdm display manager, tend not to.

To test if this is needed, login to the graphical desktop as a user and run this command: 

$ ssh-add -l

If an SSH agent is running, you will see the following and you don't need to do anything additional to start an agent for the user: 

The agent has no identities.

However, if you see the following, you need to make a change to get an agent started for the users: 

Could not open a connection to your authentication agent.

If you need to make a change to get an SSH agent running at login time, modify your display manager configuration. Depending on the display manager, this may take some investigation and experimentation.

For example, on Linux with KDE's display manager, the configuration is stored in /usr/share/config/kdm/. On OpenBSD, it is /usr/local/share/config/kdm/. The file Xsession originally contains a block reading as follows: 

case $session in
  "")
    exec xmessage -center -buttons OK:0 -default OK "Sorry, $DESKTOP_SESSION is no valid session."
    ;;
  failsafe)
    exec xterm -geometry 80x24-0-0
    ;;
  custom)
    exec $HOME/.xsession
    ;;
  default)
    exec /usr/bin/startkde
    ;;
  *)
    eval exec "$session"
    ;;
esac

Change that code to the following, with the change highlighted: 

case $session in
  "")
    exec xmessage -center -buttons OK:0 -default OK "Sorry, $DESKTOP_SESSION is no valid session."
    ;;
  failsafe)
    exec xterm -geometry 80x24-0-0
    ;;
  custom)
    exec $HOME/.xsession
    ;;
  default)
    exec ssh-agent /usr/bin/startkde 
    ;;
  *)
    eval exec "$session"
    ;;
esac

If that doesn't work on OpenBSD, do this: 

# cd /usr/local/bin
# mv startkde4 startkde4-real
# cat > startkde4 << EOF
#!/bin/sh
/usr/bin/ssh-agent /usr/local/bin/startkde4-real
EOF
# chmod 755 startkde4

KDE may be changing its logic, at least in the Mageia distribution. I had to instead modify /etc/X11/wmsession.d/01KDE to refer to a simple shell script wrapper. For some reason I was unable to put the exec ssh-agent startkde line directly in that file, so instead I have: 

NAME=KDE
ICON=kde-wmsession.xpm
DESC=The K Desktop Environment
# EXEC = /usr/bin/startkde
EXEC = /usr/local/bin/start-kde
SCRIPT:
# exec /usr/bin/startkde
exec /usr/local/bin/start-kde

The script /usr/local/bin/start-kde looks like this: 

#!/bin/sh
/usr/bin/ssh-agent  /usr/bin/startkde

If you are stuck with CDE and its dtlogin interface (decent technology for about 1990), modernize. Seriously.

If you must live in the past, you need to make a change within the large and complicated file /usr/dt/bin/Xsession. Look for a block like the below, and insert the highlighted code. 

# 
# Session startup clients and args
# 
if [ "$SESSIONTYPE" = "altDt" ]; then
      dtstart_session[0]="/usr/bin/ssh-agent $SDT_ALT_SESSION"
      dtstart_hello[0]="$SDT_ALT_HELLO"
else
      dtstart_session[0]="/usr/bin/ssh-agent $DT_BINPATH/dtsession"
      dtstart_hello[0]="$DT_BINPATH/dthello &"
fi
dtstart_session[1]="$HOME/.xsession"
dtstart_session[2]="$HOME/.x11start"

Log out, log back in, and run this command: 

$ ssh-add -l

You should see a message that the SSH agent has no identities. That's fine, that's what we were looking for. If you instead see a message that the SSH agent cannot be contacted, then you got something wrong. Either you didn't really log all the way out and back in again, or (more likely, I would hope) you didn't get things set up correctly.

Install Keychain

Even with the above changes, the KDM display manager may still ignore what you want. It was able to get it right, once. Here is what we see on OpenBSD with KDE 3.5.10: 

% pstree | egrep -C 4 'startkde|ssh-agent'
 |-+= 16874 root /usr/local/bin/kdm
 | |-+= 07660 _x11 /usr/X11R6/bin/X vt05 :0 -auth /etc/X11/xdm/authdir/A:0-r7LN2x (Xorg)
 | | \--- 23894 root X: [priv] (Xorg)
 | \-+- 26766 root kdm: :0 (kdm)
 |   \-+= 17300 cromwell /bin/sh /usr/local/bin/startkde
 |     |--= 09760 cromwell /usr/bin/ssh-agent /usr/local/bin/startkde
 |     \--- 02140 cromwell kwrapper ksmserver
 |-+= 31281 cromwell kdeinit: kdeinit Running... (kdeinit)
 | |--- 09133 cromwell kdeinit: kdeinit: klauncher --new-startup (kdeinit)
 | |--- 05026 cromwell kdeinit: kdeinit: kwin -session 101dd1d820716f00013....

And on CentOS with KDE 3.5.4 and gdm 2.16.0: 

%  pstree -u | egrep -C 4 'startkde|ssh-agent'
     |-eggcups(cromwell)
     |-events/0
     |-gconfd-2(cromwell)
     |-gdm-binary---gdm-binary-+-Xorg
     |                         `-tcsh(cromwell)-+-ssh-agent
     |                                          `-startkde---kwrapper
     |-gdm-rh-security
     |-gpg-agent(cromwell)
     |-hald(haldaemon)---hald-runner(root)-+-hald-addon-acpi(haldaemon)
     |                                     |-hald-addon-keyb(haldaemon)

But on Mageia with kdm 4.10.2: 

%  pstree -u | egrep -C 4 'startkde|ssh-agent'
        |                    |-xterm---luit---tcsh-+-egrep
        |                    |                     `-pstree
        |                    `-xterm---luit---tcsh---su(root)---bash
	|-kdm-+-X
	|     `-kdm---startkde(cromwell)---kwrapper4
        |-kglobalaccel(cromwell)
        |-kmix(cromwell)---{kmix}
        |-knotify4(cromwell)---{knotify4}
	|-konsole(cromwell)-+-tcsh---vim---{vim}

The solution is to install the keychain package.

It works by creating files named your-hostname-csh and your-hostname-sh which can set environment variables specifying the PID and socket used by the long-running SSH agent. New shells or the KDE desktop are started plugged into those agents. It also handles the GnuPG agent, and, in case it's installed, the fish shell: 

% ls -la .keychain
total 76
 4 drwx------   2 cromwell cromwell  4096 Jun 14 15:02 ./
56 drwxr-xr-x 205 cromwell cromwell 57344 Jun 16 15:26 ../
 4 -rw-------   1 cromwell cromwell    82 Jun 14 15:02 your-hostname-csh
 0 -rw-------   1 cromwell cromwell     0 Jun 14 15:02 your-hostname-fish
 4 -rw-------   1 cromwell cromwell   112 Jun 14 15:02 your-hostname-sh
% more .keychain/*
::::::::::::::
.keychain/your-hostname-csh
::::::::::::::
setenv SSH_AUTH_SOCK /tmp/ssh-WeBckDhIvaF7/agent.11515;
setenv SSH_AGENT_PID 2269;
::::::::::::::
.keychain/your-hostname-fish
::::::::::::::
set -e SSH_AUTH_SOCK; and set -x -U SSH_AUTH_SOCK /tmp/ssh-WeBckDhIvaF7/agent.11515
set -e SSH_AGENT_PID; and set -x -U SSH_AGENT_PID 2269
::::::::::::::
.keychain/your-hostname-sh
::::::::::::::
SSH_AUTH_SOCK=/tmp/ssh-WeBckDhIvaF7/agent.11515; export SSH_AUTH_SOCK;
SSH_AGENT_PID=2269; export SSH_AGENT_PID;

At least on Mageia, the package adds these two files:
/etc/profile.d/99keychain.csh
/etc/profile.d/99keychain.sh
If these detect the directory ~/.keychain, they check to see if an SSH agent is running. If not, it starts one and prompts you for your passphrase. Either on the command-line or with a graphical pop-up depending on your interface

If you don't get those two system-wide files, put the appropriate code in your shell startup file. I use tcsh so I have this in my ~/.cshrc file (the shell reads that if there is no .tcshrc): 

# Things needed only in interactive shells
if ( "$?prompt2" == "1" ) then
	eval `keychain --eval id_rsa id_dsa id_ecdsa id_ed25519`
	if (-f $HOME/.keychain/$host-csh) then
		source $HOME/.keychain/$host-csh
	endif
endif

Re-start the SSH service

Re-start the SSH service however your version of *nix does this. You may find that the following signals your SSH daemon to re-read its configuration while continuing to run: 

# pkill -HUP sshd

If not, consult the below table or your system documentation. If you reconfigured your display manager, also restart that.

Linux # systemctl restart sshd
old Linux # /etc/init.d/sshd restart
Solaris and similar # /etc/init.d/sshd stop
# /etc/init.d/sshd start
BSD # /etc/rc.d/sshd restart

Be careful — some of the above commands may terminate existing SSH connections. If you are trying to do this work remotely, it may leave you locked out of the system!

Now your system is considerably more secure.