Understanding RSA/DSA authentication

posted onJuly 27, 2001

by hitbsecnews

In this series, you'll learn how RSA and DSA authentication work, and see how to set up passwordless authentication the right way. In the first article of the series, Daniel Robbins focuses on introducing the RSA and DSA authentication protocols and showing you how to get them working over the network. Many of us use the excellent OpenSSH (see Resources later in this article) as a secure, encrypted replacement for the venerable telnet and rsh commands. One of OpenSSH's more intriguing features is its ability to authenticate users using the RSA and DSA authentication protocols, which are based on a pair of complementary numerical keys. As one of its main appeals, RSA and DSA authentication promise the capability of establishing connections to remote systems without supplying a password. While this is appealing, new OpenSSH users often configure RSA/DSA the quick and dirty way, resulting in passwordless logins, but opening up a big security hole in the process.

What is RSA/DSA authentication?

SSH, specifically OpenSSH (a completely free implementation of SSH), is an incredible tool. Like telnet or rsh, the ssh client can be used to log in to a remote machine. All that's required is for this remote machine to be running sshd, the ssh server process. However, unlike telnet, the ssh protocol is very secure. It uses special algorithms to encrypt the data stream, ensure data stream integrity and even perform authentication in a safe and secure way.

However, while ssh is really great, there is a certain component of ssh functionality that is often ignored, dangerously misused, or simply misunderstood. This component is OpenSSH's RSA/DSA key authentication system, an alternative to the standard secure password authentication system that OpenSSH uses by default.

OpenSSH's RSA and DSA authentication protocols are based on a pair of specially generated cryptographic keys, called the private key and the public key. The advantage of using these key-based authentication systems is that in many cases, it's possible to establish secure connections without having to manually type in a password.

While the key-based authentication protocols are relatively secure, problems arise when users take certain shortcuts in the name of convenience, without fully understanding their security implications.

In this article, we'll take a good look at how to correctly use RSA and DSA authentication protocols without exposing ourselves to any unnecessary security risks. In my next article, I'll show you how to use ssh-agent to cache decrypted private keys, and introduce keychain, an ssh-agent front-end that offers a number of convenience advantages without sacrificing security. If you've always wanted to get the hang of the more advanced authentication features of OpenSSH, then read on.