Public-key cryptography, such as that used for PGP, uses one key for encryption, and one key for decryption. Traditional cryptography, however, uses the same key for encryption and decryption; this key must be known to both parties, and thus somehow transferred from one to the other securely.
To alleviate the need to securely transmit the encryption key, public-key encryption uses two separate keys: a public key and a private key. Each person's public key is available by anyone to do the encryption, while at the same time each person keeps his or her private key to decrypt messages encrypted with the correct public key.
There are advantages to both public key and private key cryptography, and you can read about those differences in the RSA Cryptography FAQ, listed at the end of this section.
PGP (Pretty Good Privacy) is well-supported on Linux. Versions 2.6.2 and 5.0 are known to work well. For a good primer on PGP and how to use it, take a look at the PGP FAQ: http://www.pgp.com/service/export/faq/55faq.cgi
Be sure to use the version that is applicable to your country. Due to export restrictions by the US Government, strong-encryption is prohibited from being transferred in electronic form outside the country.
US export controls are now managed by EAR (Export Administration Regulations). They are no longer governed by ITAR.
There is also a step-by-step guide for configuring PGP on Linux available at http://mercury.chem.pitt.edu/~angel/LinuxFocus/English/November1997/article7.html. It was written for the international version of PGP, but is easily adaptable to the United States version. You may also need a patch for some of the latest versions of Linux; the patch is available at ftp://metalab.unc.edu/pub/Linux/apps/crypto.
There is a project maintaining a free re-implementation of pgp with open source. GnuPG is a complete and free replacement for PGP. Because it does not use IDEA or RSA it can be used without any restrictions. GnuPG is in compliance with OpenPGP. See the GNU Privacy Guard web page for more information: http://www.gnupg.org/.
More information on cryptography can be found in the RSA cryptography FAQ, available at http://www.rsa.com/rsalabs/newfaq/. Here you will find information on such terms as "Diffie-Hellman", "public-key cryptography", "digital certificates", etc.