Privacy is at great risk today - especially for communications on the public Internet. There are certain things most people just don't want for others to be privy to, such as their personal finances, their current salary or other income, online orders, etc. In some cases (e.g. which books you have bought, or even looked at, on, there is no way you can protect that - will happily release that information to anyone willing to pay, or for free to the government. Other things, like a love letter to your wife, you have more control over who might gain access to the messages - you only need to worry about the security while the message is in transit. If it is encrypted before it leaves your client software and decrypted when it is received by her client software (this is called end-to-end security) then it is pretty safe from prying eyes (unless some hacker has access to either your or her computer).

Cryptography (and in particular encryption) has the ability to provide privacy in many situations, especially while it is stored, or is in transit over the Internet. Symmetric key encryption is virtually uncrackable (e.g. AES-128 or similar), assuming your key management is secure. Most attacks are against the key management part of a secure system (or against the data before it leaves your system or after it arrives at the destination, via Trojan horses on your computer). In many cases, social engineering is all that is needed (this was the primary tool of Kevin Mitnick, one of the most notorious hackers).


Key Management with Only Symmetric Key Cryptography

I once created a secure communication system for the U.S. Internal Revenue Service (the funding fell through on the RFP, so they never deployed it, although they did sign off on the design as meeting all their needs). This was before the Internet (it used modems via public phone lines). It was also before public key cryptography (e.g. RSA). The only technology available was symmetric key, and the only algorithm I could use was DES (it was the official government standard, and it was considered strong at the time). The hardest part of the system was key management. I created an ANSI X9.17 (now FIPS PUB 171) compliant Key Management System they could use to exchange keys securely. All of this was done with only Symmetric Key cryptography. There are much better and easier ways to accomplish key management today, with the advent of asymmetric key cryptography.

In general, two people need a single symmetric key to communicate securely, although even that should be updated periodically (based on time and/or volume of traffic). You can't share the key in the clear over the primary communications channel, or the hacker will see it too. If you encrypt it with another key (KEK - Key Encrypting Key) then the security depends on the KEK - you just move it back one level (although much less traffic is encrypted with the KEK - only session keys). There is still the problem of how to distribute the KEKs.

Each pair of communicating users actually need their own unique keys (or KEKs). For N users, you need N * (N-1) / 2 keys. This can get to be very large very quickly. For 2 users, that is 1. For 3 users t is 3. For 4 users it is 6. For 10 users it is 45. For 50 users it is 1225. For 1000 users it is 499,500. It is proportional to the square of the number of users (N2). And remember each of these needs to be periodically updated. We haven't even thought about how to verify that a given key is the correct one for a given user.

With a central Key Management System (KMS) like X9.17, it is possible for each end user to have a keying relationship with the KMS, which allows them to exchange keys (via two hops) with any other user that has a keying relationship with the KMS. Now the number of keys is proportional to N (far better than N squared). But it is clumsy to exchange keys via the KMS, and now the KMS is the ultimate hacker prize. If the hacker can compromise the KMS, they have compromised the entire system. All keys must be in plaintext at some point as they pass through from one user to another. And the KMS must be accessible by all users of the system. You must really trust the people who create and run the KMS.

Basically, key management is extremely difficult and unsatisfactory if only symmetric key is available.


Enter Asymmetric Key Cryptography

Asymmetric key cryptography solves the problem nicely. Each end user has exactly two keys. Specifically a matched pair of keys - one public, one private. Now the number of keys is proportional to N, but no central facility is needed (except to sign the certificates in which the public keys are embedded). During normal usage, Alice exchanges traffic directly with Bob, not via some central KMS. Data is encrypted end-to-end, even if it is relayed through several hops (e.g. S/MIME Email).

Actually asymmetric key cryptography has great properties for key management, but very poor properties for bulk data encryption or decryption. It is really only suitable for encrypting tiny amounts of data, say 128 to 512 bits at a time. How convenient - about the size of a symmetric key or message digest. In real world secure systems, that is all that is ever processed by the asymmetric key algorithms. Bulk encryption and decryption is handled by symmetric key algorithms (which are great for that). With digital signatures, the heavy lifting is done with a Message Digest algorithm (say processing the entire contents of War & Peace), and the asymmetric algorithm encrypts or decrypts only the tiny digest (512 bits or less).

There is a higher level construct called the Digital Envelope. The standard way to protect a message in a Digital Envelope is covered in RFC 5751, "Secure/Multipurpose Internet Mail Extensions (S/MIME) Version 3.2 Message Specification", January 2010 (which also covers Digital Signatures).


Digital Envelope in Math Notation

In math notation a Digital Envelope looks like this. Note - although this section uses "AES" and "RSA", any symmetric key and asymmetric key encryption algorithms can be used. AES and RSA are the most common such algorithms used.

M = Message

AES = Symmetric encryption algorithm, AES-1 = Symmetric decryption algorithm

SSK = Symmetric Session Key

EM = Encrypted Message

RSA = Asymmetric encryption algorithm, RSA-1 = Asymmetric decryption algorithm

Kpub = public key of recipient, Kpriv = private key of recipient

ESK = Encrypted Session Key

SSK' = Recovered Symmetric Session Key

M' = Recovered Message

Creating a Digital Envelope

SSK = randomly generated symmetric key for chosen symmetric algorithm


ESK = RSA(SSK, Kpub)

Send recipient M, MD and typically Kpub

Send EM and ESK to recipient

Opening a Digital Envelope

Receive EM and ESK from sender

SSK' = RSA-1(ESK, Kpriv)   (recovered symmetric session key)

M' = AES-1(EM, SSK')       (recovered message)

Destroy SSK'


Details on Using Digital Envelopes

Anyone can obtain the public key of the recipient needed to compose a digitally enveloped message, but since they need that key when they are composing the message (as opposed to reading it as with digital signatures), there must be some way for users to obtain the public keys of any recipients before the message is sent. Often once obtained, these are kept in some kind of local address book, along with the name and email address of the recipient. Public keys can be published in an LDAP directory, on a web page, or obtained by requesting a signed message from the recipient (usually the required public key is attached to a signed message from them).

The recipient only needs their own private key to open a digitally enveloped message, which they should already have. However, often usage of such a private key requires some kind of authorization, you when you try to open the message you may be asked to authorize use of the private key (possibly including supplying a passphrase, if the private key was stored with "strong protection").

Often this functionality is embedded in email clients (e.g. Outlook, Windows Live Mail and Thunderbird). The algorithms and message format (S/MIME) can actually be used in a wide variety of applications, such as secure file storage, secure transmission of files through DropBox or similar file transfer schemes, etc.

S/MIME is an "end to end" security mechanism. The functionality must be implemented in the clients, not in the server. This makes web based secure mail difficult to do (there must be some secure way for the web application running on the web server to use the private key on your client computer). One workaround is to add local functionality that runs in the browser on the client computer, using Java or ActiveX controls. See here for details on using S/MIME with Outlook Web Access (OWA). The techniques used may introduce security problems that cancel out any advantage to using S/MIME. It is better to use "thick" clients for secure email.

Unlike digitally signed email messages, I cannot easily create an example digitally enveloped message since you would need a client digital certificate and corresponding private key, and I would need to obtain and use that to send you a digitally enveloped message that only you could open. I may at some point create an "autoresponder" service that will accept a digitally signed message from anyone, and reply with a digitally enveloped message to that person.


Secure Communications Systems

An end-to-end secure communications system consists of several parts:

1. A Public Key Infrastructure to issue and manage valid client digital certificates for each user.

2. Client software that can create and use digital signatures and digital envelopes (e.g. Outlook or Windows Live Mail). It must be possible to load the above client digital certificates into this software, and obtain the client digital certficiate of any intended recipient(s). The client software also needs some way to determine the revocation status and trust of all client certs used. This could require installing the root and intermediate certs of the client cert hierarchy in every user's client software.

3. Communications channels and servers that the secured messages can flow through (these need not be secure). For example. Internet E-mail.

If you use a commercial online CA for the client certificates (e.g. VeriSign) the root cert of their client certificate hierarchy is installed in most client software by default. They also provide CRLs and OCSP for determining certificate revocation status. VeriSign unfortunately has no way to verify anything other than that you can receive email messages at the supplied email address, so you can't really trust any other information in their client certs (this is why the Subject DN of VeriSign client certs contains the phrase "personna non validated").

If you use a private hierarchy CA for the client certificates, you may need to download and install the root and intermediate certs for the client cert hiearchy into all client software (or some other way must be provided for client software to determine whether certs chain up to a trusted root). There must also be some way (typically CRL or OCSP) for the client software to determine revocation status of each client cert at time of use. If the CA is run by an organization that employs or has a customer relationship with the users, the information in the certs may be very carefully verified, so you can trust it.

In either case, each user can publish their client digital certificate to allow others to obtain it when they want to send them digital enveloped messages. This can be via an LDAP directory or web pages, or sent attached to digtially signed message (and then saved in a local address book). Ideally client software should be able to retrieve recipient's certs from the CA via some kind of protocol.