The "U" Encoding for Encoded-Words in Email

The "Encoded Word" conventions have been used extensively in email headers and elsewhere to permit the encoding of non-ASCII characters where only ASCII ones are normally permitted. That existing encoded-word specification defines only two kinds of encoding, one of which cannot be understood easily by people ("B", the MIME "Base64" encoding) and the other of which ("Q", so-called Quoted Printable) has been widely discredited. This document specifies a third encoding, based on the "\u'NNNN'" convention, that is easily accessible by users and much more closely tied to contemporary practices. Unlike the "B" and "Q" encodings, which were specified at a time when many coded character sets were in common use, it is now appropriate to tie a new encoding specifically to Unicode and the corresponding ISO Standard , viewing conversion to local character sets, if necessary at all, to be a local matter. Consequently, this specification permits only the combination "=?iso-10646-UCS-4?u?". Note in Draft: If we were really going to do this, it would make sense to define a charset that would actually reflect Unicode code points, not some encoding of them. Neither of the currently-registered "iso-10646-UCS-4" nor "UTF-32" and its variations are quite right for that purpose. Cf. http://www.iana.org/assignments/character-sets If adopted, it is intended not only as an alternative to "Q" and "B", but also as an alternative to the %-encoding of Section 2.1 of the URI Specification of UTF-8 (and other) strings. %-encoding was more than adequate for its original purpose of encoding eight-bit character sets, notably ISO 8859-1, but is problematic for email (especially addresses and fields related to them) because "%" has an important historic (and still occasionally used) meaning in those contexts and because its use to encode already-encoded forms of multi-octet character sets, such as UTF-8 and Unicode, creates strings that are at least as difficult for end users to interpret as Base64.

This document, if approved, updates the Encoded-Word specification and the specification for the use encoded-words with language information to permit use of an additional encoding type, type "U".

Some reasonable understanding of Encoded-Words and the Quoted-Printable, Base64, and %-encoding conventions are required to understand this introductory material but not the proposal itself. The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as defined in RFC 2119.

RFC Editor: In the unlikely event that you see this subsection, it should be removed before publication. The current version of the proposal is intended for possible discussion in the EAI, IRI, and PRECIS WGs to see if it sheds light on other issues being discussed in those WGs. If discussions are of interest, they should occur on the mailing lists associated with those groups. This Internet Draft is, at this point, intended only to promote discussion of a possibly-useful building block for other work. It is not proposed for adoption by the IETF for any purpose.

A new encoding form for encoded words is defined with code "u". The associated encoded-text string is consistent with the rules in Section 4 of RFC 2047, i.e., it consists of ASCII characters with space, tab, and "?" characters excluded. Non-ASCII characters are encoded using the \u'NNNN' form, where "NNNN" consists of four to six hexadecimal digits designating a Unicode (ISO 10646) code point. That encoding convention is defined in RFC 5137 together with an explanation of why the quotes should be required. As an example, the German equivalent of the string "This is nuts", would appear in the extended form of RFC 2231 (updated by verified Erratum 478 ) as =?iso-10646-UCS-4+de?u?Das ist verr\u'00FC'ckt?=