Internet Draft: The Qpopper MIME Mangling and Macro Extensions to POP3 R. Gellens Document: draft-gellens-pop-mangle-00.txt Qualcomm Expires: August 2006 February 2005 The Qpopper MIME Mangling (X-MANGLE) and Macro (MDEF) Extensions to POP3 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2006). All Rights Reserved. Abstract This document describes two extensions to the POP protocol that have been supported for several years by some client and servers. One extension, X-MANGLE, allows clients to request that MIME body parts be removed or converted to a simpler format, that only selected headers be transmitted, and/or to for the message to be transcoded to a different character set. The other extension, X-MACRO, allows clients to define macros which are expanded when used, saving repetitive transmissions. Gellens [Page 1] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 These extensions have been useful in at least two situations: they allow clients on constrained devices to avoid downloading body parts which cannot be used on the device, and clients which use the TOP command to "peek" at messages can have the preview transformed into more usable content, as well as avoiding the transmission of undesired headers. Gellens [Page 2] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 Table of Contents 1 Conventions Used in this Document . . . . . . . . . . . . . . 3 2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 MIME Mangling (X-MANGLE) . . . . . . . . . . . . . . . . . . . 4 4 Macro Definition (X-MDEF) . . . . . . . . . . . . . . . . . . . 6 5 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 6 Security Considerations . . . . . . . . . . . . . . . . . . . . 7 7 Normative References . . . . . . . . . . . . . . . . . . . . . 8 8 Informative References . . . . . . . . . . . . . . . . . . . . 8 9 Author's Address . . . . . . . . . . . . . . . . . . . . . . . 8 Intellectual Property Statement . . . . . . . . . . . . . . . . 8 Full Copyright Statement . . . . . . . . . . . . . . . . . . . 9 1 Conventions Used in this Document The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in "Key words for use in RFCs to Indicate Requirement Levels" [KEYWORDS]. In examples, "C:" is used to indicate lines sent by the client, and "S:" indicates those sent by the server. Line breaks within a command example are for editorial purposes only. This document is intended for information only. It is not proposing a standard. 2 Introduction Since May of 2000, the Qpopper open-source POP server has supported the MIME mangling (X-MANGLE) and macro (X-MACRO) extensions to the [POP3] protocol. These extensions have been useful in at least two situations: o clients on constrained devices can avoid downloading body parts which cannot be used on the device, can have the content transcoded into a more desirable character set, and can have undesired headers removed prior to transmission; and o clients which use the TOP command to "peek" at messages can have the preview transformed into more usable content, and can make the preview more useful by eliminating extraneous headers. Gellens [Page 3] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 The Eudora for PalmOS client (part of the Eudora Internet Suite for PalmOS) makes use of both extensions. The Eudora clients for both MacOS and Windows make use of the X-MANGLE extension when using the TOP command. With the prevalence of attachments and formatted mail, these extensions have been useful in solving the following problems: o clients on constrained devices or links often download attachments that waste bandwidth and must be parsed and discarded, since they cannot be used on the device; o clients on slow links or memory-limited devices waste bandwidth receiving undesired headers; o when the TOP command is used to "peek" at messages, especially when using a bandwidth-constrained connection (such as a non-3G mobile link or a slow dialup with a laptop), often the captured portion of the message is unreadable, being a mess of HTML tags (since many clients show these previews in their raw form, and, being truncated, the HTML may not be syntactically valid and hence difficult for the client to parse) 3 MIME Mangling (X-MANGLE) This extension is announced by including X-MANGLE in the CAPA [POP-EXT] response. The X-MANGLE extension can be used as a modifier to the TOP, RETR, and LIST commands. The result is to: o Remove MIME body parts o Optionally remove all headers except those requested o Optionally encode in a specified character set The syntax is: mangle = "mangle(" mangling ")" mangling = attribute "=" value *(";" attribute "=" value ) attribute = "text" / "headers" / "charset" / "lang" / "wrap" value = text-type / header-list / charset text-type = "plain" / "html" / "enriched" header-list = header *("," header) charset = "us-ascii" / iso-8859-r / "utf-8" / "cp1252" / "iso_2022-jp" iso-8859-r = "iso-8859-" range range = "1"-"15" Gellens [Page 4] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 The following command results in transmitting message 10 as a single part of content-type text/html with only those headers which were requested: retr 10 mangle(text=html;headers=to:,cc:,from:,date:) The message can be sent as one of three text types: text/plain, text/html, and text/enriched. The source format can be any of these types or text/plain;format=flowed. If the headers are not specified, the default is to return the following ones: "to:", "cc:", "subject:", "from:", "date:", "reply-to:", "x-Priority:", and "x-UIDL:". Note that "lang" and "wrap" are currently ignored. The value supplied with the "headers" attribute is a comma-separated list of desired headers. Header in the message are matched up to what is entered, so "headers=sub" will cause both the "Subject:" and "Subversive:" headers to be returned, if both exist in the message. In order to be sure to match only the exact header desired, clients may wish to include the ":". The "Mime-Version:" and "Content-Type:" headers are always included, and reflect the mangling that has been applied. Note that there must not be any spaces within the 'mangle' string (since spaces are used to separate parameters to POP commands). Case is not significant with the X-MANGLE extension. 'mangle(text=plain)' is the same as 'mAnglE(TeXT=PLaiN)', 'headers=to' is the same as 'headers=TO'. Examples: top 1 100 mangle(text=plain;headers=to:,subject:) retr 1 mangle(text=plain;headers=to:,subject:) retr 1 mangle(text=plain;charset=us-ascii;headers=to:,subject:) Examples of use: The client retrieves message 1 as it exists in the message store: C: retr 1 S: +OK 665 octets S: Received: from foo.example.net (localhost [127.0.0.1]) S: by hotearth.example.net (8.12.10/8.12.3/1.0) with S: ESMTP id j3L0Lq9B014701 S: for ; S: Wed, 20 Apr 2005 17:21:52 -0700 (PDT) Gellens [Page 5] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 S: Received: (from randy@localhost) S: by hotearth.example.net (8.12.10/8.12.1/Submit) id S: j3L0Lqqj014700 for poptest8; S: Wed, 20 Apr 2005 17:21:52 -0700 (PDT) S: Date: Wed, 20 Apr 2005 17:21:52 -0700 (PDT) S: From: Randall Gellens S: Message-Id: <200504210021.j3L0Lqqj014700@hotearth.example.net> S: To: poptest8@example.net S: Subject: test #1 S: X-UIDL: o=6"!2HZ!!LR+!!7C!"! S: Status: RO S: S: This is a test. S: . The client retrieves message 1 as text/html and with only those headers which start with 'foo' (in this case, none): C: retr 1 mangle(text=html;headers=foo) S: +OK Message follows S: Mime-version: 1.0 S: Content-Type: text/html;charset="us-ascii" S: S:
This is a test.
    S: 
S: . Note that some servers (such as Qpopper) also support the "-no-mime" user name hack. As a way to enable MIME-mangling with clients that do not implement X-MANGLE, add "-no-mime" to the user name. For example, if the userid is "mary", enter it in the client as "mary-no-mime". 4 Macro Definition (X-MDEF) This extension is announced by including X-MACRO in the CAPA [POP-EXT] response. The client uses the MDEF command to define a macro. The server may impose limits on the length of the macro name and/or value and the number of macros. In the case of Qpopper, the macro (name and value plus parenthesis) must be no greater than 500 octets (100 in older versions of the server), and clients may define up to ten macros. Redefining a macro of the same name is permitted; the new macro replaces the old one. Gellens [Page 6] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 Syntax: macro = "MDEF" macro-name "(" macro-value ")" macro-name = 1*printable macro-value = 1*printable printable = Note: must not have any spaces within mdef command (since spaces are used to separate parameters to POP commands). Once defined, the name of a macro can be used as a parameter to any POP command. When used, the macro value replaces the macro name as the command parameter. Example: mdef mang(mangle(text=plain;headers=to:,subject:)) The above example defines a macro with the name 'mang' whose value is "mangle(text=plain;headers=to:,subject:)". One situation in which macros are helpful is when clients using bandwidth-constrained links wish to use the X-MANGLE extension multiple times, with identical values in each usage. For example, rather than a client entering the idential X-MANGLE extension as a parameter to every TOP or RETR command, the client first defines the extension as the value of a macro, then uses the macro name with the TOP or RETR commands. Example of use: C: mdef html(mangle(text=html;headers=foo)) S: +OK Macro "html" accepted C: retr 1 html S: +OK Message follows S: Mime-version: 1.0 S: Content-Type: text/html;charset="us-ascii" S: S:
This is a test.
    S: 
S: . 5 IANA Considerations None. Gellens [Page 7] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 6 Security Considerations Alteration of message text destroys any security wrapper (digital signature) and is useless within encrypted content. However, since the alteration is only for the text sent in direct response to the client, the original unaltered message remains in the message store. Servers need to be careful that macro expansion not exceed buffer sizes. 7 Normative References [POP3] RFC 1939 [POP-EXT] RFC 2449 8 Informative References None. 9 Author's Address Randall Gellens QUALCOMM Incorporated 5775 Morehouse Drive San Diego, CA 92121 randy@example.net Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. Gellens [Page 8] Expires August 2006 Internet Draft X-Mangle and X-Macro February 2006 The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Full Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Gellens [Page 9] Expires August 2006