Lemonade Internet Draft: CONVERT S. H. Maes Document: draft-ietf-lemonade-convert-02 R. Cromwell (Editors) Expires: August 2006 February 2006 CONVERT Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2006). Abstract CONVERT defines extensions to the IMAPv4 Rev1 protocol [RFC3501] for optimization in a mobile setting, aimed at allowing adaptation and transcoding of attachments as needed by the client. Conversion (adaptation, transcoding) may be requested by the client and performed by the server on a best effort basis or decided by the server based on its knowledge of the client capabilities, user or administrator preferences or its settings. Conventions used in this document Maes [Page 1] February 2006 In examples, "C:" and "S:" indicate lines sent by the client and server respectively. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. An implementation is not compliant if it fails to satisfy one or more of the MUST or REQUIRED level requirements for the protocol(s) it implements. An implementation that satisfies all the MUST or REQUIRED level and all the SHOULD level requirements for a protocol is said to be "unconditionally compliant" to that protocol; one that satisfies all the MUST level requirements but not all the SHOULD level requirements is said to be "conditionally compliant." When describing the general syntax, some definitions are omitted as they are defined in [RFC3501]. Table of Contents Status of this Memo...............................................1 Copyright Notice..................................................1 Abstract..........................................................1 Conventions used in this document.................................1 Table of Contents.................................................2 1. Introduction...................................................3 2. Relation with other E-mail specifications......................3 3. Interactions between the Client and Server when supporting CONVERT.............................................4 4. Discovery with the CAPABILITY and GETANNOTATION Commands.......4 4.1. CAPABILITY................................................4 4.2. GETANNOTATION.............................................4 5. CONVERT BODY and BINARY data item extension....................5 6. CONVERT transcoding parameters.................................7 7. FETCH response extensions......................................7 8. Status responses, Response code extensions.....................7 9. Formal Syntax..................................................8 Security Considerations..........................................10 Normative References.............................................10 Informative References...........................................11 Normative Appendices.............................................11 A. Security Issues for Proxy-Based Implementations............11 Future Work......................................................12 Version History..................................................12 Acknowledgments..................................................13 Authors Addresses................................................13 Intellectual Property Statement..................................13 Disclaimer of Validity...........................................14 Copyright Statement..............................................14 Maes Expires – August 2006 [Page 2] February 2006 1. Introduction CONVERT is based on IMAPv4 Rev1 [RFC3501]. It defines additional enhancements for optimization in a mobile setting: extensions to the IMAPv4 Rev1 protocol [RFC3501] that allows adaptation and transcoding of body parts as needed by the client. Conversion (adaptation, transcoding) may be requested by the client and performed by the server on a best effort basis or decided by the server based on its knowledge of the client capabilities, user or administrator preferences or its settings. These are important features required in particular to support mobile email use cases [MEMAIL], [OMA-ME-RD]. A server that supports CONVERT can convert leaf body parts to other formats to be viewed on a mobile device. The client can explicitly request a particular conversion. The server complies on a best effort basis. When not possible, the server determines based on its own strategy (e.g. based on knowledge of the client as discussed hereafter) how to convert. If the server knows the characteristics of the device or can determine them (out of scope of CONVERT), the attachments can also be optimized for the capabilities of the devices (e.g. form factor of pictures). This is a recommended server behavior. 2. Relation with other E-mail specifications The Lemonade Profile [LEMONADEPROFILE] specifies the Lemonade Pull Model that governs the exchanges among mail servers or between desktop mail client and mail servers. Lemonade investigates adding mobile optimizations for the next version of the profile. CONVERT should be seen as a way to address the issues of mobile optimization and an input to the Lemonade Profile work. It addresses the topic of attachment conversions identified by the Lemonade work as critical for mobile email. CONVERT does not address conversion and streaming of media as also identified of interest by lemonade. This has not been identified as relevant to mobile e-mail [MEMAIL] and [OMA-ME-RD]. CONVERT depends on the ANNOTATEMORE extension [ANNOTATEMORE] to support discover of supported conversion formats. In addition, to use CONVERT, the client and server MUST support the IMAP Binary specification [RFC3516]. Maes Expires – August 2006 [Page 3] February 2006 3. Interactions between the Client and Server when supporting CONVERT This document does not specify how conversions are to be done by the server. It is however noted that the conversion MUST NOT affect the document stored in the message storage server in order to remain accessible unaffected through other clients that do not request conversion as well as in order to allow replies or forward of the documents as they were originally received by the server. 4. Discovery with the CAPABILITY and GETANNOTATION Commands 4.1. CAPABILITY A server that supports CONVERT MUST return "CONVERT", "ANNOTATEMORE", and "BINARY" and in the CAPABILITY response. Example: A server that implements LDELIVER. C: a001 CAPABILITY S: * CAPABILITY IMAP4rev1 AUTH=LOGIN IDLE CONVERT BINARY ANNOTATEMORE S: a001 OK CAPABILITY completed 4.2. GETANNOTATION To determine which conversions are supported, mailbox annotations are used. For each mime-type that the client is interested in, it MAY determine which conversions are supported. For any proposed conversion, the client MAY discover a list of optional parameters that the conversion SHOULD accept. MIME type/subtype are mapped to a hierarchy under the root “/convert”. For each mime type under “/convert”, a value for “types.shared” SHOULD exist which is a semicolon separated list of output formats. Example: Discover all image conversions C: a GETANNOTATION "INBOX" "/convert/image/*" "types.shared" S: * ANNOTATION "INBOX" "/convert/image/jpeg" ("types.shared” "image/jpeg;image/png;image/gif;image/wbmp”) S: a OK GETANNOTATION complete The above example shows that the server supports one kind of input image transcoding, from image/jpeg to four different outputs: JPEG, PNG, GIF, and WBMP. For a given conversion, optional transcoding parameters MAY be present. These are mapped into a value “params.shared” under “/convert/sourcetype/destinationtype”. Maes Expires – August 2006 [Page 4] February 2006 Example: Discover optional parameters for image/jpeg -> image/gif. C: a GETANNOTATION "INBOX" "/convert/image/jpeg/image/gif" "params.shared" S: * ANNOTATION "INBOX" "/convert/image/jpeg/image/gif" ("params.shared” "width;height;depth;interlaced”) S: a OK GETANNOTATION complete The above example shows that to convert from image/jpeg to image/gif, the transcoding supports the following types of option parameters: width, height, depth, and interlaced. A client MAY use these values to check whether or not a desired conversion is possible, or it may present the parameters as a GUI preferences pane for the user to customize. A baseline set of register transcoding parameter names should be standardized (see [OMA-STI]) in the future, and it is beyond the scope of this spec to allow the client to discover the underlying legal values that these parameters may take. 5. CONVERT BODY and BINARY data item extension CONVERT is a FETCH extension used to transcode the media type of a leaf MIME part into another media type, and/or the same media type, with different encoding parameters. It adds new options to the section-spec part of the BODY data item, a new FETCH response data item BODYPARTSTRUCTURE, and new response codes. It is also expected to work with IMAP BINARY data item extension, whose grammar is modified as well. The response to a CONVERT request always includes a BODYPARTSTRUCTURE. CONVERT’s syntax is modeled after the HEADER.FIELDS syntax in RFC3501, and is generally structured as: BODY[section-part.CONVERT[.STRICT] (“media type” “subtype” (parameters))] BODY.PEEK[section-part.CONVERT[.STRICT] (“media type” “subtype” (parameters))] BINARY[section-part.CONVERT[.STRICT] (“media type” “subtype” (parameters))] BINARY.PEEK[section-part.CONVERT[.STRICT] (“media type” “subtype” (parameters))] Maes Expires – August 2006 [Page 5] February 2006 BINARY.SIZE[section-part.CONVERT[.STRICT] (“media type” “subtype” (parameters))] Example: The client fetches body part section 3 in the message with the message sequence number of 2 and asks to have that attachment converted to pdf format. C: a001 FETCH 2 BODY[3.CONVERT (“APPLICATION” “PDF”)] S: * 2 FETCH (BODYPARTSTRUCTURE[3] ("APPLICATION" "PDF" () NIL NIL "Base64" 2135 NIL NIL NIL) BODY[3] {2135} ) S: a001 OK FETCH COMPLETED Example: The client requests for conversion of a text/html section as text/plain and asks for a charset of us-ascii. The server cannot respect the charset request because there are non-us-ascii characters in the html code. Thus, in the untagged response, the server returns the charset of UTF-8 and utilizes a content transfer encoding capable of representing the full 8-bit range, along with the converted text. C: a001 FETCH 2 BODY[3.CONVERT (“text” “plain” (“charset” “us- ascii”))] S: * 2 FETCH (BODYPARTSTRUCTURE[3] ("TEXT" "PLAIN" () NIL NIL "Base64" 2135 181 NIL NIL NIL) BODY[3] {2135} the document in text/plain format ) S: a001 OK FETCH COMPLETED Example: The client requests for conversion of a text/html section as text/plain, but only wants 1000 bytes, starting from byte 2001. C: a001 FETCH 2 BODY[3.CONVERT (“TEXT” “PLAIN” (“CHARSET” “us- ascii”))]<2001.1000> S: * 2 FETCH (BODYPARTSTRUCTURE[3] ("TEXT" "PLAIN" () NIL NIL "7bit" 2135 181 NIL NIL NIL) BODY[3]<2001> {135} bytes 2001 - 2135 of the document in text/plain format ) S: a001 OK FETCH COMPLETED The server is not required to respect a particular transcoding request or its request parameters unless the STRICT qualifier is used, although it MAY try to make a best effort to fulfill that request if it is omitted. Indeed, the server may know a priori information about the client obtained through a different mechanism outside the scope of CONVERT (e.g. dynamically through device description mechanisms or when the device was associated to the account). These preferences may be used to predefine what conversions are possible. Ideally the client should request the same conversions. Maes Expires – August 2006 [Page 6] February 2006 In addition, this information may also allow attachment adaptation (e.g. picture form factor) instead of solely format conversion. 6. CONVERT transcoding parameters CONVERT servers MAY support additional transcoding parameters for each media type. All CONVERT compliant servers MUST minally support recognition of charset for text/* mime types, although they may decline to honor some requests. For media types other than text, it is beyond the scope of this document to define conversion parameters. In general however, CONVERT compliant servers MAY choose to support additional parameters, and if so, they SHOULD follow the OMA STI 1.0 spec [OMA-STI] adopting the same parameter names as defined in second 5.2.4 and above for the most popular image/*, video/*, and audio/* codecs As an example, in section 5.2.6.2 of [OMA-STI], the parameters "width" and "height" are defined. The following example illustrates how these OMA STI parameters are used with CONVERT. C: a001 UID FETCH 100 BINARY[2.CONVERT (“IMAGE” “JPG” (“WIDTH” “128” “HEIGHT” “96”))] S: * 2 FETCH (UID 100 BODYPARTSTRUCTURE[2] ("IMAGE" "JPG" () NIL NIL "8bit » 4182 NIL NIL NIL) BINARY[2] ~{4182} ) S: a001 OK UID FETCH COMPLETED 7. FETCH response extensions The BODYPARTSTRUCTURE data item is introduced when using the CONVERT extension. It follows the exact syntax specified in the [RFC3501] BODYSTRUCTURE data item, but contains information for only the converted part. All information contained in BODYPARTSTRUCTURE pertains to the state of the part after it is converted, such as the converted mime-type, sub-type, size, or charset. The client must respect the return values and not assume the conversion request succeeds exactly as requested unless the STRICT qualifier is used. 8. Status responses, Response code extensions Some transcodings may require parameters. If a transcoding request is sent for a format which requires parameters, the server can reply with a BAD response. Likewise, malformed mime types may also generate BAD responses. Maes Expires – August 2006 [Page 7] February 2006 If the server if unable to perform the requested conversion because a resource is unavailable (internal error, transcoding service down) than a BAD response should be returned. If a request is denied because of an operational error, such as lack of disk space, or because the requested conversion for some reason cannot be performed, and there is no fallback for this particular device (such as the case where a proprietary document format has no existing transcoding implementation, and the server recognizes that the client has no default viewer for it), the server SHOULD return a NO response. Otherwise, the server should return an OK response. The client in general can tell from the BODYPARTSTRUCTURE response whether or not its request was honored exactly, but may not know exactly why it differents. The following extension response codes are provided for OK and NO responses to disambiguate those situations, or warn about possible important data loss. INFORMATIONLOSS – the conversion was satisfied for conversion request, but it may have resulted in the loss of important data (primarily of use for loss of text data, since richmedia is often compressed with loss) BADPARAMETERS “(“ convert-params “)” – the listed parameters were not understood, or could not be honored for the reasons noted in section- text. In particular, a CONVERT.STRICT request may be fail because the server has no way to honor it. SERVEROVERRIDE – the server override the request because it determined it could substitute a better one based on preferences, device capability knowledge, or server policy. If CONVERT.STRICT is used, the server may not return SERVEROVERRIDE. It must either honor the request, or fail. 9. Formal Syntax The following syntax specification uses the augmented Backus-Naur Form (ABNF) notation as used in [ABNF], and incorporates by reference the Core Rules defined in that document. This syntax augments the grammar specified in [RFC3501] and [RFC3516]. In the ABNF section-msgtext grammar in section 9 of [RFC3501], Section-msgtext is hereby amended to read: Maes Expires – August 2006 [Page 8] February 2006 section-msgtext =/ “CONVERT” SP convert-params convert-params = "(" (media-basic / default-conversion) [SP "(" transcoding-params ")"] ")" transcoding-params = transcoding-param-name SP transcoding-param- value *(SP transcoding-param-name SP transcoding-param-value) transcoding-param-name = astring transcoding-param-value = astring default-conversion = "NIL" In the ABNF syntax “section-binary” of [RFC3516], is amended to: section-binary = "[" [section-part [“.CONVERT”[.STRICT] SP convert-params] "]" In the ABNF syntax “msg-att-static” of [RFC3501], is amended to: msg-att-static =/ “BODYPARTSTRUCTURE” “(“ body-type-1part “)” In the ABNF syntax “resp-text-code” of [RFC3501], is amended to: Resp-text-code =/ “INFORMATIONLOSS” / “SERVEROVERRIDE” / “BADPARAMETERS” SP “(“ bad-param-list “)” bad-param-list = transcoding-params In addition, the following ABNF describes the syntax of the GETANNOTATION entries in Section 4.2 convert-entry-req = available-conversions / available- transcoding-parameters available-conversions = “/convert/” from-mime-type from-mime-type = “*” /(astring [“/” (astring / “*”)] ; i.e. “*” or “type/*” or “type/subtype” from-concrete-mime-type = astring “/” astring ; i.e. “type/subtype” to-mime-type = astring “/” astring Maes Expires – August 2006 [Page 9] February 2006 available-transcoding-parameters = “/convert/” from-concrete- mime-type “/” to-mime-type ;i.e. /convert/fromtype/fromsubtype/totype/tosubtype Finally, two standard annotation attributes are defined. Under available-conversions entry, there will be an attribute named “types.shared” with the following ABNF: types-shared-value = from-concrete-mime-type(“;” from-concrete- mime-type)* And under an available-transcoding-parameters entry, there will be an attribute named “params.shared” with the following ABNF: params-shared-value = transcoding-param-name (“;” transcoding- param-name)* Security Considerations It is to be noted that some conversions may present security threatens (e.g. converting a document to a damaging executable). Clients should be careful when requesting conversions or processing transformed attachments. Servers should avoid dangerous conversions if possible. It may of interest to consider, when possible, server- side verification of the converted attachments before providing to the server. When an implementation is proxy-based, this may create new security issues. These issues are discussed in detail in Appendix C, because the issues are dependent on the implementation of this protocol rather than inherent to the protocol itself. On bandwidth limited mobile networks where users pay per data volumes, spam may become an important issue. It can be mitigated with appropriate filters and server-side spam prevention tools. These are of course outside the scope of CONVERT. It is also recommended that clients be explicitly registered with the server through separate channels / application. Exchanges should then be paired. Normative References [ANNOTATEMORE] Daboo, C., "IMAP ANNOTATEMORE Extension" , draft-daboo-imap-annotatemore-07, 2005. Maes Expires – August 2006 [Page 10] February 2006 Informative References [LEMONADEPROFILE] Maes, S.H. and Melnikov A., "Lemonade Profile", draft-ietf-lemonade-profile-XX.txt, (work in progress). [MEMAIL] Maes, S.H., “Lemonade and Mobile e-mail", draft-maes- lemonade-mobile-email-xx.txt, (work in progress). [OMA-ME-RD] Open Mobile Alliance Mobile Email Requirement Document, (Work in progress). http://www.openmobilealliance.org/ [OMA-STI] Open Mobile Alliance, Standard Transcoding Interface Specification, version 1.0, [Work in progress] (http://member.openmobilealliance.org/ftp/Public_documents/BAC/STI /Permanent_documents/OMA-STI-V1_0-20050209-D.zip). [P-IMAP] Maes, S.H., Lima R., Kuang, C., Cromwell, R., Ha, V. and Chiu, E., Day, J., Ahad R., Jeong W-H., Rosell G., Sini, J., Sohn S-M., Xiaohui F. and Lijun Z., "Push Extensions to the IMAP Protocol (P-IMAP)", draft-maes-lemonade-p-imap-xx.txt, (work in progress). [RFC2119] Brader, S. "Keywords for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. http://www.ietf.org/rfc/rfc2119 [RFC3501] Crispin, M. "IMAP4, Internet Message Access Protocol Version 4 rev1", RFC 3501, March 2003. http://www.ietf.org/rfc/rfc3501 Normative Appendices A. Security Issues for Proxy-Based Implementations In some implementations, the client may connect to a proxy that sits in an operator network, but the backend email storage server sits in a separate enterprise network. The enterprise network is assumed to be secure, but the operator network may not be trusted. If unencrypted information lies in the operator network, that information is vulnerable to attacks. If the CONVERT extensions are all implemented in the enterprise network, then the proxy on the carrier should be an encrypted SSL pass-through proxy. SSL ensures confidentiality and integrity of the proxied datastream, ensuring that the proxy cannot monitor the content of messages, nor inject commands to modify or corrupt the enterprise email server to corrupt the user's mailbox. The additional cost for this design is that the backend enterprise email Maes Expires – August 2006 [Page 11] February 2006 server and the client devices must have additional processing to handle the overhead of SSL. If the CONVERT compliant server is implemented as a backend IMAP server with additional command processing done on the proxy, there are more complex security issues. This proxy must be able to send commands to the backend server to accomplish its tasks, as well as read information coming from the backend server. An attacker who compromises the proxy thus can send commands to the backend to change the state of the mail storage, possibly corrupting it. In addition, it can read responses from the mail server that might contain confidential email information. This proxy may also send bogus responses back to the client. Clearly, this setup is not an ideal issue and many complications that make this problem complex to solve. The suggestion recommended is to remedy the problem of unencrypted, untagged FETCH responses that may contain confidential information. Encrypted responses should be used in place of any untagged FETCH responses, which contain encrypted message information to be passed through the proxy on the operator network. The key exchange for encryption should not occur through the proxy. It has to be done through another channel: manually entered by user (e.g. password), or via an HTTP SSL request to the enterprise server. Any other additional server responses containing sensitive information (passwords, etc.) should be encrypted. It is beyond the scope of this document to define the implementation of transcoding services. In general, it is recommended that they reside within the same domain as the IMAP server, and are not performed by third party services, which may compromise the privacy of the data being transcoded. Future Work Standardize baseline set of conversions which SHOULD be supported for Lemonade Profile, as well as parameters. Version History Release 02 Fixed a normative example to be informative. Added formal syntax for BODYPARTSTUCTURE, response text codes, and formal structure of composite GETANNOTATE values. Release 01 Corrected some grammatical mistakes. Clarified meaning of GETTANNOTATION response properties. Changed CONVERT grammar to merge media type and subtype into a single parameter instead of two parameters. Clarified that BODYSTRUCTURERESPONSE is always returned Maes Expires – August 2006 [Page 12] February 2006 for CONVERT requests. Moved transcoding parameter discussion to main body from appendix. Release 00 Initial release published in October 2005 based on draft-maes- lemonade-lconvert-00 and the comments received at the London face to face meeting end of September 2005. Acknowledgments The authors want to thank all who have contributed key insight and extensively reviewed and discussed the concepts of CONVER and its early introduction P-IMAP [P-IMAP]. In particular, this includes the authors of the LCONVERT draft (draft-maes-lemonade-lconvert-00): Rafiul Ahad – Oracle Corporation, Eugene Chiu – Oracle Corporation, Ray Cromwell – Oracle Corporation, Jia-der Day – Oracle Corporation, Vi Ha – Oracle Corporation, Wook-Hyun Jeong – Samsung Electronics Co. LTF, Chang Kuang – Oracle Corporation, Rodrigo Lima – Oracle Corporation, Stephane H. Maes – Oracle Corporation, Gustaf Rosell - Sony Ericsson, Jean Sini – Symbol Technologies, Sung-Mu Son – LG Electronics, Fan Xiaohui - CHINA MOBILE COMMUNICATIONS CORPORATION (CMCC), Zhao Lijun - CHINA MOBILE COMMUNICATIONS CORPORATION (CMCC). Authors Addresses Stephane H. Maes Oracle Corporation 500 Oracle Parkway M/S 4op634 Redwood Shores, CA 94065 USA Phone: +1-650-607-6296 Email: stephane.maes@oracle.com Ray Cromwell Oracle Corporation 500 Oracle Parkway Redwood Shores, CA 94065 USA Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information Maes Expires – August 2006 [Page 13] February 2006 on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf- ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Maes Expires – August 2006 [Page 14]