IMAP Extensions Working Group B. Leiba Internet Draft IBM T.J. Watson Research Center Document: draft-ietf-imapext-list-extensions-06.txt May 2004 Expires November 2004 IMAP4 LIST Command Extensions Status of this Document This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. A revised version of this draft document will be submitted to the RFC editor as an Proposed Standard for the Internet Community. Discussion and suggestions for improvement are requested, and should be sent to ietf-imapext@imc.org. This document will expire before 31 November 2004. Distribution of this memo is unlimited. This documents obsoletes RFC 3348 and updates RFC 2193. Abstract IMAP4 has two commands for listing mailboxes: LIST and LSUB. As we have added extensions that have required specialized lists (see [MboxRefer] for an example) we have had to expand the number of list commands, since each extension must add its function to both LIST and LSUB, and these commands are not, as they are defined, extensible. If we've needed the extensions to work together, we've had to add a set of commands to mix the different options, the set increasing in size with each new extension. This document describes an extension to the base LIST command that will allow these additions to be done with mutually compatible options to the LIST command, avoiding the exponential increase in specialized list commands. 1. Conventions used in this document In examples, "C:" indicates lines sent by a client that is connected to a server. "S:" indicates lines sent by the server to the client. The words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" are used in this document as specified in RFC 2119 [Keywords]. 2. Introduction and overview The extensions to the LIST command will be accomplished by amending the syntax to allow options to be specified. The list of options will replace the several commands that are currently used to mix and match the information requested. The new syntax is backward- compatible, with no ambiguity: if the first word after the command name begins with a parenthesis, the new syntax is being used; if it does not, it's in the original syntax. By adding options to the LIST command, we are announcing the intent to phase out and eventually to deprecate the RLIST and RLSUB commands described in [MboxRefer]. We are also defining the mechanism to request extended mailbox information, such as is described in the "Child Mailbox Extension" [ChildMbox]. The base LSUB command is not deprecated by this extension; rather, this extension adds a way to obtain subscription information with more options, with those server implementations that support it. Clients that simply need a list of subscribed mailboxes, as provided by the LSUB command, SHOULD continue to use that command. This document defines an IMAP4 extension that is identified by the capability string "LISTEXT". The LISTEXT extension makes the following changes to the IMAP4 protocol, which are described in more details in sections 3 and 4 of this document: a. defines new syntax for LIST command options. b. adds LIST command options: SUBSCRIBED, REMOTE and CHILDREN c. adds new mailbox flags "\NonExistent", "\PlaceHolder", "\HasChildren" and "\HasNoChildren". 3. LIST Command Options The LIST command syntax is extended by adding a parenthesized list of command options between the command name and the reference name (see the formal syntax in section 6 for specific details). Command options will be defined in this document and in approved extension documents; each option will be enabled by a capability string (one capability may enable multiple options), and a client MUST NOT send an option for which the server has not advertised support. A server MUST respond to options it does not recognize with a NO response. This extension is identified by the capability string "LISTEXT", and support for it is a prerequisite for any future extensions that require specialized forms of the LIST command. Such extensions MUST refer to this document and MUST add their function through command options as described herein. This extension also defines extensions to the LIST response, allowing a series of extended fields at the end, a parenthesized list of tagged data (also referred to as "extended data item"). The first element of an extended field is a tag, which identifies type of the data. Tags MUST be registered with IANA, as described in section 8.5 of this document. An example of such extended set might be ((tablecloth (("fringe" "lacy")("color" "white")))(X-Sample "text")) or... ((tablecloth ("fringe" "lacy"))(X-Sample "text" "and even more text")) See the formal grammar, below, for the full syntatic details. The server MAY return data in the extended fields that was not solicited by the client. The client MUST ignore all extended fields it doesn't recognize. The options defined in this specification are SUBSCRIBED - causes the LIST command to list subscribed mailboxes, rather than the actual mailboxes. This will often be a subset of the actual mailboxes. It's also possible for this list to contain the names of mailboxes that don't exist. In any case, the list MUST include exactly those mailbox names that match the selection criteria and are subscribed to. This option is intended to supplement the LSUB command. Of particular note are the mailbox flags as returned by this option, compared with what is returned by LSUB. With the latter, the flags returned may not reflect the actual flag status on the mailbox, and the \NoSelect flag has a special meaning (it indicates that this mailbox is not, itself, subscribed, but that it has child mailboxes that are). With the SUBSCRIBED option described here, the flags are accurate and complete, and have no special meanings. "LSUB" and "LIST (SUBSCRIBED)" are, thus, not the same thing, and some servers must do significant extra work to respond to "LIST (SUBSCRIBED)". Because of this, clients SHOULD continue to use "LSUB" unless they specifically want the additional information offered by "LIST (SUBSCRIBED)". This option defines a new mailbox flag, "\NonExistent", that indicates that a mailbox is subscribed to, but does not actually exist. The "\NonExistent" flag MUST be supported and MUST be accurately computed. REMOTE - causes the LIST command to show remote mailboxes as well as local ones, as described in [MboxRefer]. This option is intended to replace the RLIST command and, in conjunction with the SUBSCRIBED option, the RLSUB command. This option is only available on servers that also support RFC 2193. CHILDREN - Requests mailbox child information as originally proposed in [ChildMbox]. See section 4, below, for details. Support for this is optional, but this option MUST be accepted by all servers (though it MAY be ignored). The LISTEXT capability also defines a new mailbox flag, "\PlaceHolder", that indicates that the designated mailbox does not meet the selection criteria of the given LIST command, but that it has one or more child mailboxes that do <>. The LSUB command indicates this condition by using the "\NoSelect" flag, but the LIST (SUBSCRIBED) command MUST NOT do that, since "\NoSelect" retains its original meaning here. Further, the "\PlaceHolder" flag is more general, in that it can be used with any extended set of selection criteria. 4. The CHILDREN Option The CHILDREN option implements the Child Mailbox Extension, originally proposed by Mike Gahrns and Raymond Cheng, of Microsoft Corporation. Most of the information in this section is taken directly from their original specification [ChildMbox]. The CHILDREN option is simply an indication that the client wants this information; a server MAY provide it even if the option is not specified, or MAY ignore the option entirely. Many IMAP4 [IMAP4] clients present to the user a hierarchical view of the mailboxes that a user has access to. Rather than initially presenting to the user the entire mailbox hierarchy, it is often preferable to show to the user a collapsed outline list of the mailbox hierarchy (particularly if there is a large number of mailboxes). The user can then expand the collapsed outline hierarchy as needed. It is common to include within the collapsed hierarchy a visual clue (such as a ''+'') to indicate that there are child mailboxes under a particular mailbox. When the visual clue is clicked the hierarchy list is expanded to show the child mailboxes. The Child Mailbox Extension provides a mechanism for a client to efficiently determine if a particular mailbox has children, without issuing a LIST "" * or a LIST "" % for each mailbox name. The Child Mailbox Extension defines two new attributes that MAY be returned within a LIST response: \HasChildren and \HasNoChildren. While these attributes MAY be returned in response to any LIST command, the CHILDREN option is provided to indicate that the client particularly wants this information. If the CHILDREN option is present, the server SHOULD return these attributes even if their computation is expensive. \HasChildren - The presence of this attribute indicates that the mailbox has child mailboxes. A server SHOULD NOT set this attribute if there are child mailboxes, and the user does not have permissions to access any of them. In this case, \HasNoChildren SHOULD be used. In many cases, however, a server may not be able to efficiently compute whether a user has access to all child mailboxes. As such a client MUST be prepared to accept the \HasChildren attribute as a hint. That is, a mailbox MAY be flagged with the \HasChildren attribute, but no child mailboxes will appear in the LIST response. \HasNoChildren - The presence of this attribute indicates that the mailbox has NO child mailboxes that are accessible to the currently authenticated user. In some instances a server that supports the Child Mailbox Extension might not be able to determine whether a mailbox has children. For example it may have difficulty determining whether there are child mailboxes when LISTing mailboxes while operating in a particular namespace. In these cases, a server MAY exclude both the \HasChildren and \HasNoChildren attributes in the LIST response. As such, a client can not make any assumptions about whether a mailbox has children based upon the absence of a single attribute. In particular, some servers may not be able to combine the SUBSCRIBED and CHILDREN options. Such servers MUST honour the SUBSCRIBED option, and they will simply ignore the CHILDREN option if both are requested. It is an error for the server to return both a \HasChildren and a \HasNoChildren attribute in a LIST response. Note: the \HasNoChildren attribute should not be confused with the IMAP4 [IMAP4] defined attribute \NoInferiors which indicates that no child mailboxes exist now and none can be created in the future. 5. Examples The first example shows the complete local hierarchy that will be used for the other examples. C: A01 LIST "" "*" S: * LIST (\Marked \NoInferiors) "/" "inbox" S: * LIST () "/" "Fruit" S: * LIST () "/" "Fruit/Apple" S: * LIST () "/" "Fruit/Banana" S: * LIST () "/" "Tofu" S: * LIST () "/" "Vegetable" S: * LIST () "/" "Vegetable/Broccoli" S: A01 OK done In the next example, we'll see the subscribed mailboxes. This is similar, but not equivalent, to . Note that the mailbox called "Fruit/Peach" is subscribed to, but does not actually exist (perhaps it was deleted while still subscribed). And the "Fruit" mailbox is not subscribed to, but it has two subscribed children. C: A02 LIST (SUBSCRIBED) "" "*" S: * LIST (\Marked \NoInferiors) "/" "inbox" S: * LIST (\PlaceHolder) "/" "Fruit" S: * LIST () "/" "Fruit/Banana" S: * LIST (\NonExistent) "/" "Fruit/Peach" S: A02 OK done The next example shows the use of the CHILDREN option. The client, without having to list the second level of hierarchy, now knows which of the top-level mailboxes have sub-mailboxes (children) and which do not. Note that it's not necessary for the server to return the \HasNoChildren flag for the inbox, because the \NoInferiors flag already implies that, and has a stronger meaning. C: A03 LIST (CHILDREN) "" "%" S: * LIST (\Marked \NoInferiors) "/" "inbox" S: * LIST (\HasChildren) "/" "Fruit" S: * LIST (\HasNoChildren) "/" "Tofu" S: * LIST (\HasChildren) "/" "Vegetable" S: A03 OK done In this example we see more mailboxes, which reside on another server to which we may obtain referrals. This is similar to the command . We also see the mixing of two options. Note that in the case of the remote mailboxes, the server might or might not be able to include CHILDREN information; it includes it if it can, and omits it if it can't. C: A04 LIST (REMOTE CHILDREN) "" "%" S: * LIST (\Marked \NoInferiors) "/" "inbox" S: * LIST (\HasChildren) "/" "Fruit" S: * LIST (\HasNoChildren) "/" "Tofu" S: * LIST (\HasChildren) "/" "Vegetable" S: * LIST () "/" "Bread" S: * LIST (\HasChildren) "/" "Meat" S: A04 OK done 6. Formal Syntax The following syntax specification uses the augmented Backus-Naur Form (BNF) as described in [ABNF]. Terms not defined here are taken from [IMAP4]. child-mbox-flag = "\HasChildren" / "\HasNoChildren" ; flags for Child Mailbox Extension, at most one ; possible per LIST response list = "LIST" [SP list-options] SP mailbox SP list-mailbox list-options = "(" [option *(SP option)] ")" mailbox-list = "(" [mbx-list-flags] ")" SP (DQUOTE QUOTED-CHAR DQUOTE / nil) SP mailbox [SP mbox-list-extended] mbox-list-extended = "(" [mbox-list-extended-item *(SP mbox-list-extended-item)] ")" mbox-list-extended-item = "(" mbox-list-extended-item-data ")" mbox-list-extended-item-data = mbox-list-extended-item-tag SP nstring-list mbox-list-extended-item-tag = vendor-tag / standard-tag ; A tag registration template is described in section ; 8.5 of this document. vendor-tag = "V-" atom ; a vendor specific tag for extended list data standard-tag = atom ; a tag for extended list data defined in a Standard ; Track or Experimental RFC. nstring-list = nstring / "(" [nstring-list *(SP nstring-list)] ")" ;; a recursive list definition mbox-list-oflag = child-mbox-flag / "\NonExistent" / "\PlaceHolder" option = "SUBSCRIBED" / "CHILDREN" / "REMOTE" / option-extension ; An option registration template is described in section ; 8.3 of this document. option-extension = option-vendor / option-public option-vendor = "V-" atom ; a vendor specific option option-public = atom ; an option defined in a Standard Track or ; Experimental RFC 7. Security Considerations This document describes syntactic changes to the specification of the IMAP4 commands LIST, LSUB, RLIST, and RLSUB, and the modified LIST command has the same security considerations as those commands. They are described in [IMAP4] and [MboxRefer]. The Child Mailbox Extension provides a client a more efficient means of determining whether a particular mailbox has children. If a mailbox has children, but the currently authenticated user does not have access to any of them, the server SHOULD respond with a \HasNoChildren attribute. In many cases, however, a server may not be able to efficiently compute whether a user has access to all child mailboxes. If such a server responds with a \HasChildren attribute, when in fact the currently authenticated user does not have access to any child mailboxes, potentially more information is conveyed about the mailbox than intended. In most situations this will not be a security concern, because if information regarding whether a mailbox has children is considered sensitive, a user would not be granted access to that mailbox in the first place. 8. IANA Considerations 8.1. Guidelines for IANA It is requested that IANA creates two new registries for LISTEXT options and LISTEXT extended response data. The templates and the initial registrations are detailed below. 8.2. Registration procedure and Change control Registration of a LISTEXT option is done by filling in the template in section 8.3 and sending it via electronic mail to . Registration of a LISTEXT extended data item is done by filling in the template in section 8.5 and sending it via electronic mail to . IANA has the right to reject obviously bogus registrations, but will perform no review of claims made in the registration form. A LISTEXT option/extended data item name that starts with "V-" is reserved for vendor specific options/extended data items. All options, whether they are vendor specific or global, should be registered with IANA. If a LISTEXT extended data item is returned as a result of requesting a particular LISTEXT option, the name of the option SHOULD be used as the name of the LISTEXT extended data item. LISTEXT option/extended data item names are case insensitive. While the registration procedures do not require it, authors of LISTEXT options/extended data items are encouraged to seek community review and comment whenever that is feasible. Authors may seek community review by posting a specification of their proposed mechanism as an Internet- Draft. LISTEXT options/extended data items intended for widespread use should be standardized through the normal IETF process, when appropriate. Comments on registered LISTEXT options/extended response data should first be sent to the "owner" of the mechanism and/or to the IMAPEXT WG mailing list. Submitters of comments may, after a reasonable attempt to contact the owner, request IANA to attach their comment to the registration itself. If IANA approves of this, the comment will be made accessible in conjunction with the registration LISTEXT options/ extended response data itself. Once a LISTEXT registration has been published by IANA, the author may request a change to its definition. The change request follows the same procedure as the registration request. The owner of a LISTEXT registration may pass responsibility for the registered option/extended data item to another person or agency by informing IANA; this can be done without discussion or review. The IESG may reassign responsibility for a LISTEXT option/extended data item. The most common case of this will be to enable changes to be made to mechanisms where the author of the registration has died, moved out of contact or is otherwise unable to make changes that are important to the community. LISTEXT registrations may not be deleted; mechanisms which are no longer believed appropriate for use can be declared OBSOLETE by a change to their "intended use" field; such LISTEXT options/extended data items will be clearly marked in the lists published by IANA. The IESG is considered to be the owner of all LISTEXT options/extended data items which are on the IETF standards track. 8.3. Registration template for LISTEXT options To: iana@iana.org Subject: Registration of LISTEXT option X LISTEXT option name: LISTEXT option description: Published specification (optional, recommended): Security considerations: Intended usage: (One of COMMON, LIMITED USE or OBSOLETE) Person & email address to contact for further information: Owner/Change controller: (Any other information that the author deems interesting may be added below this line.) 8.4. Initial LISTEXT option registrations It is requested that the LISTEXT option registry is being populated with the following entries: 1) To: iana@iana.org Subject: Registration of LISTEXT option SUBSCRIBED LISTEXT option name: SUBSCRIBED LISTEXT option description: Causes the LIST command to list subscribed mailboxes, rather than the actual mailboxes. Published specification : this RFC, section 3. Security considerations: this RFC, section 7. Intended usage: COMMON Person & email address to contact for further information: Alexey Melnikov Owner/Change controller: IESG 2) To: iana@iana.org Subject: Registration of LISTEXT option REMOTE LISTEXT option name: REMOTE LISTEXT option description: causes the LIST command to return remote mailboxes as well as local ones, as described in RFC 2193. Published specification : this RFC, section 3. Security considerations: this RFC, section 7. Intended usage: COMMON Person & email address to contact for further information: Alexey Melnikov Owner/Change controller: IESG 3) To: iana@iana.org Subject: Registration of LISTEXT option CHILDREN LISTEXT option name: CHILDREN LISTEXT option description: Requests mailbox child information. Published specification : this RFC, sections 3 and 4. Published specification : this RFC Security considerations: this RFC, section 7. Intended usage: COMMON Person & email address to contact for further information: Alexey Melnikov Owner/Change controller: IESG 8.5. Registration template for LISTEXT extended data item To: iana@iana.org Subject: Registration of LISTEXT extended data item X LISTEXT extended data item tag: LISTEXT extended data item description: Which LISTEXT option(s) causes this extended data item to be returned (if any): Published specification (optional, recommended): Security considerations: Intended usage: (One of COMMON, LIMITED USE or OBSOLETE) Person & email address to contact for further information: Owner/Change controller: (Any other information that the author deems interesting may be added below this line.) 9. References 9.1. Normative References [Keywords]; Bradner, S.; "Key words for use in RFCs to Indicate Requirement Levels"; RFC 2119; Harvard University; March 1997. [ABNF]; Crocker, D., and Overell, P. "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997. [IMAP4] Crispin, M., "Internet Message Access Protocol - Version 4rev1", RFC 3501, University of Washington, March 2003. [MboxRefer]; Gahrns, M.; "IMAP4 Mailbox Referrals"; RFC 2193; Microsoft Corporation; September 1997. [ChildMbox]; Gahrns, M. & Cheng, R.; "IMAP4 Child Mailbox Extension"; RFC 3348; Microsoft Corporation; July 2002. 10. Acknowledgements Mike Gahrns and Raymond Cheng of Microsoft Corporation originally devised the Child Mailbox Extension and proposed it in 1997; the idea, as well as most of the text in section 4, is theirs. This document is the result of discussions on the IMAP4 mailing list and is meant to reflect consensus of this group. In particular, Mark Crispin, Cyrus Daboo, Timo Sirainen, Ken Murchison, Alexey Melnikov, Rob Siemborski, Steve Hole, Arnt Gulbrandsen, Larry Greenfield, Phlip Guenther and Pete Maclean were active participants in this discussion or made suggestions to this document. 11. Author's Address Barry Leiba IBM T.J. Watson Research Center 30 Saw Mill River Road Hawthorne, NY 10532 Phone: 1-914-784-7941 Email: leiba@watson.ibm.com 12. Full Copyright Statement Copyright (C) The Internet Society 2004. All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. 13. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.