INTERNET-DRAFT Hadmut Danisch Category: Experimental Apr 2003 Expires: Oct 15, 2003 An advanced Mail Transfer Protocol draft-danisch-email-mtp-00.txt Status of this Memo This document is an Internet-Draft and is subject to all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Abstract This memo suggests an advanced mail transfer protocol in order to find a modern, flexible, extensible, efficient, and robust successor of SMTP. The presented protocol uses a HTTP-ish syntax. It is designed to give the receiver of a message finer control over accepting or rejecting the message, and to allow negotiation of transport details. The memo is still in a very early state and intended to initiate a discussion, not to present a finished protocol. The protocol can be understood as a workbench for future mail extensions. Hadmut Danisch Experimental [Page 1] INTERNET-DRAFT MTP Apr 2003 Table of Contents 1. Status of the draft . . . . . . . . . . . . . . . . . . . . . . 3 2. Protocol Design Considerations . . . . . . . . . . . . . . . . . 3 2.1. Shortcomings of SMTP . . . . . . . . . . . . . . . . . . . 3 2.2. Individual Extensions . . . . . . . . . . . . . . . . . . 3 2.3. Simple but powerful MTA Structure . . . . . . . . . . . . 3 2.4. A HTTP-ish approach . . . . . . . . . . . . . . . . . . . 4 3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. The MESSAGE command . . . . . . . . . . . . . . . . . . . 4 3.2. The CONTINUE command . . . . . . . . . . . . . . . . . . . 5 3.3. The BODY command . . . . . . . . . . . . . . . . . . . . . 5 3.4. The STARTTLS command . . . . . . . . . . . . . . . . . . . 6 3.4.1 STARTTLS before MESSAGE . . . . . . . . . . . . . . 6 3.4.2 STARTTLS after MESSAGE . . . . . . . . . . . . . . 6 3.4.3 STARTTLS after STARTTLS . . . . . . . . . . . . . . 6 3.4.4 STARTTLS after reply . . . . . . . . . . . . . . . 7 3.5. The Status codes . . . . . . . . . . . . . . . . . . . . . 7 3.5.1 Positive 2xy . . . . . . . . . . . . . . . . . . . 7 3.5.2 Intermediate 3xy . . . . . . . . . . . . . . . . . 7 3.5.3 Transient Negative 4xy . . . . . . . . . . . . . . 7 3.5.4 Permanent Negative 5xy . . . . . . . . . . . . . . 8 3.5.5 Status Follows 6xy . . . . . . . . . . . . . . . . 8 3.6. Envelope Address Informations . . . . . . . . . . . . . . 8 4. DNS Configuration . . . . . . . . . . . . . . . . . . . . . . . 9 5. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.1. Transport related security . . . . . . . . . . . . . . . . 9 5.2. Message related security . . . . . . . . . . . . . . . . . 10 6. Example Extensions . . . . . . . . . . . . . . . . . . . . . . . 10 6.1. SASL Authentication . . . . . . . . . . . . . . . . . . . 10 6.2. Pre-Transfer digital signature verification . . . . . . . 10 6.3. Mail path encryption enforcement . . . . . . . . . . . . . 11 6.4. Header based message rejection . . . . . . . . . . . . . . 11 6.4.1 Cookie/Reference engine . . . . . . . . . . . . . . 11 6.4.2 Content Type blocking . . . . . . . . . . . . . . . 11 6.5. Payment . . . . . . . . . . . . . . . . . . . . . . . . . 12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Hadmut Danisch Experimental [Page 2] INTERNET-DRAFT MTP Apr 2003 1. Status of the draft This draft is currently in the first, rough stage of protocol design. It does not provide a detailed grammer or a BNF specification. It is a proposal which presents the basic considerations and is supposed to start further discussion. 2. Protocol Design Considerations 2.1. Shortcomings of SMTP While the Simple Mail Transfer Protocol[1, 2] has proved to be robust and effective in performing the task of transporting e-mail, it doesn't fulfill today's requirements, such as simple extensibility, better authentication and authorization support, and better support for plausibility checks (e.g. to block Spam and fraud). After all, SMTP is more than 20 years old, whereas modern requirements developed within the last 5-10 years. The large number of existing extensions to the syntax and semantics of SMTP show, that pure SMTP doesn't fulfill these requirements and that is too inflexible to be extended without modification of its syntax. Currently a number of efforts exist to further extend the syntax of SMTP. From a protocol designer's point of view, those extensions don't solve the problem, they just apply additional patchwork. A protocol should allow functional extensions without the need to be redefined and syntactically extended. 2.2. Individual Extensions Users and organizations with individual requirements should be able to simply implement extensions without changing the protocol itself or modifying the MTA software. Again, the protocol must be suitable to be extended through plugins and scripts. 2.3. Simple but powerful MTA Structure A transfer protocol should allow to design the client and server software simple, orthogonal, and extensible. Instead of packing as much complicated functions as possible into a monolithic message transfer agent (MTA), which needs to be modified for every single protocol extension, the MTA should be a simple skeleton, which can be functionally extended through plugins and skripts, similar to HTTP web servers. The protocol was designed with the structure of the well known Apache HTTP server [3] in mind. Each request has to go through several steps of processing. For each step a program hook exist which allows to extend the functionality through plugins. It is even possible to extend the server functionality Hadmut Danisch Experimental [Page 3] INTERNET-DRAFT MTP Apr 2003 with script languages, making the server extremely flexible and highly extensible without changing the transport protocol itself. Some of today's MTAs try to have something similar (e.g. the milter interface of sendmail [4] ), without reaching the functionality due to the limitations of SMTP. 2.4. A HTTP-ish approach The Hypertext Transfer Protocol [5] is a good example for a simple, robust, flexible, and extensible protocol. Therefore, the proposed protocol is based on the HTTP principle. Since the receiving MTA must be able to reject an e-mail before transmission of the message body, the message transfer will be split into two or more parts. 3. Protocol Overview The protocol is based on a stream connection, which means TCP in the context of the Internet. The sending MTA (client) opens a TCP connection to the receiving MTA (server). In contrast to SMTP, the receiving MTA does not issue a greeting message. Similar to HTTP, the client sends a command, which is answered by the server. The connection is kept open to allow an unlimited number of commands. When the client doesn't wish to issue further commands, it may quietly terminate the connection. Obviously, the server may terminate the connection under certain conditions, such as timeout, protocol or policy violation, attack detection, server failure. There are only four commands, which will be described in detail below: MESSAGE CONTINUE BODY STARTTLS Each of these commands will be answered with a HTTP style reply by the server, i. e. a three digit reply code and a human readable reply message, followed by a additional lines of informations and terminated by an empty line. E-mail messages still comply with the Internet Message Format as defined in [6]. 3.1. The MESSAGE command The MESSAGE command is the initial request to accept an e-mail Hadmut Danisch Experimental [Page 4] INTERNET-DRAFT MTP Apr 2003 message. The MESSAGE command consists of a line of the form MESSAGE protocol/version and is followed by the complete message header (or only parts of it) and further parameters in a HTTP style. The command is terminated by an empty line. The command should usually include at least the most important parts of the header, but is not necessarily required to do so. It may leave the transfer of any header line for subsequent CONTINUE commands (see below). It is the receiver's choice whether to accept the message with the header given so far, or to request more informations. From the protocol point of view, there is not even the need to have a sender or receiver address at all. It's up to the receiver to take the message or not. When the command has been issued, the server sends a HTTP-style reply, which consists of a reply code line and further parameters. The reply code line consists of a Status Code and a textual status description. The reply may contain further parameters. The reply informs the client about whether (and how) the server is willing to accept the message. A MESSAGE command with a positive reply can be followed by a BODY command (see below) to complete the message transfer. If a MESSAGE command is issued immediately after a preceding MESSAGE command, the preceding one is cancelled. A MESSAGE command always starts a fresh message transfer. 3.2. The CONTINUE command The CONTINUE command is basically of the same form as the MESSAGE command. It's purpose is to add parameters to a former MESSAGE command, when the server required further informations through a intermediate reply (see below), e.g. sender authentication, without retransmitting the whole header. 3.3. The BODY command After a MESSAGE or CONTINUE command, which was positively acknowledged, the client may issue one BODY command. The BODY command is followed by the exact number of octets as given in the content size header line. This is the message body belonging to the header given to the last MESSAGE command. When the server has issued a positive reply, message transfer is completed and the server has taken over responsibility for the message, similar to Hadmut Danisch Experimental [Page 5] INTERNET-DRAFT MTP Apr 2003 SMTP. Sending a BODY command without having received a positive 2xy reply to the preceding MESSAGE or CONTINUE command is a protocol violation. The server may (and should) immediately terminate the connection then. 3.4. The STARTTLS command The STARTTLS command starts the transport layer security protocol as defined in [7], similar to the starttls command defined in SMTP. It may be issued before (typically when requested by the client) or after (typically when requested by the server through an intermediate code) a MESSAGE or CONTINUE command. A STARTTLS command might cause an intermediate reply if the receiving MTA has access to more than one TLS key/certificate - e.g. if it is providing mail services for several domains - and the information given so far is ambiguous. A STARTTLS command may even be given after TLS was already started to restart TLS with a different sender and/or receiver key. Details are still to be discussed. However, several scenarios can be given as an example: 3.4.1. STARTTLS before MESSAGE If the STARTTLS command was issued before the MESSAGE command, the receiving MTA has no idea about the message recipient's identity, and thus can't choose the appropriate TLS key. The receiving MTA will have to use a default key, which might be accepted by the sending MTA or not. 3.4.2. STARTTLS after MESSAGE The sending MTA might issue a first MESSAGE command before STARTTLS to provide a minimum of information to allow the receiving MTA to choose the appropriate key. A first MESSAGE command might provide the recipient's envelope address only, then a STARTTLS could initiate the TLS encryption. Once the connection is encrypted, the sender could provide the remaining parts of the header through a CONTINUE command. To avoid revealing too much of the recipient's identity, instead of the full envelope address, the first MESSAGE command could contain only a TLS-To: header line which provides the domain part only, thus allowing the receivers to choose the appropriate key and authenticate before. 3.4.3. STARTTLS after STARTTLS The sending MTA can issue a STARTTLS command even after an earlier Hadmut Danisch Experimental [Page 6] INTERNET-DRAFT MTP Apr 2003 STARTTLS command in order to change the key. This could be the case when the receiver started with a default key, or if another message is to be transmitted to the same receiver, but requires different authentication (i.e. is a message to a different recipient). 3.4.4. STARTTLS after reply The receiving MTA could answer a MESSAGE command with a negative or intermediate reply if the currently used TLS key is not accepted as authorized for the requested message. As a reaction, the client might decide to start TLS or to choose a different key (if available). 3.5. The Status codes The Status codes are not yet defined in detail. The following sections describe the principle, which is basically similar to the reply code scheme known from SMTP and HTTP. 3.5.1. Positive 2xy A positive reply to a MESSAGE or CONTINUE command means, the server is willing to accept the message. A reply to a BODY command means that the message has be successfully received and accepted. A positive reply to a STARTTLS command means, that the server is willing to start TLS. 3.5.2. Intermediate 3xy An intermediate reply may be given to a MESSAGE or CONTINUE command and signals, that the server is not yet willing to accept the message, but that the client is expected to solve the problem or complete the request immediately. Such problems could be: - server requires authentication - server requires encryption - server requires additional information - anything which needs to be done for any extension 3.5.3. Transient Negative 4xy A transient negative reply requests the client to retransmit the message again, e.g. with different authentication, at a later time, to a different server. Hadmut Danisch Experimental [Page 7] INTERNET-DRAFT MTP Apr 2003 The receiver could/should issue further informations about the reason and the delay, e.g. a delay period or a date when the receiving MTA is expected to accept the message. It could also contain a redirection to another MTA or a different recipients address. It could ask the sending MTA to refresh its DNS informations before expiry. If the receiving MTA is in doubt about the authenticity of the sender's address, it can automatically send a cookie (i. e. a random number) to the given address and ask the sending MTA to include that cookie in the message header. That kind of challenge-response scheme is well known from mailing list processors and could be fully automated. 3.5.4. Permanent Negative 5xy A permanent negative reply informs the client that the message was finally rejected, e.g. virus detection, user unknown, etc. 3.5.5. Status Follows 6xy There is a design problem in replying to message delivery: If the message is delivered to more than one recipient, how can the receiving MTA reply with different Status codes for these recipients? For example, delivery to one recipient might be successful, while a second recipient doesn't exist or doesn't accept that particular content. Should the MTA confirm reception or issue an error code? As a preliminary proposal, the MTA could issue a 6xy code saying that details follow for every particular recipient. Then, for every single recipient a status line has to be issued, e.g. 600 Status Follows Status: 2xy Recipient ok Status: 3xy Sender authentication requested Stauts: 4xy Recipient redirect Stauts: 5xy No such user 3.6. Envelope Address Informations The presented protocol doesn't eliminate the need to distinguish between the sender and receiver addresses given in the message header and those actually used for transmission. The reason is simple: If a message is sent to two persons A and B, the header of the message must contain a To: line with both addresses, but each persons MTA must deliver the message only to one person. In contrast to SMTP, the protocol does not contain a separate transmission of envelope addresses. Instead, messages are required Hadmut Danisch Experimental [Page 8] INTERNET-DRAFT MTP Apr 2003 to contain additional header lines as Deliver-From: Deliver-To: which are treated exactly like the envelope addresses, i.e. they are modified and removed by the MTA (e.g. to keep Bcc: receivers secret). Since the receiver can base its decision to accept or reject the message on the full message header - which might contain further informations or authentication and authorization details, either initially presented by the sender or requested by the receiver - instead of only on the envelope, as with SMTP, the receiving MTA - and even the sending one - can achieve a much finer security policy. 4. DNS Configuration Instead of SMTP's MX records, SRV records as defined in[8] are used to find the receiver for a given e-mail address. 5. Security Except for the STARTTLS command, the protocol itself is not designed to provide security mechanisms. Some security mechanisms, e.g. against unsolicited messages, are still under development, so they should not be part of the protocol. Instead, they should be seen as extensions and could be implemented as plugins (see below for examples). 5.1. Transport related security Transport related security covers the stream connection between two communicating MTAs only. This might include verifications outside the protocol, e.g. whether the MTA is willing to accept mail for the given receiver from the sender's IP address, or content security related checks (virus filters etc.). It could also cover explicit authentication like defined in [9] , which would be implemented through an intermediate request containing challenges as parameters. The client would authenticate through parameters given to a CONTINUE command. The client may start strong encryption and authentication simply by issuing the STARTTLS command. The server might require encryption through an intermediate reply after a MESSAGE command, giving the client a list of mechanisms the server is willing to accept. Hadmut Danisch Experimental [Page 9] INTERNET-DRAFT MTP Apr 2003 5.2. Message related security Message related security covers the security between the effective sender and receiver, and therefore is beyond the scope of a Transfer Protocol. Usually, messages are encrypted or signed by mail user agents conforming to S/MIME or PGP standards and transported like plain messages. However, since the MESSAGE command is followed by the complete header of the message, the receiving MTA will be informed about the MIME type of the message before transmission of the body. It might be the receivers policy to reject messages which's MIME type doesn't indicate an encrypted or signed message. 6. Example Extensions This section just gives some simple (and not necessarily serious) examples of possible extensions, just to give an idea of what is meant. 6.1. SASL Authentication As with today's ESMTP protocol, the sending MTA could be requested to authenticate through SASL. This could be implemented as an extension/plugin. After the sending MTA has transmitted the minimum header information (i.e. envelope addresses), the receiver can decide whether authentication is required. If so, an Intermediate reply code can be issued to request information exchange wrapped in parameter lines of CONTINUE commands and replies. 6.2. Pre-Transfer digital signature verification To allow the receiving MTA to perform a first check of authenticity before receiving the (possibly large) message body, the header could contain a digital signature covering a minimum set of header lines, e.g. From: To: Subject: Date: Message-ID: Content-Type: Content-Length: Content-Digest: where Content-Digest is a new Header type which simply contains a cryptographic message digest (such as MD5 or SHA1) of the message body. After receiving the header of the message, the receiving MTA Hadmut Danisch Experimental [Page 10] INTERNET-DRAFT MTP Apr 2003 could check the authenticity of the header. If the signature verifies, the MTA allows transmission of the body and immediately verifies the given Content-Digest. Since an eavesdropper could record those header lines and try to send another message with the same header lines and same signature (replay attack), the receiving MTA could accept only messages not older than a certain period of time (e.g. 1 month) and keep score of all Message-IDs (and Message-Digests) successfully received within that time interval. 6.3. Mail path encryption enforcement The message header might contain a header line (inserted by the message author or the MUA) to instruct all MTAs on the mail path to transmit the mail through TLS encrypted connections only. If this is implemented as an extension, the MTA could also be required to transmit the message to MTA's oonly which support that extension. 6.4. Header based message rejection In contrast to SMTP, the receiving MTA has access to the full message header and can request further informations - cookies, passwords, TLS authentication - before receiving the possibly large message body. So the decision can be based on a wide range of criteria, e.g. the message transport history (Received: entries) or the content type of the message. 6.4.1. Cookie/Reference engine In order to distinguish solicited mails from unsolicited ones, it could be checked whether an incoming message has a References: header line refering directly to another message recently sent. If so, it can be considered as beeing a direct reply. Another way to track back messages or to limit reachability of mail addresses is to require the message to contain headerlines presenting cookies, similar to HTTP requests. Such a cookie could have been set through a another, earlier message containing a "Set- Cookie" header line (e.g. a cookie valid for mail between two certain addresses). It could be limited to a certain time interval (e.g. if presented in a News article or on a web page). 6.4.2. Content Type blocking The receiving MTA could request the sending MTA to present a complete list of all Content Types of attachments before transmitting a (possibly large) body. The MTA could then reject the Hadmut Danisch Experimental [Page 11] INTERNET-DRAFT MTP Apr 2003 message if it contains a message type the recipient doesn't want to have (e.g. "5xy Don't send me XYZ documents of 10 MByte, I don't have the XYZ software to read them..." or "5yy This company doesn't accept XYZ documents for security reasons."). 6.5. Payment It could be possible to charge the sender or the receiver of a message for sending/accepting a message and to ask to perform some digital payment protocol as part of the message transfer. References 1. J. Postel, "Simple Mail Transfer Protocol," RFC 821 (August 1982). 2. J. Klensin, "Simple Mail Transfer Protocol," RFC 2821 (April 2001). 3. Apache HTTP Server. http://www.apache.org. 4. Sendmail SMTP MTA. http://www.sendmail.org. 5. T. Berners-Lee and others, "Hypertext Transfer Protocol HTTP/1.1," RFC 2616 (June 1999). 6. P. Resnick, "Internet Message Format," RFC 2822 (April 2001). 7. T. Dierks, C. Allen, "The TLS Protocol," RFC 2246 (January 1999). 8. A. Gulbrandsen, P. Vixie, L. Esibov, "A DNS RR for specifying the location of services (DNS SRV)," RFC 2782 (February 2000). 9. J. Myers, "Simple Authentication and Security Layer (SASL)," RFC 2222 (October 1997). Author's Address Hadmut Danisch Tennesseeallee 58 76149 Karlsruhe Germany Phone: ++49-721-843004 E-Mail: rfc@danisch.de Hadmut Danisch Experimental [Page 12] INTERNET-DRAFT MTP Apr 2003 Comments Please send comments to rfc@danisch.de. Expiry This drafts expires on Oct 15, 2003 Hadmut Danisch Experimental [Page 13]