| Internet-Draft | Structured and Dynamic Email | January 2023 |
| Happel | Expires 31 July 2023 | [Page] |
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 31 July 2023.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Email can be considered a successful technology, based on its broad adoption and based on the fact dozens of email-related RFCs have been written over time [MailRFCs].¶
Despite of these various RFCs, which are dealing with a multitude of small improvements, the major inner workings of email remain widely unchanged since their inception.¶
In particular, email is still mostly a text-based medium, which requires a lot of human attention, even though more and more so called "transactional" emails are authored in an automated fashion. Accordingly, the task of dealing with large amounts of email is still a major challenge for many users.¶
Most existing attempts that have been made to improve this situation focus on particular use cases. The main point of this paper is to suggest that it might be worthwhile to consider if there exists a generalizable core underneath those use cases. Such a more streamlined approach might help to condense specifications and to increase adoption by email servers and clients.¶
Furthermore, we argue that such an approach might enable novel use cases by leveraging the specific and partially unique properties of email technology, which include:¶
Essentially, this entails that a consciously designed approach for the automated processing of formally structured information in emails could be considered a "private API" of email users. While this certainly raises security and trust concerns, the opportunities behind such unifying approach seem worthwhile discussing.¶
A large share of today's emails is of transactional nature, i.e., sent by automated agents or processes to human users. While those emails often have relatively clear semantics (such as Invoice or Reservation), the medium of those emails is still human-readable text. Users and their email software cannot leverage knowledge inside and about those emails to make their processing a more efficient and enjoyable task.¶
Using informal HTML email conventions, such emails can be considered as small websites, adapted and personalized for a certain use case. However, they are mostly a "dead end" for interaction. Due to the lack of explicit semantics in the underlying email, particular actions afforded by transactional mail (such as Approval or Rating) cannot be easily made actionable for users. Also, many interactions require to switch from email to a regular web browser, resulting in a process disruption which is coined Medienbruch in German language.¶
Several attempts have been made to improve this situation. We discuss some of them in the following.¶
Within the IETF, various RFCs have been devised to structure certain types of email interaction. Probably most notable, iMIP [RFC2447] allows users to deal with meeting workflows in a structured manner. Further examples structure interaction with message delivery notifications [RFC8098], mailinglist subscriptions [RFC8058] or specify header-based semantic indicators for certain domains [RFC6477] or Emoji-based semantics of approval/disapproval in mailinglist discussions [RFC9078].¶
In more administrative use cases, special kinds of email body parts are used for abuse reporting [RFC6650] or administrative reporting [RFC6522]. In a USENET context [RFC1036], so-called "control messages" are defined for what can be considered certain API calls, such as for creating a USENET group.¶
Looking outside the IETF, the "Email Markup" approach [EMarkup] by Google allows allows senders to integrate certain Schema.org [SchemaOrg] annotations (such as for hotel reservations or parcel delivery) in their email, such that email clients can provide customized support, including certain easy to use response actions.¶
Email markup is still in use, but has a number of limitations:¶
More recently, AMP email [AMPemail] (also initiated by Google) and Microsoft Actionable Messages [AM] have been specified. Compared to Email Markup, their focus is less on semantically annotating email content, but on more rich interaction, e.g., allowing the submission of simple forms. In addition, both approaches can include dynamic elements, by retrieving certain data, or even replacing the complete email body at reading time.¶
Both approaches however suffer from similar limitations as those already pointed out for Email Markup. There also seems to be a lack of consensus, since Microsoft Outlook is reported to have removed initial support for AMP email [OutAMP].¶
To summarize, we see a need for a vendor neutral standard for structured and dynamic email. It should enable email senders to provide structured information about the semantics of their message and allow recipients (may it be human users or their agents) for rich and structured interaction with those emails.¶
Such standard should also take into account the various RFCs specifying email semantics for specific use cases and ideally provide a more generalized framework for such use cases, which can be more consistently and easily implemented by vendors. For example, there exist various "informal" standards for certain types of email messages, that have not yet been standardized in dedicated RFCs, such as out-of-office (aka vacation notice) emails.¶
Structured email should work with existing protocols such as IMAP [RFC9051], SMTP [RFC5321], and novel ones such as JMAP [RFC8620]. Accordingly, most extensions would probably realized in the scope of the Internet Message Format [RFC5322] or in areas not yet addressed by standardization (see, e.g., Section 4.3, Section 4.5, Section 4.6, Section 4.7).¶
A major advantage of email technology is the fact, that email is backwards compatible. For example, most HTML emails are accompanied by a plain text version to be used as a fallback in case an email client cannot deal with HTML. In a similar fashion, structured email could be provided as an additional option for clients able and willing to deal with it, thus avoiding a classic "chicken/egg" problem for novel technology.¶
With respect to structured markup, an incremental and decentral approach is proposed. Instead of an authoritative fixed set of schemas, as in the case of Email Markup, users should be allows to create and extend schemas on their own.¶
This draft can and will not yet provide a full specification for the problems stated. In this section, we will single out certain aspects that might be addressed by such specifications.¶
As described before, an (optional and alternate) structured description of the semantics of email messages is supposed be in the core of this proposal. While existing approaches such as Email Markup offer different options for how to include structured data, an alternative, machine-readable presentation of the message content in addition to human readable text and HTML representations, might be the most natural approach.¶
For the sake of efficient processing and perhaps also for reasons of data privacy, certain structured data about an email might also be captured in its headers.¶
Header fields of certain message parts (such as attachments) might also be enriched by semantic annotation.¶
Email messages stored in IMAP are currently immutable. Receivers are not supposed to modify their actual content. Hence, the main means to add and store additional data on the receiving side are:¶
As of today, this already yields certain data portability problems - e.g., IMAP flags are often not considered when exporting raw email message files. Accordingly, we see a need for the standardized persistent storage of data which is added by algorithmic processing or by users and their email clients.¶
Once emails contain structured data about their content, this might create a demand for related extensions to the Sieve email filtering language [RFC5228].¶
Email lives in tight symbiosis with strongly related data such as address books, calendars, or tasks. However, besides specific use cases such as meeting organization, an integration between such data is mainly subject to non-standardized functionality of client software.¶
Notably, email also operates in a wider context such as services which send transactional emails and also applications running on the same Desktop or Mobile device as the email client.¶
Transactional emails are sent by services and organizations which are in a relation to the user. Further related data may be managed in outside applications or would be a candidate for being managed within email client software.¶
From a broader architectural perspective, email can be considered a technology particularly suited to support novel trends in data sovereignty and decentralized data storage, since by design it bridges the public Internet data space with the private data spaces of mailbox users.¶
Structured email will open new opportunities for exchanging data in all those cases. Accordingly, standards may help to enable and organize interoperability.¶
In order to allow users to send structured mails, they need to know if and which kind of structured emails a recipient can process. There might be multiple ways of conveying such information:¶
An example for the latter could be:¶
For dynamic email features, an endpoint must be available to provide updated data for an email currently read by a user. Similar, if a structured email allows for the execution of a certain action or the submission of form data, there needs to be a receiving endpoint in place. For interoperability reasons, those endpoint APIs should be standardized.¶
Structured and automated interaction certainly raises various security concerns (see Section 5).¶
While not the focus of this current draft, we assume that parts of the proposed approach can also be used on a special "trust layer". E.g., certain structured message exchanges, in combination with signing or encryption could help to establish trust necessary for further structured communication.¶
While it is clear that trust, security, and anti-spam considerations are core to the technology suggested in this document, this aspect is not discussed in depth at this stage in order to reduce complexity.¶
While existing approaches such as Email Markup or Actionable Messages enforce strict control by registration processes, we think that a more open process is better suited for the decentralized character of email.¶
Various trust mechanisms such as DKIM [RFC6376] validation, challenge-response emails, or trusted receivers in address books/white lists might be considered as a prerequisite for structured email processing.¶
This document has no IANA actions at this time.¶