openpgp D. Gillmor Internet-Draft ACLU Intended status: Informational May 22, 2019 Expires: November 23, 2019 Common PGP/MIME Message Mangling draft-dkg-openpgp-pgpmime-message-mangling-00 Abstract An e-mail message with OpenPGP encryption and/or signature has standard form. However, some mail transport agents damage those forms in transit. A user-friendly mail user agent that encounters such a mangled message may wish to try to process it anyway, despite the message's non-standard structure. This document aims to be a sort of bestiary of common message mangling patterns, along with guidance for implementers for how to cope with messages structured in these common ways. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 23, 2019. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Gillmor Expires November 23, 2019 [Page 1] Internet-Draft Common PGP/MIME Message Mangling May 2019 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 2. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 3 3. Cryptographic Properties . . . . . . . . . . . . . . . . . . 4 4. Encrypted Messages . . . . . . . . . . . . . . . . . . . . . 4 4.1. "Mixed Up" Encryption . . . . . . . . . . . . . . . . . . 4 4.1.1. Detection of "Mixed Up" Encryption . . . . . . . . . 5 4.1.2. Repair of "Mixed Up" Encryption . . . . . . . . . . . 5 5. Clearsigned Messages . . . . . . . . . . . . . . . . . . . . 5 5.1. Extra MIME footer . . . . . . . . . . . . . . . . . . . . 6 5.2. Content-Re-Encoding . . . . . . . . . . . . . . . . . . . 6 6. Performance Considerations . . . . . . . . . . . . . . . . . 6 7. Security Considerations . . . . . . . . . . . . . . . . . . . 6 7.1. Only Repair for Cryptographic Improvement . . . . . . . . 6 7.2. Avoiding Another E-Fail . . . . . . . . . . . . . . . . . 6 7.3. Do Not Attempt Repair Inside End-to-End Encryption . . . 7 7.4. Interactions with DKIM Verification . . . . . . . . . . . 7 7.5. "Helpful" MTA mangling . . . . . . . . . . . . . . . . . 7 8. Privacy Considerations . . . . . . . . . . . . . . . . . . . 7 9. User Considerations . . . . . . . . . . . . . . . . . . . . . 8 9.1. Indications of Mangling Can Be Confusing . . . . . . . . 8 9.2. Repair Indicators . . . . . . . . . . . . . . . . . . . . 8 9.3. Undoing Repair . . . . . . . . . . . . . . . . . . . . . 8 9.4. Attempting Decryption During Repair . . . . . . . . . . . 8 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 11. Document Considerations . . . . . . . . . . . . . . . . . . . 8 12. Example Messages . . . . . . . . . . . . . . . . . . . . . . 9 12.1. Example Input Encrypted Message . . . . . . . . . . . . 9 12.2. Input Message Mangled by "Mixed Up" . . . . . . . . . . 9 13. Example Implemenations . . . . . . . . . . . . . . . . . . . 10 13.1. Example: Detecting "Mixed Up" Encryption . . . . . . . . 10 13.2. Example: Repairing "Mixed Up" Encryption . . . . . . . . 11 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 14.1. Normative References . . . . . . . . . . . . . . . . . . 11 14.2. Informative References . . . . . . . . . . . . . . . . . 12 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 Gillmor Expires November 23, 2019 [Page 2] Internet-Draft Common PGP/MIME Message Mangling May 2019 1. Introduction 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 1.2. Terminology o "Mail User Agent" (or "MUA") refers to a program designed to send and receive e-mail. Common examples include Thunderbird, Outlook, and Mail.app. Webmail implementations, like Roundcube or the Gmail interface can also be considered a MUA. o "Mail Transport Agent" (or "MTA") refers to mail-handling software within the network. Common examples include Postfix and Exchange. o "PGP/MIME" is the message structure for OpenPGP protections described in [RFC3156]. o "encrypted message" is used to mean any message where the outermost PGP/MIME cryptographic feature of the message itself is encryption. Some encrypted messages might be signed, but the inner signature cannot typically be mangled by an "MTA" that can't decrypt. o "clearsigned message" is used here to refer specifically to messages that are not encrypted, but are signed with a PGP/MIME signature. o "signed message" is used to refer to a message that has a valid OpenPGP cryptographic signature applied by the message sender, whether the message is encrypted or not. 2. Problem Statement Some MTAs modify message headers, bodies, and structure in transit. Some of those modifications ("manglings") can transform a message in such a way that the end-to-end cryptographic properties of the message are lost. Clearsigned messages can become unverifiable, and encrypted messages can become impossible to decrypt. MUAs that receive these mangled messages are unable to fulfil their users goals because of damage done to the message by the mangling MTA. Gillmor Expires November 23, 2019 [Page 3] Internet-Draft Common PGP/MIME Message Mangling May 2019 In some cases, those manglings can take a regular enough form that a clever MUA can "repair" them, restoring the cryptographic properties of the message for the user. This document aims to collect common manglings, and document how an MUA can detect them and (if possible) repair them safely for the user. 3. Cryptographic Properties TODO: describe message-wide cryptographic envelope vs. cryptographic payload and their relevance to this draft. TODO: we're assuming a single layer of cryptographic protection. These steps get more complicated if there are multiple nested layers (encrypted and then signed? signed and then encrypted? triple- wrapped?). Should this document address that situation? 4. Encrypted Messages This section aims to collect examples of repairable mangling known to be performed by some currently-active MTA on encrypted messages. Each section offers a brief description of the mangling, a way to detect it, and a proposed method of repair. 4.1. "Mixed Up" Encryption One common mangling takes an incoming multipart/encrypted e-mail and transforms it into a multipart/mixed e-mail, shuffling body parts at the same time. An encrypted message typically has this clean structure C (see example in Section 12.1): C multipart/encrypted C.1 application/pgp-encrypted C.2 application/octet-stream But after processing, it has mangled structure M (see example in Section 12.2): M multipart/mixed M.1 text/plain (0 bytes) M.2 application/pgp-encrypted M.3 application/octet-stream Some versions of Microsoft Exchange are known to do this. Gillmor Expires November 23, 2019 [Page 4] Internet-Draft Common PGP/MIME Message Mangling May 2019 4.1.1. Detection of "Mixed Up" Encryption Check that all of the following conditions hold: o The message itself (M) is Content-Type: multipart/mixed. o The message has exactly three MIME subparts, all direct children of the message itself. o The first part (M.1) is Content-Type: text/plain, and has 0 bytes. o The second part (M.2) is Content-Type: application/pgp-encrypted, and contains (after removing any Content-Transfer-Encoding) exactly the string "Version: 1" o The third part (M.3) is Content-Type: application/octet-stream. o After removing any Content-Transfer-Encoding, the decoded third part (M.3) is an ASCII-armored OpenPGP block of type "PGP MESSAGE" (see see section 6.2 of [RFC4880]) Please see Section 13.1 for an example implementation. 4.1.2. Repair of "Mixed Up" Encryption If the detection is successful, repair can be attempted with the following steps: o Convert the message's Content-Type: value to multipart/encrypted, while leaving any other parameters untouched. o Add (or reset) the parameter protocol=application/pgp-encrypted to the Content-Type: header. o Drop the first sub-part (M.1). Please see Section 13.2 for an example implementation. 5. Clearsigned Messages This section aims to collect examples of repairable mangling known to be performed by some currently-active MTA on clearsigned messages. Each section offers a brief description of the mangling, a way to detect it, and a proposed method of repair. Gillmor Expires November 23, 2019 [Page 5] Internet-Draft Common PGP/MIME Message Mangling May 2019 5.1. Extra MIME footer TODO: describe the mailman case 5.2. Content-Re-Encoding TODO: despite the content of multipart/signed parts being ostensibly invariant (see page 4 of [RFC1847]), some MTAs mangle them. Describe some reversible manglings, like gratuitous application of Content- Transfer-Encoding. 6. Performance Considerations Trying multiple repair techniques can be expensive. A resource- constrained MUA may want to limit itself to detection of possible mangling, rather than trying every possible repair it knows of and indicating 7. Security Considerations There are several ways that attempting repair on a mangled message can go dangerously wrong. 7.1. Only Repair for Cryptographic Improvement If a message repair produces a seemingly-better-structured message, but the repaired message still cannot be decrypted or verified, then the MUA should abandon the repair and instead render the message in its original form. Additional message modification that does not provide a cryptographic advantage over the received form of the message offers no benefit to the user. TODO: is a decryptable, unsigned message "better" than a verifiably signed message? What is "an improvement" isn't necessarily clear here. 7.2. Avoiding Another E-Fail Promiscuous message repair may inject new vulnerabilities, or modify a message beyond recognition. An MUA attempting message repair should use caution to avoid re-combining potentially malicious material with material that has better cryptographic guarantees. In particular, a responsible MUA should take care not to claim cryptographic protections over material that does not have them. Gillmor Expires November 23, 2019 [Page 6] Internet-Draft Common PGP/MIME Message Mangling May 2019 7.3. Do Not Attempt Repair Inside End-to-End Encryption If a message or part of a message arrived end-to-end encrypted, none of the recommendations in this draft should be read as encouraging their application to any cleartext material within that end-to-end encryption. These techniques are specifically for recovering from damage done by the MTA, which by design does not have access to the cleartext of an end-to-end encrypted message. 7.4. Interactions with DKIM Verification A mangling MTA may also add DKIM headers after mangling. A subsequent MTA that handles the message may verify the message by checking DKIM headers, storing the results of that check in an Authentication-Results header (see [RFC7601]). If the MUA performs message repair, the repaired message might not pass DKIM check. An MUA that relies on Authentication-Results headers for any purpose on such a repaired message should consider re-validating the DKIM header itself directly after message repair (though this introduces new problems for stored/old messages, as older DKIM signing keys may no longer be published in the DNS). Note that a repaired message may have cryptographic properties that obviates the need for DKIM verification; an MUA that requires messages to have a valid Authentication-Results: header might waive that requirement for a signed message (whether any repair routine was invoked or not). 7.5. "Helpful" MTA mangling An MTA may intend to help the user by modifying the message. For example, in a clearsigned message, the MTA may modify a potentially- malicious URL in the text, or it might change an attachment name or MIME-type in an attachment. If an MUA attempts message repair in a way that reverses these modifications, it may negate whatever security advantages the MTA hoped to offer. It is not clear that this is a risk for repair of an encrypted message, though, since the MTA by design cannot perform such a modification on the cleartext of an encrypted message. 8. Privacy Considerations TODO: privacy considerations? Gillmor Expires November 23, 2019 [Page 7] Internet-Draft Common PGP/MIME Message Mangling May 2019 9. User Considerations Cryptographic properties of messages are only useful if the user understands them. 9.1. Indications of Mangling Can Be Confusing o TODO: about how indicators of mangled messages can cause user- experience confusion (e.g. parts of https://efail.de/ and https://github.com/RUB-NDS/Johnny-You-Are-Fired ) 9.2. Repair Indicators o TODO: how should a repairing MUA indicate that a repair has happened (or is possible) to the user? 9.3. Undoing Repair o TODO: Should a repairing MUA allow the user to reverse the repair and see the broken message? If so, what else should happen if the user takes this action (e.g., how is a subsequent "Reply" or "Forward" action affected?) 9.4. Attempting Decryption During Repair Attempting decryption may require access to secret key material, which itself may cause interaction with the user. If multiple repairs are attempted, each attempt to decrypt may require multiple decryptions. In some scenarios, multiple use of the secret key may be annoying to the user. It might be preferable to try to decrypt the apparent underlying block of text exactly once, regardless of message structure or repair, and if successful, either stash the session key, or store the cleartext in memory, while manipulating exterior structure. 10. IANA Considerations This document requires no actions from IANA. 11. Document Considerations [ RFC Editor: please remove this section before publication ] This document is currently edited as markdown. Minor editorial changes can be suggested via merge requests at https://gitlab.com/dkg/draft-openpgp-pgpmime-message-mangling or by e-mail to the author. Please direct all significant commentary to the public IETF OpenPGP mailing list: openpgp@ietf.org Gillmor Expires November 23, 2019 [Page 8] Internet-Draft Common PGP/MIME Message Mangling May 2019 12. Example Messages 12.1. Example Input Encrypted Message From: Michael Elkins To: Michael Elkins Mime-Version: 1.0 Content-Type: multipart/encrypted; boundary=foo; protocol="application/pgp-encrypted" --foo Content-Type: application/pgp-encrypted Version: 1 --foo Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- hIwDY32hYGCE8MkBA/wOu7d45aUxF4Q0RKJprD3v5Z9K1YcRJ2fve87lMlDlx4Oj eW4GDdBfLbJE7VUpp13N19GL8e/AqbyyjHH4aS0YoTk10QQ9nnRvjY8nZL3MPXSZ g9VGQxFeGqzykzmykU6A26MSMexR4ApeeON6xzZWfo+0yOqAq6lb46wsvldZ96YA AABH78hyX7YX4uT1tNCWEIIBoqqvCeIMpp7UQ2IzBrXg6GtukS8NxbukLeamqVW3 1yt21DYOjuLzcMNe/JNsD9vDVCvOOG3OCi8= =zzaA -----END PGP MESSAGE----- --foo-- 12.2. Input Message Mangled by "Mixed Up" Gillmor Expires November 23, 2019 [Page 9] Internet-Draft Common PGP/MIME Message Mangling May 2019 From: Michael Elkins To: Michael Elkins Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=foo --foo Content-Type: text/plain; charset="us-ascii" --foo Content-Type: application/pgp-encrypted Version: 1 --foo Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- hIwDY32hYGCE8MkBA/wOu7d45aUxF4Q0RKJprD3v5Z9K1YcRJ2fve87lMlDlx4Oj eW4GDdBfLbJE7VUpp13N19GL8e/AqbyyjHH4aS0YoTk10QQ9nnRvjY8nZL3MPXSZ g9VGQxFeGqzykzmykU6A26MSMexR4ApeeON6xzZWfo+0yOqAq6lb46wsvldZ96YA AABH78hyX7YX4uT1tNCWEIIBoqqvCeIMpp7UQ2IzBrXg6GtukS8NxbukLeamqVW3 1yt21DYOjuLzcMNe/JNsD9vDVCvOOG3OCi8= =zzaA -----END PGP MESSAGE----- --foo-- 13. Example Implemenations These examples use python3. As Code Examples, they are licensed under the Simplified BSD License as noted above. 13.1. Example: Detecting "Mixed Up" Encryption Gillmor Expires November 23, 2019 [Page 10] Internet-Draft Common PGP/MIME Message Mangling May 2019 import email.message def is_mixed_up(msg: email.message.Message) -> bool: if msg.get_content_type() == 'multipart/mixed': pts = msg.get_payload() if len(pts) == 3 and \ pts[0].get_content_type() == 'text/plain' and \ pts[0].get_payload(decode=True) == b'' and \ pts[1].get_content_type() == 'application/pgp-encrypted' and \ pts[1].get_payload(decode=True).strip() == b'Version: 1' and \ pts[2].get_content_type() == 'application/octet-stream': encpart = pts[2].get_payload(decode=True) # FIXME: this is not a true check for RFC4880 ASCII armor: lines = encpart.replace(b'\r\n', b'\n').split(b'\n') if lines[0] == b'-----BEGIN PGP MESSAGE-----' and \ lines[-1] == b'-----END PGP MESSAGE-----': return True return False 13.2. Example: Repairing "Mixed Up" Encryption import copy import email.message import typing def un_mix_up(msg: email.message.Message) -> \ typing.Optional[email.message.Message]: if is_mixed_up(msg): ret = copy.deepcopy(msg) ret.set_type('multipart/encrypted') ret.set_param('protocol', 'application/pgp-encrypted') ret.set_payload(msg.get_payload()[1:]) return ret return None 14. References 14.1. Normative References [RFC1847] Galvin, J., Murphy, S., Crocker, S., and N. Freed, "Security Multiparts for MIME: Multipart/Signed and Multipart/Encrypted", RFC 1847, DOI 10.17487/RFC1847, October 1995, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . Gillmor Expires November 23, 2019 [Page 11] Internet-Draft Common PGP/MIME Message Mangling May 2019 [RFC3156] Elkins, M., Del Torto, D., Levien, R., and T. Roessler, "MIME Security with OpenPGP", RFC 3156, DOI 10.17487/RFC3156, August 2001, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . 14.2. Informative References [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. Thayer, "OpenPGP Message Format", RFC 4880, DOI 10.17487/RFC4880, November 2007, . [RFC7601] Kucherawy, M., "Message Header Field for Indicating Message Authentication Status", RFC 7601, DOI 10.17487/RFC7601, August 2015, . Author's Address Daniel Kahn Gillmor American Civil Liberties Union 125 Broad St. New York, NY 10004 USA Email: dkg@fifthhorseman.net Gillmor Expires November 23, 2019 [Page 12]