SMTP TLS ReportingGoogle, Incdmargolis (at) google.comComcast, Incalex_brotman (at) comcast.comYahoo!, Incrbinu (at) yahoo-inc (dot com)Microsoft, Incjanet.jones (at) microsoft (dot com)Google, Incrisher (at) google (dot com)
Applications
Using TLS in ApplicationsA number of protocols exist for establishing encrypted channels between SMTP
Mail Transfer Agents, including STARTTLS, DANE TLSA, and MTA-STS. These
protocols can fail due to misconfiguration or active attack, leading to
undelivered messages or delivery over unencrypted or unauthenticated channels.
This document describes a reporting mechanism and format by which sending
systems can share statistics and specific information about potential failures
with recipient domains. Recipient domains can then use this information to both
detect potential attackers and diagnose unintentional misconfigurations.
The STARTTLS extension to SMTP allows SMTP clients and hosts to
establish secure SMTP sessions over TLS. The protocol design is based on
"Opportunistic Security" (OS) , which maintains interoperability with
clients that do not support STARTTLS but means that any attacker who can delete
parts of the SMTP session (such as the "250 STARTTLS" response) or redirect the
entire SMTP session (perhaps by overwriting the resolved MX record of the
delivery domain) can perform a downgrade or interception attack.
Because such "downgrade attacks" are not necessarily apparent to the receiving
MTA, this document defines a mechanism for sending domains to report on failures
at multiple stages of the MTA-to-MTA conversation.
Recipient domains may also use the mechanisms defined by MTA-STS (TODO: Add ref)
or DANE to publish additional encryption and authentication
requirements; this document defines a mechanism for sending domains that are
compatible with MTA-STS or DANE to share success and failure statistics with
recipient domains.
Specifically, this document defines a reporting schema that covers failures in
routing, STARTTLS negotiation, and both DANE and MTA-STS (TODO: Add
ref) policy validation errors, and a standard TXT record that recipient domains
can use to indicate where reports in this format should be sent.
This document is intended as a companion to the specification for SMTP MTA
Strict Transport Security (MTA-STS, TODO: Add ref).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in .
We also define the following terms for further use in this document:
MTA-STS Policy: A definition of the expected TLS availability, behavior, and
desired actions for a given domain when a sending MTA encounters
problems in negotiating a secure channel. MTA-STS is defined in [TODO]DANE Policy: A mechanism by which administrators can supply a record that can
be used to validate the certificate presented by an MTA. DANE is defined
in .TLSRPT Policy: A policy specifying the endpoint to which sending MTAs should
deliver reports.Policy Domain: The domain against which an MTA-STS or DANE Policy is defined.Sending MTA: The MTA initiating the delivery of an email message.This document is intended as a companion to the specification for SMTP MTA
Strict Transport Security (MTA-STS, TODO: Add RFC ref).SMTP-TLSRPT defines a mechanism for sending domains that are compatible with
MTA-STS or DANE to share success and failure statistics with recipient
domains. DANE is defined in and MTA-STS is defined in [TODO : Add
RFC ref]A domain publishes a record to its DNS indicating that it wishes to
receive reports. These SMTP TLSRPT policies are distributed via DNS from the
Policy Domain's zone, as TXT records (similar to DMARC policies) under the name
_smtp-tlsrpt. For example, for the Policy Domain example.com, the
recipient's TLSRPT policy can be retrieved from _smtp-tlsrpt.example.com.
Policies consist of the following directives:
v: This value MUST be equal to TLSRPTv1.rua: A URI specifying the endpoint to which aggregate information about
policy validation results should be sent (see , "Reporting
Schema", for more information). Two URI schemes are supported: mailto
and https. As with DMARC , the policy domain can specify a
comma-separated list of URIs.In the case of https, reports should be submitted via POST () to
the specified URI. Report submitters MAY ignore certificate validation
errors when submitting reports via https.In the case of mailto, reports should be submitted to the specified
email address (). When sending failure reports via SMTP, sending
MTAs MUST deliver reports despite any TLS-related failuresand SHOULD NOT
include this SMTP session in the next report. This may mean that the reports
are delivered in the clear. Additionally, reports sent via SMTP MUST contain a
valid DKIM signature by the reporting domain. Reports lacking
such a signature MUST be ignored by the recipient. DKIM signatures must not
use the "l=" attribute to limit the body length used in the signature.The formal definition of the _smtp-tlsrpt TXT record, defined using
& , is as follows:
If multiple TXT records for _smtp-tlsrpt are returned by the resolver, records
which do not begin with v=TLSRPTv1; are discarded. If the number of resulting
records is not one, senders MUST assume the recipient domain does not implement
TLSRPT. If the resulting TXT record contains multiple strings, then the record
MUST be treated as if those strings are concatenated together without adding
spaces.
Parsers MUST accept TXT records which are syntactically valid (i.e.
valid key-value pairs separated by semi-colons) and implementing a superset of
this specification, in which case unknown fields SHALL be ignored.
The report is composed as a plain text file encoded in the I-JSON format
().
Aggregate reports contain the following fields:
Report metadata:
The organization responsible for the reportContact information for one or more responsible parties for the
contents of the reportA unique identifier for the reportThe reporting date range for the reportPolicy, consisting of:
One of the following policy types:
(1) The MTA-STS policy applied (as a string)
(2) The DANE TLSA record applied (as a string, with each RR entry of the
RRset listed and separated by a semicolon)
(3) The literal string no-policy-found, if neither a DANE nor
MTA-STS policy could be found.The domain for which the policy is appliedThe MX hostAn identifier for the policy (where applicable)Aggregate counts, comprising result type, sending MTA IP, receiving MTA
hostname, session count, and an optional additional information field
containing a URI for recipients to review further information on a failure
type.Note that the failure types are non-exclusive; an aggregate report may contain
overlapping counts of failure types when a single send attempt encountered
multiple errors. Reporters may report multiple applied policies (for example, an
MTA-STS policy and a DANE TLSA record for the same domain and MX); even in the
case where only a single policy was applied, the "policies" field of the report
body MUST be an array and not a singular value.
The report SHOULD cover a full day, from 0000-2400 UTC. This should allow for
easier correlation of failure events. To avoid a Denial of Service against the
system processing the reports, the reports should be delivered after some
delay, perhaps several hours.
success-count: This indicates that the sending MTA was able to successfully
negotiate a policy-compliant TLS connection, and serves to provide a
"heartbeat" to receiving domains that reporting is functional and tabulating
correctly. This field contains an aggregate count of successful connections
for the reporting system.
failure-count: This indicates that the sending MTA was unable to
successfully establish a connection with the receiving platform.
, "Result Types", will elaborate on the failed negotiation
attempts. This field contains an aggregate count of failed connections.The list of result types will start with the minimal set below, and is expected
to grow over time based on real-world experience. The initial set is:
starttls-not-supported: This indicates that the recipient MX did not support
STARTTLS.certificate-host-mismatch: This indicates that the certificate presented did
not adhere to the constraints specified in the MTA-STS or DANE policy, e.g.
if the MX does not match any identities listed in the Subject Alternate Name
(SAN) .certificate-expired: This indicates that the certificate has expired.certificate-not-trusted: This a label that covers multiple certificate
related failures that include, but not limited to errors such as
untrusted/unknown CAs, certificate name constraints, certificate chain errors
etc. When using this declaration, the reporting MTA SHOULD utilize the
failure-reason to provide more information to the receiving entity.validation-failure: This indicates a general failure for a reason not
matching a category above. When using this declaration, the reporting MTA
SHOULD utilize the failure-reason to provide more information to the
receiving entity.tlsa-invalid: This indicates a validation error in the TLSA record
associated with a DANE policy. None of the records in the RRset were found to
be valid.dnssec-invalid: This would indicate that no valid records were returned from
the recursive resolver. The request returned with SERVFAIL for the requested
TLSA record. It should be noted that if the reporter's systems are having
problems resolving destination DNS records due to DNSSEC failures, it's
possible they will also be unable to resolve the TLSRPT record, therefore
these types of reports may be rare.sts-policy-invalid: This indicates a validation error for the overall
MTA-STS policy.sts-webpki-invalid: This indicates that the MTA-STS policy could not be
authenticated using PKIX validation.
When a negotiation failure can not be categorized into one of the "Negotiation
Failures" stated above, the reporter SHOULD use the validation-failure
category. As TLS grows and becomes more complex, new mechanisms may not be
easily categorized. This allows for a generic feedback category. When this
category is used, the reporter SHOULD also use the failure-reason-code to give
some feedback to the receiving entity. This is intended to be a short text
field, and the contents of the field should be an error code or error text, such
as "X509_V_ERR_UNHANDLED_CRITICAL_CRL_EXTENSION".
Transient errors due to too-busy network, TCP timeouts, etc. are not required to
be reported.
The JSON schema is derived from the HPKP JSON schema (cf. Section 3)
organization-name: The name of the organization responsible for the report.
It is provided as a string.date-time: The date-time indicates the start- and end-times for the report
range. It is provided as a string formatted according to Section 5.6,
"Internet Date/Time Format", of . The report should be for a full
UTC day, 0000-2400.email-address: The contact information for a responsible party of the
report. It is provided as a string formatted according to Section 3.4.1,
"Addr-Spec", of .report-id: A unique identifier for the report. Report authors may use
whatever scheme they prefer to generate a unique identifier. It is provided as
a string.policy-type: The type of policy that was applied by the sending domain.
Presently, the only three valid choices are tlsa, sts, and the literal
string no-policy-found. It is provided as a string.policy-string: A string representation of the policy, whether TLSA record
( section 2.3) or MTA-STS policy. Examples: TLSA:
"_25._tcp.mx.example.com. IN TLSA ( 3 0 1 \
1F850A337E6DB9C609C522D136A475638CC43E1ED424F8EEC8513D7 47D1D085D )" MTA-STS:
"version: STSv1\nmode: report\nmx: mx1.example.com\nmx: \
mx2.example.com\nmx: mx.backup-example.com\nmax_age: 12345678"domain: The Policy Domain is the domain against which the MTA-STS or DANE
policy is defined. In the case of Internationalized Domain Names
(), the domain is the Punycode-encoded A-label () and
not the U-label.mx-host-pattern: The pattern of MX hostnames from the applied policy. It is
provided as a string, and is interpreted in the same manner as the "Checking
of Wildcard Certificates" rules in Section 6.4.3 of . In the case
of Internationalized Domain Names (), the domain is the
Punycode-encoded A-label () and not the U-label.result-type: A value from , "Result Types", above.ip-address: The IP address of the sending MTA that attempted the STARTTLS
connection. It is provided as a string representation of an IPv4 (see below)
or IPv6 () address in dot-decimal or colon-hexadecimal notation.receiving-mx-hostname: The hostname of the receiving MTA MX record with
which the sending MTA attempted to negotiate a STARTTLS connection.receiving-mx-helo: (optional) The HELO or EHLO string from the banner
announced during the reported session.total-successful-session-count: The aggregate number (integer) of
successfully negotiated TLS-enabled connections to the receiving site.total-failure-session-count: The aggregate number (integer) of failures to
negotiate a TLS-enabled connection to the receiving site.failed-session-count: The number of (attempted) sessions that match the
relevant result-type for this section.additional-info-uri: An optional URI pointing to additional
information around the relevant result-type. For example, this URI might
host the complete certificate chain presented during an attempted STARTTLS
session.failure-reason-code: A text field to include a TLS-related error code or
error message.
For report purposes, an IPv4 Address is defined as:
IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet
dec-octet = DIGIT ; 0-9
/ %x31-39 DIGIT ; 10-99
/ "1" 2DIGIT ; 100-199
/ "2" %x30-34 DIGIT ; 200-249
/ "25" %x30-35 ; 250-255
Reports can be delivered either as an email message via SMTP or via HTTP
POST.
The filename is RECOMMENDED to be constructed using the following ABNF:
The extension MUST be "json" for a plain JSON file, or "json.gz" for a
JSON file compressed using GZIP.
"unique-id" allows an optional unique ID generated by the Sending MTA to
distinguish among multiple reports generated simultaneously by different
sources within the same Policy Domain. For example, this is a possible
filename for the gzip file of a report to the Policy Domain "example.net"
from the Sending MTA "mail.sender.example.com":
The report SHOULD be subjected to GZIP compression for both email and HTTPS
transport. Declining to apply compression can cause the report to be too large
for a receiver to process (a commonly observed receiver limit is ten megabytes);
compressing the file increases the chances of acceptance of the report at some
compute cost.
The report MAY be delivered by email. To make the reports machine-parsable
for the receivers, we define a top-level media type multipart/report with
a new parameter report-type="tlsrpt". Inside it, there are two parts: The
first part is human readable, typically text/plain, and the second part is
machine readable with a new media type defined called application/tlsrpt+json.
If compressed, the report should use the media type application/tlsrpt+gzip.
In addition, the following two new top level message header fields are defined:
The TLS-Report-Submitter value MUST match the value found in the filename
and the domain from the contact-info from the report body. These
message headers MUST be included and should allow for easy searching for all
reports submitted by a report domain or a particular submitter, for example
in IMAP :
s SEARCH HEADER "TLS-Report-Domain" "example.com"It is presumed that the aggregate reporting address will be equipped to process
new message header fields and extract MIME parts with the prescribed media type
and filename, and ignore the rest. These additional headers SHOULD be included
in the DKIM signature for the message.
The .Subject field for report submissions SHOULD conform to the
following ABNF:
The first domain-name indicates the DNS domain name about which the
report was generated. The second domain-name indicates the DNS
domain name representing the Sending MTA generating the report.
The purpose of the Report-ID: portion of the field is to enable the
Policy Domain to identify and ignore duplicate reports that might be
sent by a Sending MTA.
For instance, this is a possible Subject field for a report to the
Policy Domain "example.net" from the Sending MTA
"mail.sender.example.com". It is line-wrapped as allowed by :
Note that, when sending failure reports via SMTP, sending MTAs MUST NOT honor
MTA-STS or DANE TLSA failures.
The report MAY be delivered by POST to HTTPS. If compressed, the report SHOULD
use the media type application/tlsrpt+gzip, and application/tlsrpt+json
otherwise (see section , "IANA Considerations").
A reporting entity SHOULD expect a "successful" response from the accepting
HTTPS server, typically a 200 or 201 HTTP code . Other codes could
indicate a delivery failure, and may be retried as per local policy. The
receiving system is not expected to process reports at receipt time, and MAY
store them for processing at a later time.
In the event of a delivery failure, regardless of the delivery method, a
sender SHOULD attempt redelivery for up to 24hrs after the initial attempt. As
previously stated the reports are optional, so while it is ideal to attempt
redelivery, it is not required. If multiple retries are attempted, ideally they
would be on a logarithmic scale.
As stated above, there are a variable number of ways to declare information
about the data therein. If it should be the case that these objects were to
disagree, then the report data contained within the JSON body MUST be considered
the authoritative source for those data elements.
The following are the IANA considerations discussed in this document.
Below is the Internet Assigned Numbers Authority (IANA) Permanent Message Header
Field registration information per .
This document registers a new parameter report-type="tlsrpt" under
multipart/report top-level media type for use with .
The media type suitable for use as a report-type is defined in the
following section.
This document registers multiple media types, beginning with Table 1 below.
Type name: application
Subtype name: tlsrpt+json
Required parameters: n/a
Optional parameters: n/a
Encoding considerations: Encoding considerations are identical to
those specified for the application/json media type. See
.
Security considerations: Security considerations relating to SMTP
TLS Reporting are discussed in Section 7.
Interoperability considerations: This document specifies format of
conforming messages and the interpretation thereof.
Published specification: Section 5.3 of this document.
Applications that use this media type: Mail User Agents (MUA) and
Mail Transfer Agents.
Additional information:
Person & email address to contact for further information: See
Authors' Addresses section.
Intended usage: COMMON
Restrictions on usage: n/a
Author: See Authors' Addresses section.
Change controller: Internet Engineering Task Force
(mailto:iesg@ietf.org).
Type name: application
Subtype name: tlsrpt+gzip
Required parameters: n/a
Optional parameters: n/a
Encoding considerations: Binary
Security considerations: Security considerations relating to SMTP
TLS Reporting are discussed in Section 7.
Interoperability considerations: This document specifies format of
conforming messages and the interpretation thereof.
Published specification: Section 5.3 of this document.
Applications that use this media type: Mail User Agents (MUA) and
Mail Transfer Agents.
Additional information:
Person & email address to contact for further information: See
Authors' Addresses section.
Intended usage: COMMON
Restrictions on usage: n/a
Author: See Authors' Addresses section.
Change controller: Internet Engineering Task Force
(mailto:iesg@ietf.org).
This document creates a new registry, "STARTTLS Validation Result Types". The
initial entries in the registry are:
The above entries are described in section , "Result Types." New
result types can be added to this registry using "Expert Review" IANA
registration policy.
SMTP TLS Reporting provides transparency into misconfigurations or attempts to
intercept or tamper with mail between hosts who support STARTTLS. There are
several security risks presented by the existence of this reporting channel:
Flooding of the Aggregate report URI (rua) endpoint: An attacker could flood
the endpoint with excessive reporting traffic and prevent the receiving domain
from accepting additional reports. This type of Denial-of-Service attack would
limit visibility into STARTTLS failures, leaving the receiving domain blind to
an ongoing attack.Untrusted content: An attacker could inject malicious code into the report,
opening a vulnerability in the receiving domain. Implementers are advised to
take precautions against evaluating the contents of the report.Report snooping: An attacker could create a bogus TLSRPT record to receive
statistics about a domain the attacker does not own. Since an attacker able to
poison DNS is already able to receive counts of SMTP connections (and, absent
DANE or MTA-STS policies, actual SMTP message payloads), this does not present
a significant new vulnerability.Reports as DDoS: TLSRPT allows specifying destinations for the reports that
are outside the authority of the Policy Domain, which allows domains to
delegate processing of reports to a partner organization. However, an attacker
who controls the Policy Domain DNS could also use this mechanism to direct the
reports to an unwitting victim, flooding that victim with excessive reports.
DMARC defines a solution for verifying delegation to avoid such
attacks; the need for this is greater with DMARC, however, because DMARC
allows an attacker to trigger reports to a target from an innocent third party
by sending that third party mail (which triggers a report from the third party
to the target). In the case of TLSRPT, the attacker would have to induce the
third party to send the attacker mail in order to trigger reports from the
third party to the victim; this reduces the risk of such an attack and the
need for a verification mechanism.Finally, because TLSRPT is intended to help administrators discover
man-in-the-middle attacks against transport-layer encryption, including attacks
designed to thwart negotiation of encrypted connections (by downgrading
opportunistic encryption or, in the case of MTA-STS, preventing discovery of a
new MTA-STS policy), we must also consider the risk that an adversary who can
induce such a downgrade attack can also prevent discovery of the TLSRPT TXT
record (and thus prevent discovery of the successful downgrade attack).
Administrators are thus encouraged to deploy TLSRPT TXT records with a large TTL
(reducing the window for successful attacks against DNS resolution of the
record) or to deploy DNSSEC on the deploying zone.
Figure: Example JSON report for a messages from Company-X to Company-Y, where
100 sessions were attempted to Company Y servers with an expired certificate and
200 sessions were attempted to Company Y servers that did not successfully
respond to the STARTTLS command. Additionally 3 sessions failed due to
"X509_V_ERR_PROXY_PATH_LENGTH_EXCEEDED".