Greasing HTTP
made in
Prahran
VIC
Australia
mnot@mnot.net
https://www.mnot.net/
General
Internet-Draft
Like many network protocols, HTTP is vulnerable to ossification of its extensibility points. This draft explains why HTTP ossification is a problem and establishes guidelines for exercising those extensions by ‘greasing’ the protocol to combat it.
RFC EDITOR: please remove this section before publication
The issues list for this draft can be found at https://github.com/mnot/I-D/labels/http-grease.
The most recent (often, unpublished) draft is at https://mnot.github.io/I-D/http-grease/.
Recent changes are listed at https://github.com/mnot/I-D/commits/gh-pages/http-grease.
See also the draft’s current status in the IETF datatracker, at
https://datatracker.ietf.org/doc/draft-nottingham-http-grease/.
Like many network protocols, HTTP is vulnerable to ossification of its extensibility points. Ossification happens when a significant number of the systems that generate, transmit, handle, or consume the protocol don’t accept a new extension, thereby making it more difficult to deploy extensions.
For example, TCP has effectively been ossified by middleboxes that assume that new TCP options will not be deployed; likewise, the Protocol field in IP has been effectively ossified as well, since so many networks will only accept TCP or UDP traffic.
Addressing this issue is important; protocol extensibility allows adaptation to new circumstances as well as application to new use cases. Inability to deploy new extensions creates pressure to misuse the protocol – often leading to undesirable side effects – or to use other protocols, reducing the value that the community gets from a shared, standard protocol.
While there are a few ways that protocol designers can mitigate ossification, this document focuses on a technique that’s well suited to many of the ossification risks in HTTP: ‘greasing’ extensibility points by exercising them, so that they don’t become ‘rusted shut.’
) pioneered greasing techniques in IETF protocols; this document explains how they apply to HTTP. It focuses on generic HTTP features; other documents cover versioned extensibility points (e.g., see ).
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.
As an application protocol, HTTP has several extensibility points. For example, methods, status codes, header and trailer fields, cache directives, range units and content codings are all HTTP extension points.
Each extension point defines how unrecognised values should be handled; in most cases, they should be ignored (e.g. header fields, cache directives and range units), while in a few cases they have other handling (e.g., unrecognised methods result in a 405 status code; unrecognised status codes devolve to a more generic x00 status code).
Implementations and other components that diverge from these defined behaviours risk ossifying that extensibility point.
For example, it is increasingly common for Web Application Firewalls (WAFs), bot detection services and similar components to reject HTTP requests that contain header fields with certain characters or strings, even though syntactically valid, and even though the header fields are not necessarily recognised by the recipient.
This behaviour has become prevalent enough to make it difficult for Web browsers and other clients to introduce new request header fields. That difficulty is aggravated by two factors:
A relatively large number of vendors create these components, but have little coordination between them, leading to wide variances in behaviour, and
Many of these components’ deployments are not updated regularly and reliably, leading to difficulty in addressing ossification issues even when they are identified.
To avoid ossification of request header fields, it is Best Current Practice to grease them, as explained below. Other HTTP extensibility points might be added in the future, and it is not to be inferred that greasing other HTTP extensibility points is not good practice.
HTTP clients SHOULD grease request header fields. There are two aims in doing so:
Preserving the ability to add new request header fields over time
Preserving the ability to add new request header fields with values containing common syntax
Clients can grease a given request at their discretion. For example, a client implementation might add one or more grease request header fields to every request it makes, or it might add one to every third or tenth request.
Depending on the deployment model of the client, it might do this in production releases automatically (especially if there are ways that it can modify how grease values are sent with a high degree of control, in case too many errors are encountered), or it might do so only in pre-releases.
Grease field names SHOULD be hard to predict; e.g., they SHOULD NOT have any identifying prefix, suffix, or pattern. However, they MUST NOT be likely to conflict with unregistered or future field names, and the grease coordinator MUST avoid potentially offensive or confusing terms. They also MUST conform to the syntactic requirements for field names in HTTP (, Section 4.3).
This can be achieved in different ways (which SHOULD vary from time to time), for example:
Combine two or three dictionary words or proper nouns with a hyphen (e.g., ‘Skateboard-Clancy’, ‘Murray-Fortnight-Scout’)
Append digits to a dictionary word (e.g., ‘Turnstile23’)
Generate a string using a hash or similar function (e.g., ‘dd722785c01b’)
Grease field names are not required to be registered in the IANA HTTP Field Name Registry, unless they are intended to be used over an extended period of time (e.g., more than one year). However, they MAY be registered as Provisional with a reference to this RFC or another explanatory resource, to help interested parties to find out what they are used for. Such registered values SHOULD be removed after the client stops using that field.
Greasing clients SHOULD not reuse other clients’ grease fields names, unless they coordinate.
Grease field values can be fixed strings, or dynamically generated at runtime. It is RECOMMENDED that greasing clients exercise the various types in .
If an error is encountered by a greasing client, it SHOULD NOT re-issue the request without the grease value, since hiding the consequences of the failure doesn’t serve the purpose of greasing.
Greasing clients SHOULD announce new field names they intend to grease on the http-grease@ietf.org mailing list.
Some HTTP extensibility points are becoming (or have become) ossified because of security considerations; receiving implementations believe that it is more secure to reject unknown values, or that they can identify undesirable peers through their use of extensions.
This document does not directly address these concerns, nor does it directly disallow such behaviour. Instead, it aims to encourage the ability to accommodate new extensions more quickly than is now possible.
Key words for use in RFCs to Indicate Requirement Levels
In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.
Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words
RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.
HTTP Semantics
The Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypertext information systems. This document defines the semantics of HTTP: its architecture, terminology, the "http" and "https" Uniform Resource Identifier (URI) schemes, core request methods, request header fields, response status codes, response header fields, and content negotiation. This document obsoletes RFC 2818, RFC 7231, RFC 7232, RFC 7233, RFC 7235, RFC 7538, RFC 7615, RFC 7694, and portions of RFC 7230.
Applying Generate Random Extensions And Sustain Extensibility (GREASE) to TLS Extensibility
This document describes GREASE (Generate Random Extensions And Sustain Extensibility), a mechanism to prevent extensibility failures in the TLS ecosystem. It reserves a set of TLS protocol values that may be advertised to ensure peers correctly handle unknown values.
GREASE for HTTP/2
Reserves several values in the HTTP/2 registries to exercise the requirement that clients and servers ignore unknown values.
Structured Field Values for HTTP
This document describes a set of data types and associated algorithms that are intended to make it easier and safer to define and handle HTTP header and trailer fields, known as "Structured Fields", "Structured Headers", or "Structured Trailers". It is intended for use by specifications of new HTTP fields that wish to use a common syntax that is more restrictive than traditional HTTP field values.