]> Open Future

paul@openfuture.eu

Mozilla

mt@lowentropy.net

Web and Internet Transport AI Preferences AI Preferences Opt-Out Vocabulary This document defines a vocabulary for expressing preferences regarding how digital assets are used by automated processing systems. This vocabulary allows for the declaration of restrictions or permissions for use of digital assets by such systems. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the AI Preferences Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at . Note to Readers As detailed below, this is a working document. Its contents DO NOT REFLECT CONSENSUS of the Working Group either in whole or part. Presense or absense of any particular text does not indicate consensus, and this document is published solely as a basis of further discussion.

Introduction This document defines a vocabulary of preferences regarding how automated systems process digital assets -- in particular, the training and use of AI models. This vocabulary can be used to describe the types of uses that a declaring party may wish to explicitly restrict or allow. The vocabulary is intended to be used in jurisdictions where expressing preferences results in legal obligations, as well as where there are no associated legal obligations. In either case, expressing preferences is without prejudice to applicable laws, including the applicability of exceptions and limitations to copyright. defines the data model for AI Preferences. defines the terms of the vocabulary. explains how to use AI Preferences in a data processing application, and describes a way to serialize preferences into a string. describes a process for determining the preference for a category of use. defines mechanisms to associate preferences with assets. Other means of association might be defined separately in the future.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses the following terms:

Asset:

A digital file or stream of data, usually with associated metadata.

Declaring party:

The entity that expresses a preference with regards to an Asset.

Generative AI model:

An AI model that is used to generate synthetic content in one or more modalities. Alternatively, an AI model that is made available for use to generate synthetic content in one or more modalities. This definition does not include a model that is used, or made available for use, for classification, ranking, or scoring, even where the model generates rationale for its output.

Statements of Preference NOTE: This section does not yet have consensus; see "Note to Readers" above. The vocabulary is a set of categories, each of which is defined to cover a class of usage for assets. defines the core set of usage categories in detail. A statement of preference -- or usage preference -- is made about an asset. A statement of preference follows a simple data model where a preference is assigned to each of the categories of use in the vocabulary. A preference is either to allow or disallow the usage associated with the category. A statement of preference can indicate preferences about some, all, or none of the categories from the vocabulary. This can mean that the preference is unknown for some usage categories. After processing a statement of preferences the recipient associates each category of use one of three preference values: "allowed", "disallowed", or "unknown". In the absence of a statement of preference, all usage categories are assigned a preference value of "unknown". The process for consulting a statement of preference is defined in . Different declaring parties might each make their own statement of preference regarding a particular asset. The process for managing multiple statements of preference is defined in . An exemplary syntax for statements of preference is defined in .

Understanding Preferences This document and describe how statements of preference are associated with assets. The goal of these specifications is to ensure that the recipient of an asset knows what preferences have been associated with the asset. What a recipient then does with that information depends on many factors; see . There are also some caveats that need to be considered as it relates to understanding what the preferences for a given asset are (as opposed to what actions might then follow). A recipient can only apply preferences it understands. Recipients that implement this specification will understand the vocabulary terms defined in , but they might not understand terms defined in extensions; see . A recipient will only process preferences expressed through methods it has implemented. Those methods might be limited to those in or it could also include other methods (see ). If a preference is associated with an asset using a method the recipient does not understand or recognize, the recipient will remain ignorant of that preference. Depending on the way in which preferences are expressed, a recipient might be unable to tell the source of the preference. Unless the source is explicitly identified, no assumptions can be made about where a preference originates. For example, preferences in robots.txt (see ) only implies that a server is the source of those preferences. A method of associating preferences with assets could explicitly define the source of the preferences, which might involve authentication. Otherwise, no assumptions can be made about the origin of preferences. The apparent source of preferences could be representing their own preferences, the preferences of others, or the synthesis of multiple preferences from different sources.

Applying Preferences This specification enables the expression of a defined set of preferences that can be communicated and interoperably understood. Readers of this specification should understand that it does not:

• ensure that preferences are followed;
• address if, how, or when preferences should be followed or not-followed;
• address technical, legal, contractual, or other mechanisms that might create a stronger requirement to follow or not follow preferences;
• consider situations or purposes that might justify following or not-following expressed preferences.

An entity that receives usage preferences has a choice whether to follow those preferences. This specification does not determine how that choice is made. Whether and under which circumstances a preference is followed is outside the scope of this specification.

Communicating Preferences with Trained Models NOTE: Issue 245 is open to track whether this section is included or not. Compliance with preferences regarding the use of assets in training of AI models also includes conveying the preferences associated with those assets along with that model when it is distributed. This requirement can be met by providing information on the full range of preferences that were associated with assets that were included in the training inputs or by indicating what sorts of uses are consistent with those preferences.

Vocabulary Definition NOTE: This section does not yet have consensus; see "Note to Readers" above. This section defines the categories of use in the vocabulary. These categories describe concrete, observable outcomes that depend on the use of assets. The definitions seek to avoid describing internal details of implementations or their architecture.

AI Training Using an asset to modify the learned parameters of a generative AI model. The training of models that are used to perform exclusively non-generative tasks is not included in this category, even if the model is capable of generative tasks.

AI Use Using an asset as input to a generative AI model, where the asset is not directly provided by the user. NOTE: Issue 249 addresses the question of whether "direct" includes referencing assets, by URL or other means. This does not include any use that is included in the AI Training usage category ().

Search Using an asset in an application where the primary purpose of the application is to select assets and direct users to the location of those assets. This category of use only applies under the following conditions:

• Where the presentation of an asset in search output -- if selected for presentation -- includes a direct reference or link to the original location from which the asset was retrieved.
• When excerpts from the asset are displayed they serve to assist users in evaluating the relevance of the result.

This category does not include the use of assets to generate summaries. Displaying titles or excerpts from assets, changes to improve accessibility, such as translation, transcription, or text-to-speech, are included in this category. A preference to allow this category of use includes allowing any processing internal to the application that is performed on assets. Allowing this processing is conditional on the outputs of any processing being exclusively used by the search application according to the other restrictions in this section. Allowed processing therefore includes the training of AI models using the assets and the use of those models provided that the resulting models and their outputs are used exclusively in ways that meet the above conditions regarding referencing and excerpts. This category of use overrides any usage that falls into other categories, including AI Training () and AI Use ().

Vocabulary Extensions Extensions to this vocabulary are defined in a standards-track RFC that updates this document. The definition of the extension MUST define how any potential overlap between usage categories is resolved. Definitions can identify which usage category applies for any such overlap. Systems that use this vocabulary might seek to integrate the terms in this vocabulary as part of a larger data model that includes other terms not defined here. Such usage is not subject to the RFC requirement above, but special care is needed to avoid defining overlapping categories of use. describes how concepts from an alternative format might be mapped to this vocabulary.

Applying Statements of Preference After acquiring a statement of preference, which might use the process in , an application can determine the status of a specific usage category. If the statement of preference contains an explicit preference regarding that category of use -- either to allow or disallow -- that is the outcome. Otherwise, the preference for that category is unknown. This process results in one of three potential answers: allow, disallow, and unknown. Applications can use the answer to guide their behavior. One approach for dealing with an unknown outcome is to assign a default value. This document takes no position on what default might be assigned.

Combining Preferences An application might receive multiple statements of preference, obtained using different methods or from different declaring parties. This might result in conflicting preferences. Absent some other means of resolving conflicts, the following process applies to each usage category:

• If any statement of preference indicates that the usage is disallowed, the result is that the usage is disallowed.
• Otherwise, if any statement of preference allows the usage, the result is that the usage is allowed.
• Otherwise, the preference for that category is unknown.

This process ensures that the most restrictive preference applies.

More Specific Instructions A recipient of a statement of preferences that follows the model in might receive more specific instructions in two ways:

• Extensions to the vocabulary might add qualifications or conditions to preferences about usage.
• Contractual agreements or other specific arrangements might override statements of preference.

For instance, a statement of preferences might indicate a preference to disallow a category of use for an asset. If arrangements, such as legal agreements, exist that explicitly permit the use of that asset, those arrangements likely apply despite the existence of machine-readable statements of preference, unless the terms of the arrangement explicitly say otherwise.

Exemplary Serialization Format This section defines an exemplary serialization format for preferences. The format describes how the abstract model could be turned into Unicode text or sequence of bytes. The format relies on the Dictionary type defined in . The dictionary keys correspond to usage categories and the dictionary values correspond to explicit preferences, which can be either y or n; see . For example, the following states a preference to allow model training (), disallow search (), with the preference for other categories being unknown:

Usage Category Labels Each usage category in the vocabulary () is mapped to a short textual label. tabulates this mapping. Mappings for Categories

Category	Label	Reference
AI Training	train-ai
AI Use	ai-use
Search	search

These tokens are case sensitive. Tokens defined for a new usage category can only use lowercase latin characters (a-z), digits (0-9), "_", "-", ".", or "*". These are encoded using the mappings in .

Preference Labels The data model in used has two options for explicit preferences associated with each category: allow and disallow. These are mapped to single byte Tokens () of y and n, respectively.

Text Encoding Structured Fields describes a byte-level encoding of information, not a text encoding. This makes this format suitable for inclusion in any protocol or format that carries bytes. Some formats are defined in terms of strings rather than bytes. These formats might need to decode the bytes of this format to obtain a string. As the syntax is limited to ASCII , an ASCII or UTF-8 decoder can be used. This results in the strings that this document uses. Processing (see ) requires a sequence of bytes, so any format that uses strings needs to encode strings first. Again, this process can use ASCII or UTF-8.

Syntax Extensions There are two ways by which this syntax might be extended: the addition of new labels and the addition of parameters. New labels might be defined to correspond to new usage categories. addresses the considerations for defining new categories. New labels might also be defined for other types of extension that do not assign a preference to a usage category. In either case, when processing a parsed Dictionary to obtain preferences, any unknown labels MUST be ignored. The Dictionary syntax () can associate parameters with each key-value pair. This document does not define any semantics for any parameters that might be included. When processing a parsed Dictionary to obtain preferences, any unknown parameters MUST be ignored. In either case, new extensions need to be defined in an RFC that updates this document.

Processing Algorithm To process a series of bytes to recover the stated preferences, those bytes are parsed into a Dictionary (), then preferences are assigned to each usage category in the vocabulary. This algorithm produces a keyed collection of values, where each key has at most one value and optional parameters. To obtain preferences, iterate through the defined categories in the vocabulary. For the label that corresponds to that category (see ), obtain the corresponding value from the collection, disregarding any parameters. A preference is assigned as follows:

• If the value is a Token with a value of y, the associated preference is to allow that category of use.
• If the value is a Token with a value of n, the associated preference is to disallow that category of use.
• Otherwise, the preference for that category is unknown.

Note that this last alternative includes the key being absent from the collection, values that are not Tokens, and Token values that are other than y or n. All of these are not errors, they only result in the corresponding preference being unknown. This process results in an abstract data model that assigns a preference to each usage category as described in .

Multiple Preferences It is important to note that if the same key appears multiple times, the algorithms in ensure that only the last value applies. This means that duplicating a key could result in unexpected outcomes. For example, the following results in all preferences being unknown, because the type of the parameter values (a boolean and a string respectively) are not tokens: If the parsing of the Dictionary fails, preferences are unknown. This includes where keys include uppercase characters, as this format is case sensitive (more correctly, it operates on bytes, not strings).

Preference Parameters This document does not define a use for parameters. Only those parameters associated with the value that is selected according to apply. For example, the following preference carries no parameters, and a preference to allow the usage: Parameters can therefore be carried for any preference value, including where preferences are unknown. For example, the following train-ai preference has parameters even though the preference is unknown:

Alternative Formats The format defined in this document is only an exemplary way to represent preferences. The data model described in can be used without this serialization. Any alternative format needs to define the mapping both from that format to the model used in this document and from the model to the alternative format. This includes any potential for extensions to the vocabulary; see . The mapping between the data model and the alternative format does not need to be complete, it only needs to be clear and unambiguous. For example, an alternative format might only provide the ability to convey preferences for a subset of the categories of use. A mapping might then define that an unknown preference is associated with other categories.

Security Considerations Preferences are not a security mechanism. addresses what it means to express a preference. Processing a concrete instantiation of the exemplary format described in is subject to the security considerations in .

IANA Considerations This document has no IANA actions.

References Normative References This document describes a set of data types and associated algorithms that are intended to make it easier and safer to define and handle HTTP header and trailer fields, known as "Structured Fields", "Structured Headers", or "Structured Trailers". It is intended for use by specifications of new HTTP fields. This document obsoletes RFC 8941. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References ISO/IEC 10646-1 defines a large character set called the Universal Character Set (UCS) which encompasses most of the world's writing systems. The originally proposed encodings of the UCS, however, were not compatible with many current applications and protocols, and this has led to the development of UTF-8, the object of this memo. UTF-8 has the characteristic of preserving the full US-ASCII range, providing compatibility with file systems, parsers and other software that rely on US-ASCII values but are transparent to other values. This memo obsoletes and replaces RFC 2279. Google Mozilla

Acknowledgments The following individuals made significant contributions to this document: