FEC Framework M. Watson Internet-Draft Netflix, Inc. Intended status: Standards Track A. Begen Expires: May 24, 2011 Cisco November 20, 2010 Forward Error Correction (FEC) Framework draft-ietf-fecframe-framework-11 Abstract This document describes a framework for using Forward Error Correction (FEC) codes with applications in public and private IP networks to provide protection against packet loss. The framework supports applying FEC to arbitrary packet flows over unreliable transport and is primarily intended for real-time, or streaming, media. This framework can be used to define Content Delivery Protocols that provide FEC for streaming media delivery or other packet flows. Content Delivery Protocols defined using this framework can support any FEC scheme (and associated FEC codes) which is compliant with various requirements defined in this document. Thus, Content Delivery Protocols can be defined which are not specific to a particular FEC scheme, and FEC schemes can be defined which are not specific to a particular Content Delivery Protocol. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 24, 2011. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. Watson & Begen Expires May 24, 2011 [Page 1] Internet-Draft FEC Framework November 2010 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Watson & Begen Expires May 24, 2011 [Page 2] Internet-Draft FEC Framework November 2010 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Definitions and Abbreviations . . . . . . . . . . . . . . . . 6 3. Architecture Overview . . . . . . . . . . . . . . . . . . . . 9 4. Procedural Overview . . . . . . . . . . . . . . . . . . . . . 13 4.1. General . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2. Sender Operation . . . . . . . . . . . . . . . . . . . . . 14 4.3. Receiver Operation . . . . . . . . . . . . . . . . . . . . 17 5. Protocol Specification . . . . . . . . . . . . . . . . . . . . 21 5.1. General . . . . . . . . . . . . . . . . . . . . . . . . . 21 5.2. Structure of the Source Block . . . . . . . . . . . . . . 21 5.3. Packet Format for FEC Source Packets . . . . . . . . . . . 21 5.3.1. Generic Explicit Source FEC Payload ID . . . . . . . . 23 5.4. Packet Format for FEC Repair Packets . . . . . . . . . . . 23 5.4.1. Packet Format for FEC Repair Packets over RTP . . . . 23 5.5. FEC Framework Configuration Information . . . . . . . . . 24 5.6. FEC Scheme Requirements . . . . . . . . . . . . . . . . . 26 6. Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 7. Transport Protocols . . . . . . . . . . . . . . . . . . . . . 30 8. Congestion Control . . . . . . . . . . . . . . . . . . . . . . 31 8.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 31 8.2. Normative Requirements . . . . . . . . . . . . . . . . . . 32 9. Security Considerations . . . . . . . . . . . . . . . . . . . 33 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 35 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 36 12.1. Normative references . . . . . . . . . . . . . . . . . . . 36 12.2. Informative references . . . . . . . . . . . . . . . . . . 36 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 38 Watson & Begen Expires May 24, 2011 [Page 3] Internet-Draft FEC Framework November 2010 1. Introduction Many applications have a requirement to transport a continuous stream of packetized data from a source (sender) to one or more destinations (receivers) over networks which do not provide guaranteed packet delivery. Primary examples are real-time, or streaming, media applications such as broadcast, multicast or on-demand audio, video or multimedia. Forward Error Correction (FEC) is a well-known technique for improving reliability of packet transmission over networks which do not provide guaranteed packet delivery, especially in multicast and broadcast applications. The FEC Building Block defined in [RFC5052] provides a framework for definition of Content Delivery Protocols (CDPs) for object delivery (including, primarily, file delivery) which make use of separately defined FEC schemes. Any CDP defined according to the requirements of the FEC Building Block can then easily be used with any FEC scheme which is also defined according to the requirements of the FEC Building Block. Note that the term "Forward Erasure Correction" is sometimes used, erasures being a type of error in which data is lost and this loss can be detected, rather than being received in corrupted form. The focus of this document is strictly on erasures and, the term "Forward Error Correction" is more widely used. This document defines a framework for the definition of CDPs which provide for FEC protection for arbitrary packet flows over unreliable transports such as UDP. As such, this document complements the FEC Building Block of [RFC5052], by providing for the case of arbitrary packet flows over unreliable transport, the same kind of framework as that document provides for object delivery. This document does not define a complete CDP, but rather defines only those aspects that are expected to be common to all CDPs based on this framework. This framework does not define how the flows to be protected are determined, nor how the details of the protected flows and the FEC streams which protect them are communicated from sender to receiver. It is expected that any complete CDP specification which makes use of this framework will address these signaling requirements. However, this document does specify the information which is required by the FEC Framework at the sender and receiver, e.g., details of the flows to be FEC protected, the flow(s) that will carry the FEC protection data and an opaque container for FEC-Scheme-Specific Information. FEC schemes designed for use with this framework must fulfill a number of requirements defined in this document. These requirements are different from those defined in [RFC5052] for FEC schemes for Watson & Begen Expires May 24, 2011 [Page 4] Internet-Draft FEC Framework November 2010 object delivery. However, there is a great deal of commonality and FEC schemes defined for object delivery may be easily adapted for use with the framework defined in this document. Since the RTP protocol is (often) used over UDP, this framework can be applied to RTP flows as well. FEC repair packets may be sent directly over UDP or RTP. The latter approach has the advantage that RTP instrumentation, based on RTP Control Protocol (RTCP), can be used for the repair flow. Additionally, the post-repair RTCP extended reports [RFC5725] may be used to obtain information about the loss rate after FEC recovery. The use of RTP for repair flows is defined for each FEC scheme by defining an RTP payload format for that particular FEC scheme (possibly in the same document). Watson & Begen Expires May 24, 2011 [Page 5] Internet-Draft FEC Framework November 2010 2. Definitions and Abbreviations Application Data Unit (ADU): The unit of source data provided as payload to the transport layer. ADU Flow: A sequence of ADUs associated with a transport-layer flow identifier (such as the standard 5-tuple {Source IP address, source port, destination IP address, destination port, transport protocol}). AL-FEC: Application-layer Forward Error Correction. Application Protocol: Control protocol used to establish and control the source flow being protected, e.g., RTSP. Content Delivery Protocol (CDP): A complete application protocol specification which, through the use of the framework defined in this document, is able to make use of FEC schemes to provide FEC capabilities. FEC Code: An algorithm for encoding data such that the encoded data flow is resilient to data loss. Note that in general FEC codes may also be used to make a data flow resilient to corruption, but that is not considered in this document. FEC Framework: A protocol framework for definition of Content Delivery Protocols using FEC, such as the framework defined in this document. FEC Framework Configuration Information: Information which controls the operation of the FEC Framework. FEC Payload ID: Information which identifies the contents of a packet with respect to the FEC scheme. FEC Repair Packet: At a sender (respectively, at a receiver) a payload submitted to (respectively, received from) the transport protocol containing one or more repair symbols along with a Repair FEC Payload ID and possibly an RTP header. FEC Scheme: A specification which defines the additional protocol aspects required to use a particular FEC code with the FEC Framework. FEC Source Packet: At a sender (respectively, at a receiver) a payload submitted to (respectively, received from) the transport protocol containing an ADU along with an optional Explicit Source FEC Payload ID. Protection Amount: The relative increase in data sent due to the use Watson & Begen Expires May 24, 2011 [Page 6] Internet-Draft FEC Framework November 2010 of FEC. Repair Flow: The packet flow carrying FEC data. Repair FEC Payload ID: An FEC Payload ID specifically for use with repair packets. Source Flow: The packet flow to which FEC protection is to be applied. A source flow consists of ADUs. Source FEC Payload ID: An FEC Payload ID specifically for use with source packets. Source Protocol: A protocol used for the source flow being protected, e.g., RTP. Transport Protocol: The protocol used for transport of the source and repair flows, e.g., UDP and DCCP. The following definitions are aligned with [RFC5052]: Code Rate: The ratio between the number of source symbols and the number of encoding symbols. By definition, the code rate is such that 0 < code rate <= 1. A code rate close to 1 indicates that a small number of repair symbols have been produced during the encoding process. Encoding Symbol: Unit of data generated by the encoding process. With systematic codes, source symbols are part of the encoding symbols. Packet Erasure Channel: A communication path where packets are either dropped (e.g., by a congested router, or because the number of transmission errors exceeds the correction capabilities of the physical-layer codes) or received. When a packet is received, it is assumed that this packet is not corrupted. Repair Symbol: Encoding symbol that is not a source symbol. Source Block: Group of ADUs which are to be FEC protected as a single block. Source Symbol: Unit of data used during the encoding process. Systematic Code: FEC code in which the source symbols are part of the encoding symbols. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", Watson & Begen Expires May 24, 2011 [Page 7] Internet-Draft FEC Framework November 2010 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Watson & Begen Expires May 24, 2011 [Page 8] Internet-Draft FEC Framework November 2010 3. Architecture Overview The FEC Framework is described in terms of an additional layer between the transport layer (e.g., UDP or DCCP) and protocols running over this transport layer. As such, the data path interface between the FEC Framework and both underlying and overlying layers can be thought of as being the same as the standard interface to the transport layer, i.e., the data exchanged consists of datagram payloads each associated with a single ADU flow identified by the standard 5-tuple {Source IP address, source port, destination IP address, destination port, transport protocol}. In the case that RTP is used for the repair flows, the source and repair data can be multiplexed using RTP onto a single UDP flow and needs to be consequently demultiplexed at the receiver. There are various ways in which this multiplexing can be done, for example as described in [RFC4588]. It is important to understand that the main purpose of the FEC Framework architecture is to allocate functional responsibilities to separately documented components in such a way that specific instances of the components can be combined in different ways to describe different protocols. The FEC Framework makes use of an FEC scheme, in a similar sense to that defined in [RFC5052] and uses the terminology of that document. The FEC scheme defines the FEC encoding and decoding, and defines the protocol fields and procedures used to identify packet payload data in the context of the FEC scheme. The interface between the FEC Framework and an FEC scheme, which is described in this document, is a logical one, which exists for specification purposes only. At an encoder, the FEC Framework passes ADUs to the FEC scheme for FEC encoding. The FEC scheme returns repair symbols with their associated Repair FEC Payload IDs, and in some cases Source FEC Payload IDs, depending on the FEC scheme. At a decoder, the FEC Framework passes transport packet payloads (source and repair) to the FEC scheme and the FEC scheme returns additional recovered source packet payloads. This document defines certain FEC Framework Configuration Information which MUST be available to both sender and receiver(s). For example, this information includes the specification of the ADU flows which are to be FEC protected, specification of the ADU flow(s) which will carry the FEC protection (repair) data and the relationship(s) between these source and repair flows (i.e., which source flow(s) are protected by each repair flow(s)). The FEC Framework Configuration Information also includes information fields which are specific to the FEC scheme. This information is analogous to the FEC Object Transmission Information defined in [RFC5052]. Watson & Begen Expires May 24, 2011 [Page 9] Internet-Draft FEC Framework November 2010 The FEC Framework does not define how the FEC Framework Configuration Information for the stream is communicated from sender to receiver. This has to be defined by any CDP specification as described in the following sections. In this architecture we assume that the interface to the transport layer supports the concepts of data units (referred to here as Application Data Units (ADUs)) to be transported and identification of ADU flows on which those data units are transported. Since this is an interface internal to the architecture, we do not specify this interface explicitly. We do require that ADU flows which are distinct from the transport layer point of view (for example, distinct UDP flows as identified by the UDP source/destination addresses/ports) are also distinct on the interface between the transport layer and the FEC Framework. As noted above, RTP flows are a specific example of ADU flows which might be protected by the FEC Framework. From the FEC Framework point of view, RTP source flows are ADU flows like any other, with the RTP header included within the ADU. Depending on the FEC scheme, RTP can also be used as a transport for repair packet flows. In this case an FEC scheme has to define an RTP payload format for the repair data. The architecture outlined above is illustrated in the Figure 1. In this architecture, two (optional) RTP instances are shown, for the source and repair data respectively. This is because the use of RTP for the source data is separate from and independent of the use of RTP for the repair data. The appearance of two RTP instances is more natural when one considers that in many FEC codes, the repair payload contains repair data calculated across the RTP headers of the source packets. Thus, a repair packet carried over RTP starts with an RTP header of its own which is followed (after the Repair Payload ID) by repair data containing bytes which protect the source RTP headers (as well as repair data for the source RTP payloads). Watson & Begen Expires May 24, 2011 [Page 10] Internet-Draft FEC Framework November 2010 +--------------------------------------------+ | Application | +--------------------------------------------+ | | | + - - - - - - - - - - - - - - - - - - - - - - - -+ | +--------------------------------------------+ | | Application Layer | | +--------------------------------------------+ | | | | + -- -- -- -- -- -- -- -- -- -- --+ | | | RTP (Optional) | | | | | |- Configuration/Coordination +- -- -- -- -- -- -- -- -- -- -- -+ | | | | | | ADU flows | | | v | +--------------------------------------------+ +----------------+ | | FEC Framework (This document) |<--->| FEC Scheme | +--------------------------------------------+ +----------------+ | | | | Source | Repair | | | | | +-- -- -- -- --|-- --+ -- -- -- -- -- + -- --+ | | RTP Layer | | RTP Processing | | | | (Optional) | +-- -- -- |- -- -+ | | | +-- -- -- -- -- -- -- |--+ | | | | RTP (De)multiplexing | | | +-- -- -- --- -- -- -- -- -- -- -- -- -- -- -+ | | | +--------------------------------------------+ | | Transport Layer (e.g., UDP) | | +--------------------------------------------+ | | | +--------------------------------------------+ | | IP | | +--------------------------------------------+ | | Content Delivery Protocol | + - - - - - - - - - - - - - - - - - - - - - - - + Figure 1: FEC Framework architecture The content of the transport payload for repair packets is fully defined by the FEC scheme. For a specific FEC scheme, a means MAY be defined for repair data to be carried over RTP, in which case the repair packet payload format starts with the RTP header. This Watson & Begen Expires May 24, 2011 [Page 11] Internet-Draft FEC Framework November 2010 corresponds to defining an RTP payload format for the specific FEC scheme. The use of RTP for repair packets is independent of the protocols used for source packets: if RTP is used for source packets, repair packets may or may not use RTP and vice versa (although it is unlikely that there are useful scenarios where non-RTP source flows are protected by RTP repair flows). FEC schemes are expected to recover entire transport payloads for recovered source packets in all cases. For example, if RTP is used for source flows, the FEC scheme is expected to recover the entire UDP payload, including the RTP header. Watson & Begen Expires May 24, 2011 [Page 12] Internet-Draft FEC Framework November 2010 4. Procedural Overview 4.1. General The mechanism defined in this document does not place any restrictions on the ADUs which can be protected together, except that the ADU is carried over a supported transport protocol (See Section 7). The data can be from multiple source flows that are protected jointly. The FEC Framework handles the source flows as a sequence of source blocks each consisting of a set of ADUs, possibly from multiple source flows which are to be protected together. For example, each source block can be constructed from those ADUs related to a particular segment in time of the flow. At the sender, the FEC Framework passes the payloads for a given block to the FEC scheme for FEC encoding. The FEC scheme performs the FEC encoding operation and returns the following information: o Optionally, FEC Payload IDs for each of the source payloads (encoded according to an FEC-Scheme-Specific format). o One or more FEC repair packet payloads. o FEC Payload IDs for each of the repair packet payloads (encoded according to an FEC-Scheme-Specific format). The FEC Framework then performs two operations. First, it appends the Source FEC Payload IDs, if provided, to each of the ADUs, and sends the resulting packets, known as FEC source packets, to the receiver, and second it places the provided FEC repair packet payloads and corresponding Repair FEC Payload IDs appropriately to construct FEC repair packets and send them to the receiver. This document does not define how the sender determines which ADUs are included in which source blocks or the sending order and timing of FEC source and repair packets. A specific CDP MAY define this mapping or it MAY be left as implementation dependent at the sender. However, a CDP specification MUST define how a receiver determines a minimum length of time that it needs to wait to receive FEC repair packets for any given source block. FEC schemes MAY define limitations on this mapping, such as maximum size of source blocks, but SHOULD NOT attempt to define specific mappings. The sequence of operations at the sender is described in more detail in Section 4.2. At the receiver, original ADUs are recovered by the FEC Framework directly from any FEC source packets received simply by removing the Source FEC Payload ID, if present. The receiver also passes the contents of the received ADUs, plus their FEC Payload IDs to the FEC Watson & Begen Expires May 24, 2011 [Page 13] Internet-Draft FEC Framework November 2010 scheme for possible decoding. If any ADUs related to a given source block have been lost, then the FEC scheme can perform FEC decoding to recover the missing ADUs (assuming sufficient FEC source and repair packets related to that source block have been received). Note that the receiver might need to buffer received source packets to allow time for the FEC repair packets to arrive and FEC decoding to be performed before some or all of the received or recovered packets are passed to the application. If such a buffer is not provided, then the application has to be able to deal with the severe re-ordering of packets that can occur. However, such buffering is CDP and/or implementation-specific and is not specified here. The receiver operation is described in more detail in Section 4.3. The FEC source packets MUST contain information which identifies the source block and the position within the source block (in terms specific to the FEC scheme) occupied by the ADU. This information is known as the Source FEC Payload ID. The FEC scheme is responsible for defining and interpreting this information. This information MAY be encoded into a specific field within the FEC source packet format defined in this specification, called the Explicit Source FEC Payload ID field. The exact contents and format of the Explicit Source FEC Payload ID field are defined by the FEC schemes. Alternatively, the FEC scheme MAY define how the Source FEC Payload ID is derived from other fields within the source packets. This document defines the way that the Explicit Source FEC Payload ID field is appended to source packets to form FEC source packets. The FEC repair packets MUST contain information which identifies the source block and the relationship between the contained repair payloads and the original source block. This is known as the Repair FEC Payload ID. This information MUST be encoded into a specific field, the Repair FEC Payload ID field, the contents and format of which are defined by the FEC schemes. The FEC scheme MAY use different FEC Payload ID field formats for source and repair packets. 4.2. Sender Operation It is assumed that the sender has constructed or received original data packets for the session. These could be carrying any type of data. The following operations, illustrated in Figure 2, for the case of UDP repair flows and Figure 3 for the case of RTP repair flows, describe a possible way to generate compliant source and repair flows: Watson & Begen Expires May 24, 2011 [Page 14] Internet-Draft FEC Framework November 2010 1. ADUs are provided by the application. 2. A source block is constructed as specified in Section 5.2. 3. The source block is passed to the FEC scheme for FEC encoding. The Source FEC Payload ID information of each source packet is determined by the FEC scheme. If required by the FEC scheme the Source FEC Payload ID is encoded into the Explicit Source FEC Payload ID field. 4. The FEC scheme performs FEC encoding, generating repair packet payloads from a source block and a Repair FEC Payload ID field for each repair payload. 5. The Explicit Source FEC Payload IDs (if used), Repair FEC Payload IDs and repair packet payloads are provided back from the FEC scheme to the FEC Framework. 6. The FEC Framework constructs FEC source packets according to Section 5.3 and FEC repair packets according to Section 5.4 using the FEC Payload IDs and repair packet payloads provided by the FEC scheme. 7. The FEC source and repair packets are sent using normal transport-layer procedures. The port(s) and multicast group(s) to be used for FEC repair packets are defined in the FEC Framework Configuration Information. The FEC source packets are sent using the same ADU flow identification information as would have been used for the original source packets if the FEC Framework were not present (for example, in the UDP case, the UDP source and destination addresses and ports on the IP datagram carrying the source packet will be the same whether or not the FEC Framework is applied). Watson & Begen Expires May 24, 2011 [Page 15] Internet-Draft FEC Framework November 2010 +----------------------+ | Application | +----------------------+ | |(1) ADUs | v +----------------------+ +------------------+ | FEC Framework | | | | |-------------------------->| FEC Scheme | |(2) Construct source |(3) Source Block | | | blocks | | (4) FEC Encoding | |(6) Construct FEC |<--------------------------| | | source and repair | | | | packets |(5) Explicit Source FEC | | +----------------------+ Payload IDs +------------------+ | Repair FEC Payload IDs | Repair symbols | |(7) FEC source and repair packets v +----------------------+ | Transport Layer | | (e.g., UDP) | +----------------------+ Figure 2: Sender operation Watson & Begen Expires May 24, 2011 [Page 16] Internet-Draft FEC Framework November 2010 +----------------------+ | Application | +----------------------+ | |(1) ADUs | v +----------------------+ +------------------+ | FEC Framework | | | | |-------------------------->| FEC Scheme | |(2) Construct source |(3) Source Block | | | blocks | | (4) FEC Encoding | |(6) Construct FEC |<--------------------------| | | source packets and| | | | repair payloads |(5) Explicit Source FEC | | +----------------------+ Payload IDs +------------------+ | | Repair FEC Payload IDs | | Repair symbols | | |(7) Source |(7') Repair payloads | packets | | | | + -- -- -- -- -+ | | RTP | | +-- -- -- -- --+ v v +----------------------+ | Transport Layer | | (e.g., UDP) | +----------------------+ Figure 3: Sender operation with RTP repair flows 4.3. Receiver Operation The following describes a possible receiver algorithm, illustrated in Figure 4 and Figure 5 for the case of RTP repair flows, when receiving an FEC source or repair packet: 1. FEC source packets and FEC repair packets are received and passed to the FEC Framework. The type of packet (source or repair) and the source flow to which it belongs (in the case of source packets) is indicated by the ADU flow information which identifies the flow at the transport layer. In the special case that RTP is used for repair packets, and source and repair packets are multiplexed onto the same UDP flow, then RTP demultiplexing is required to demultiplex source and Watson & Begen Expires May 24, 2011 [Page 17] Internet-Draft FEC Framework November 2010 repair flows. However, RTP processing is applied only to the repair packets at this stage; source packets continue to be handled as UDP payloads (i.e., including their RTP headers). 2. The FEC Framework extracts the Explicit Source FEC Payload ID field (if present) from the source packets and the Repair FEC Payload ID from the repair packets. 3. The Explicit Source FEC Payload IDs (if present), Repair FEC Payload IDs, FEC source and repair payloads are passed to the FEC scheme. 4. The FEC scheme uses the received FEC Payload IDs (and derived FEC Source Payload IDs in the case that the Explicit Source FEC Payload ID field is not used) to group source and repair packets into source blocks. If at least one source packet is missing from a source block, and at least one repair packet has been received for the same source block then FEC decoding can be performed in order to recover missing source payloads. The FEC scheme determines whether source packets have been lost and whether enough data for decoding of any or all of the missing source payloads in the source block has been received. 5. The FEC scheme returns the ADUs to the FEC Framework in the form of source blocks containing received and decoded ADUs and indications of any ADUs which were missing and could not be decoded. 6. The FEC Framework passes the received and recovered ADUs to the application. The description above defines functionality responsibilities but does not imply a specific set of timing relationships. Source packets which are correctly received and those which are reconstructed MAY be delivered to the application out of order and in a different order from the order of arrival at the receiver. Alternatively, buffering and packet re-ordering MAY be applied to re-order received and reconstructed source packets into the order they were placed into the source block, if that is necessary according to the application. Watson & Begen Expires May 24, 2011 [Page 18] Internet-Draft FEC Framework November 2010 +----------------------+ | Application | +----------------------+ ^ | |(6) ADUs | +----------------------+ +------------------+ | FEC Framework | | | | |<---------------------------| FEC Scheme | |(2)Extract FEC Payload|(5) ADUs | | | IDs and pass IDs & | | (4) FEC Decoding | | payloads to FEC |--------------------------->| | | scheme |(3) Explicit Source FEC | | +----------------------+ Payload IDs +------------------+ ^ Repair FEC Payload IDs | Source payloads | Repair payloads | |(1) FEC source and repair packets | +----------------------+ | Transport Layer | | (e.g., UDP) | +----------------------+ Figure 4: Receiver operation Watson & Begen Expires May 24, 2011 [Page 19] Internet-Draft FEC Framework November 2010 +----------------------+ | Application | +----------------------+ ^ | |(6) ADUs | +----------------------+ +------------------+ | FEC Framework | | | | |<---------------------------| FEC Scheme | |(2)Extract FEC Payload|(5) ADUs | | | IDs and pass IDs & | | (4) FEC Decoding | | payloads to FEC |--------------------------->| | | scheme |(3) Explicit Source FEC | | +----------------------+ Payload IDs +------------------+ ^ ^ Repair FEC Payload IDs | | Source payloads | | Repair payloads | | |Source |Repair payloads |packets | | | +-- |- -- -- -- -- -- -+ |RTP| | RTP Processing | | | +-- -- -- --|-- -+ | +-- -- -- -- -- |--+ | | | RTP Demux | | +-- -- -- -- -- -- -- -+ ^ |(1) FEC source and repair packets | +----------------------+ | Transport Layer | | (e.g., UDP) | +----------------------+ Figure 5: Receiver operation with RTP repair flows Note that the above procedure might result in a situation in which not all ADUs are recovered. Watson & Begen Expires May 24, 2011 [Page 20] Internet-Draft FEC Framework November 2010 5. Protocol Specification 5.1. General This section specifies the protocol elements for the FEC Framework. Three components of the protocol are defined in this document and are described in the following sections: 1. Construction of a source block from ADUs. The FEC code will be applied to this source block to produce the repair payloads. 2. A format for packets containing source data. 3. A format for packets containing repair data. The operation of the FEC Framework is governed by certain FEC Framework Configuration Information, which is defined in this section. A complete protocol specification that uses this framework MUST specify the means to determine and communicate this information between sender and receiver. 5.2. Structure of the Source Block The FEC Framework and FEC scheme exchange ADUs in the form of source blocks. A source block is generated by the FEC Framework from an ordered sequence of ADUs. The allocation of ADUs to blocks is dependent on the application. Note that some ADUs may not be included in any block. Each source block provided to the FEC scheme consists of an ordered sequence of ADUs where the following information is provided for each ADU: o A description of the source flow with which the ADU is associated with. o The ADU itself. o The length of the ADU. 5.3. Packet Format for FEC Source Packets The packet format for FEC source packets MUST be used to transport the payload of an original source packet. As depicted in Figure 6, it consists of the original packet, optionally followed by the Explicit Source FEC Payload ID field. The FEC scheme determines whether the Explicit Source FEC Payload ID field is required. This determination is specific to each ADU flow. Watson & Begen Expires May 24, 2011 [Page 21] Internet-Draft FEC Framework November 2010 +------------------------------------+ | IP Header | +------------------------------------+ | Transport Header | +------------------------------------+ | Application Data Unit | +------------------------------------+ | Explicit Source FEC Payload ID | +------------------------------------+ Figure 6: Structure of the FEC packet format for FEC source packets The FEC source packets MUST be sent using the same ADU flow as would have been used for the original source packets if the FEC Framework were not present. The transport payload of the FEC source packet MUST consist of the ADU followed by the Explicit Source FEC Payload ID field, if required. The Explicit Source FEC Payload ID field contains information required to associate the source packet with a source block and for the operation of the FEC algorithm, and is defined by the FEC scheme. The format of the Source FEC Payload ID field is defined by the FEC scheme. In the case that the FEC scheme or CDP defines a means to derive the Source FEC Payload ID from other information in the packet (for example a sequence number used by the application protocol), then the Source FEC Payload ID field is not included in the packet. In this case, the original source packet and FEC source packet are identical. In applications where avoidance of IP packet fragmentation is a goal, CDPs SHOULD consider the Explicit Source FEC Payload ID size when determining the size of ADUs that will be delivered using the FEC Framework. This is because the addition of the Explicit Source FEC Payload ID increases the packet length. The Explicit Source FEC Payload ID is placed at the end of the packet so that in the case that Robust Header Compression (ROHC) [RFC3095] or other header compression mechanisms are used and in the case that a ROHC profile is defined for the protocol carried within the transport payload (for example RTP), then ROHC will still be applied for the FEC source packets. Applications that are used with this framework need to consider that FEC schemes can add this Explicit Source FEC Payload ID and thereby increase the packet size. In many applications, support for FEC is added to a pre-existing protocol and in this case use of the Explicit Source FEC Payload ID can break backwards compatibility, since source packets are modified. Watson & Begen Expires May 24, 2011 [Page 22] Internet-Draft FEC Framework November 2010 5.3.1. Generic Explicit Source FEC Payload ID In order to apply FEC protection using multiple FEC schemes to a single source flow, all schemes have to use the same Explicit Source FEC Payload ID format. In order to enable this, it is RECOMMENDED that FEC schemes support the Generic Explicit Source FEC Payload ID format described below. The Generic Explicit Source FEC Payload ID has a length of two octets and consists of an unsigned packet sequence number in network-byte order. The allocation of sequence numbers to packets is independent of any FEC scheme and of the source block construction, except that the use of this sequence number places a constraint on source block construction. Source packets within a given source block MUST have consecutive sequence numbers (where consecutive includes wrap-around from the maximum value which can be represented in two octets (65535) to 0). Sequence numbers SHOULD NOT be reused until all values in the sequence number space have been used. 5.4. Packet Format for FEC Repair Packets The packet format for FEC repair packets is shown in Figure 7. The transport payload consists of a Repair FEC Payload ID field followed by repair data generated in the FEC encoding process. +------------------------------------+ | IP Header | +------------------------------------+ | Transport Header | +------------------------------------+ | Repair FEC Payload ID | +------------------------------------+ | Repair Symbols | +------------------------------------+ Figure 7: Packet format for repair packets The Repair FEC Payload ID field contains information required for the operation of the FEC algorithm at the receiver. This information is defined by the FEC scheme. The format of the Repair FEC Payload ID field is defined by the FEC scheme. 5.4.1. Packet Format for FEC Repair Packets over RTP For FEC schemes which specify the use of RTP for repair packets, the packet format for repair packets includes an RTP header as shown in Figure 8. Watson & Begen Expires May 24, 2011 [Page 23] Internet-Draft FEC Framework November 2010 +------------------------------------+ | IP header | +------------------------------------+ | Transport Header (UDP) | +------------------------------------+ | RTP Header | +------------------------------------+ | Repair FEC Payload ID | +------------------------------------+ | Repair Symbols | +------------------------------------+ Figure 8: Packet format for repair packets 5.5. FEC Framework Configuration Information The FEC Framework Configuration Information is information that the FEC Framework needs in order to apply FEC protection to the ADU flows. A complete CDP specification that uses the framework specified here MUST include details of how this information is derived and communicated between sender and receiver. The FEC Framework Configuration Information includes identification of the set of source flows. For example, in the case of UDP, each source flow is uniquely identified by a tuple {Source IP address, source UDP port, destination IP address, destination UDP port}. In some applications some of these fields can contain wildcards, so that the flow is identified by a subset of the fields. In particular, in many applications the limited tuple {Destination IP address, destination UDP port} is sufficient. A single instance of the FEC Framework provides FEC protection for packets of the specified set of source flows, by means of one or more packet flows consisting of repair packets. The FEC Framework Configuration Information includes, for each instance of the FEC Framework: 1. Identification of the repair flows. 2. For each source flow protected by the repair flow(s): A. Definition of the source flow. B. An integer identifier for this flow definition (i.e., tuple). This identifier MUST be unique amongst all source flows that are protected by the same FEC repair flow. Integer identifiers can be allocated starting from zero and increasing by one for each flow. However, any random (but Watson & Begen Expires May 24, 2011 [Page 24] Internet-Draft FEC Framework November 2010 still unique) allocation is also possible. A source flow identifier need not be carried in source packets since source packets are directly associated with a flow by virtue of their packet headers. 3. The FEC Encoding ID, identifying the FEC scheme. 4. The length of the Explicit Source FEC Payload ID (in octets). 5. Zero or more FEC-Scheme-Specific Information (FSSI) elements, each consisting of a name and a value where the valid element names and value ranges are defined by the FEC scheme. Multiple instances of the FEC Framework, with separate and independent FEC Framework Configuration Information, can be present at a sender or receiver. A single instance of the FEC Framework protects packets of the source flows identified in (2) above, i.e., all packets sent on those flows MUST be FEC source packets as defined in Section 5.3. A single source flow can be protected by multiple instances of the FEC Framework. The integer flow identifier identified in (2b) above is a shorthand to identify source flows between the FEC Framework and the FEC scheme. The reason for defining this as an integer, and including it in the FEC Framework Configuration Information is so that the FEC scheme at the sender and receiver can use it to identify the source flow with which a recovered packet is associated. The integer flow identifier can therefore take the place of the complete flow description (e.g., UDP 4-tuple). Whether and how this flow identifier is used is defined by the FEC scheme. Since repair packets can provide protection for multiple source flows, repair packets would either not carry the identifier at all or can carry multiple identifiers. However, in any case, the flow identifier associated with a particular source packet can be recovered from the repair packets as part of an FEC decoding operation. A single FEC repair flow provides repair packets for a single instance of the FEC Framework. Other packets MUST NOT be sent within this flow, i.e., all packets in the FEC repair flow MUST be FEC repair packets as defined in Section 5.4 and MUST relate to the same FEC Framework instance. In the case that RTP is used for repair packets, the identification of the repair packet flow can also include the RTP payload type to be used for repair packets. Watson & Begen Expires May 24, 2011 [Page 25] Internet-Draft FEC Framework November 2010 FSSI includes the information that is specific to the FEC scheme used by the CDP. FSSI is used to communicate the information that cannot be adequately represented otherwise and is essential for proper FEC encoding and decoding operations. The motivation behind separating the FSSI required only by the sender (which is carried in Sender-Side FEC-Scheme-Specific Information (SS-FSSI) container) from the rest of the FSSI is to provide the receiver or the third party entities a means of controlling the FEC operations at the sender. Any FSSI other than the one solely required by the sender MUST be communicated via the FSSI container. The variable-length SS-FSSI and FSSI containers transmit the information in textual representation and contain zero or more distinct elements, whose descriptions are provided by the fully- specified FEC schemes. For the CDPs that choose the Session Description Protocol (SDP) [RFC4566] as their session description protocol, the ABNF [RFC5234] syntax for the SS-FSSI and FSSI containers is provided in Section 4.5 of [I-D.ietf-fecframe-sdp-elements]. 5.6. FEC Scheme Requirements In order to be used with this framework, an FEC scheme MUST be capable of processing data arranged into blocks of ADUs (source blocks). A specification for a new FEC scheme MUST include the following: 1. The FEC Encoding ID value that uniquely identifies the FEC scheme. This value MUST be registered with IANA as described in Section 10. 2. The type, semantics and encoding format of the Repair FEC Payload ID. 3. The name, type, semantics and text value encoding rules for zero or more FEC-Scheme-Specific Information elements. 4. A full specification of the FEC code. This specification MUST precisely define the valid FEC-Scheme- Specific Information values, the valid FEC Payload ID values and the valid packet payload sizes (where packet payload refers to the space within a packet dedicated to carrying encoding symbols). Furthermore, given a source block as defined in Section 5.2, Watson & Begen Expires May 24, 2011 [Page 26] Internet-Draft FEC Framework November 2010 valid values of the FEC-Scheme-Specific Information, a valid Repair FEC Payload ID value and a valid packet payload size, the specification MUST uniquely define the values of the encoding symbols to be included in the repair packet payload of a packet with the given Repair FEC Payload ID value. A common and simple way to specify the FEC code to the required level of detail is to provide a precise specification of an encoding algorithm which, given a source block, valid values of the FEC-Scheme-Specific Information, a valid Repair FEC Payload ID value and a valid packet payload size as input produces the exact value of the encoding symbols as output. 5. A description of practical encoding and decoding algorithms. This description need not be to the same level of detail as for the encoding above, however it has to be sufficient to demonstrate that encoding and decoding of the code is both possible and practical. FEC scheme specifications MAY additionally define the following: 1. Type, semantics and encoding format of an Explicit Source FEC Payload ID. Whenever an FEC scheme specification defines an 'encoding format' for an element, this has to be defined in terms of a sequence of bytes which can be embedded within a protocol. The length of the encoding format MUST either be fixed or it MUST be possible to derive the length from examining the encoded bytes themselves. For example, the initial bytes can include some kind of length indication. FEC scheme specifications SHOULD use the terminology defined in this document and SHOULD follow the following format: 1. Introduction 2. Formats and Codes 2.1 Source FEC Payload ID(s) Watson & Begen Expires May 24, 2011 [Page 27] Internet-Draft FEC Framework November 2010 2.2 Repair FEC Payload ID 2.3 FEC Framework Configuration Information 3. Procedures 4. FEC Code Specification Specifications can include additional sections including examples. Each FEC scheme MUST be specified independently of all other FEC schemes; for example, in a separate specification or a completely independent section of larger specification (except, of course, a specification of one FEC scheme can include portions of another by reference). Where an RTP Payload Format is defined for repair data for a specific FEC scheme, the RTP Payload Format and the FEC scheme can be specified within the same document. Watson & Begen Expires May 24, 2011 [Page 28] Internet-Draft FEC Framework November 2010 6. Feedback Many applications require some kind of feedback on transport performance. E.g., how much data arrived at the receiver, at what rate and when? When FEC is added to such applications, feedback mechanisms can also need to be enhanced to report on the performance of the FEC. E.g., how much lost data was recovered by the FEC? When used to provide instrumentation for engineering purposes, it is important to remember that FEC is generally applied to relatively small blocks of data (in the sense that each block is transmitted over a relatively small period of time). Thus, feedback information that is averaged over longer periods of time will likely not provide sufficient information for engineering purposes. More detailed feedback over shorter time scales might be preferred. For example, for applications using RTP transport, see [RFC5725]. Applications which used feedback for congestion control purposes MUST calculate such feedback on the basis of packets received before FEC recovery is applied. If this requirement conflicts with other uses of the feedback information then the application MUST be enhanced to support both information calculated pre- and post- FEC recovery. This is to ensure that congestion control mechanisms operate correctly based on congestion indications received from the network, rather than on post-FEC recovery information which would give an inaccurate picture of congestion conditions. New applications which require such feedback SHOULD use RTP/RTCP [RFC3550]. Watson & Begen Expires May 24, 2011 [Page 29] Internet-Draft FEC Framework November 2010 7. Transport Protocols This framework is intended to be used to define CDPs that operate over transport protocols providing an unreliable datagram service, including in particular the User Datagram Protocol (UDP) and the Datagram Congestion Control Protocol (DCCP). Watson & Begen Expires May 24, 2011 [Page 30] Internet-Draft FEC Framework November 2010 8. Congestion Control This section starts with some informative background on the motivation of the normative requirements for congestion control, which are spelled out in Section 8.2. 8.1. Motivation o The enforcement of congestion control principles has gained a lot of momentum in the IETF over the recent years. While the need for congestion control over the open Internet is unquestioned, and the goal of TCP friendliness is generally agreed for most (but not all) applications, the subject of congestion detection and measurement in heterogeneous networks can hardly be considered as solved. Most congestion control algorithms detect and measure congestion by taking (primarily or exclusively) the packet loss rate into account. This appears to be inappropriate in environments where a large percentage of the packet losses are the result of link-layer errors and independent of the network load. o The authors of this document are primarily interested in applications where the application reliability requirements and end-to-end reliability of the network differ, such that it warrants higher-layer protection of the packet stream, e.g., due to the presence of unreliable links in the end-to-end path and where real-time, scalability or other constraints prohibit the use of higher-layer (transport or application) feedback. A typical example for such applications is multicast and broadcast streaming or multimedia transmission over heterogeneous networks. In other cases, application reliability requirements can be so high that the required end-to-end reliability will be difficult to achieve. Furthermore, the end-to-end network reliability is not necessarily known in advance. o This FEC Framework is not defined, nor intended, as a QoS enhancement tool to combat losses resulting from highly congested networks. It should not be used for such purposes. o In order to prevent such mis-use, one approach is to leave standardization to bodies most concerned with the problem described above. However, the IETF defines base standards used by several bodies, including DVB, 3GPP, 3GPP2, all of which appear to share the environment and the problem described. o Another approach is to write a clear applicability statement. For example, one could restrict the use of this framework to networks with certain loss characteristics (e.g., wireless links). However, there can be applications where the use of FEC is Watson & Begen Expires May 24, 2011 [Page 31] Internet-Draft FEC Framework November 2010 justified to combat congestion-induced packet losses - particularly in lightly loaded networks, where congestion is the result of relatively rare random peaks in instantaneous traffic load - thereby intentionally violating congestion control principles. One possible example for such an application could be a no-matter-what, brute-force FEC protection of a traffic generated as an emergency signal. o A third approach is to require at a minimum that the use of this framework with any given application, in any given environment, does not cause congestion issues which the application alone would not itself cause, i.e., the use of this framework must not make things worse. o Taking above considerations into account, Section 8.2 specifies a small set of constraints for the FEC, which are mandatory for all senders compliant with this FEC Framework. Further restrictions can be imposed by certain CDPs. 8.2. Normative Requirements o The bandwidth of FEC repair data MUST NOT exceed the bandwidth of the original source data being protected (without the possible addition of an Explicit Source FEC Payload ID). This disallows the (static or dynamic) use of excessively strong FEC to combat high packet loss rates, which can otherwise be chosen by naively implemented dynamic FEC-strength selection mechanisms. We acknowledge that there are a few exotic applications, e.g., IP traffic from space-based senders, or senders in certain hardened military devices, which could warrant a higher FEC strength. However, in this specification we give preference to the overall stability and network friendliness of average applications. o Whenever the source data rate is adapted due to the operation of congestion control mechanisms, the FEC repair data rate MUST be similarly adapted. Watson & Begen Expires May 24, 2011 [Page 32] Internet-Draft FEC Framework November 2010 9. Security Considerations The application of FEC protection to a stream does not provide any kind of security protection. If security services are required for the stream, then they MUST either be applied to the original source data before FEC protection is applied, or to both the source and repair data, after FEC protection has been applied. If integrity protection is applied to source packets before FEC protection is applied, and no further integrity protection is applied to repair packets, then a denial-of-service attack is possible if an attacker is in a position to inject fake repair transport payloads. If received by a receiver, such fake repair transport payloads could cause incorrect FEC decoding resulting in incorrect ADUs being passed up to the application protocol. A similar attack is possible if an attacker is in a position to inject fake FEC Framework Configuration Information or fake FEC Payload IDs. Such incorrect decoded ADUs would then be detected by the source integrity protection and discarded, resulting in partial or complete denial of service. Therefore, in such environments, integrity protection MUST also be applied to the FEC repair transport payloads, FEC Framework Configuration Information and FEC Payload IDs, for example using IPSec to integrity protect all packets. Receivers MUST also verify the integrity of source symbols before including the source symbols into the source block. It is possible that multiple streams with different confidentiality requirements (for example, the streams may be visible to different sets of users) can be FEC protected by a single repair stream. This scenario is not recommended, since resources will be used to distribute and FEC decode encrypted data which cannot then be decrypted by at least some receivers. However, in this scenario, confidentiality protection MUST be applied before FEC encoding of the streams, otherwise repair transport payload may be used by a receiver to decode unencrypted versions of source streams which they do not have permissions to access. Watson & Begen Expires May 24, 2011 [Page 33] Internet-Draft FEC Framework November 2010 10. IANA Considerations FEC schemes for use with this framework are identified in protocols using FEC Encoding IDs. Values of FEC Encoding IDs are subject to IANA registration. For this purposes, this document creates a new registry called FEC Framework (FECFRAME) FEC Encoding IDs. The values that can be assigned within the FEC Framework (FECFRAME) FEC Encoding ID registry are numeric indexes in the range [0, 255], boundaries included. Assignment requests are granted on an IETF Consensus basis as defined in [RFC5226]. Section 5.6 defines explicit requirements that documents defining new FEC Encoding IDs should meet. Watson & Begen Expires May 24, 2011 [Page 34] Internet-Draft FEC Framework November 2010 11. Acknowledgments This document is based in part on [I-D.watson-tsvwg-fec-sf] and so thanks are due to the additional authors of that document, Mike Luby, Magnus Westerlund and Stephan Wenger. That document was in turn based on the FEC Streaming Protocol defined by 3GPP in [MBMSTS], and thus, thanks are also due to the participants in 3GPP TSG SA Working Group 4. Further thanks are due to the members of the FECFRAME Working Group for their comments and review. Watson & Begen Expires May 24, 2011 [Page 35] Internet-Draft FEC Framework November 2010 12. References 12.1. Normative references [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3095] Bormann, C., Burmeister, C., Degermark, M., Fukushima, H., Hannu, H., Jonsson, L-E., Hakenberg, R., Koren, T., Le, K., Liu, Z., Martensson, A., Miyazaki, A., Svanbro, K., Wiebke, T., Yoshimura, T., and H. Zheng, "RObust Header Compression (ROHC): Framework and four profiles: RTP, UDP, ESP, and uncompressed", RFC 3095, July 2001. [RFC5052] Watson, M., Luby, M., and L. Vicisano, "Forward Error Correction (FEC) Building Block", RFC 5052, August 2007. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008. [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. 12.2. Informative references [I-D.watson-tsvwg-fec-sf] Watson, M., "Forward Error Correction (FEC) Streaming Framework", draft-watson-tsvwg-fec-sf-00 (work in progress), July 2005. [RFC5725] Begen, A., Hsu, D., and M. Lague, "Post-Repair Loss RLE Report Block Type for RTP Control Protocol (RTCP) Extended Reports (XRs)", RFC 5725, February 2010. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. Hakenberg, "RTP Retransmission Payload Format", RFC 4588, July 2006. [I-D.ietf-fecframe-sdp-elements] Begen, A., "Session Description Protocol Elements for FEC Watson & Begen Expires May 24, 2011 [Page 36] Internet-Draft FEC Framework November 2010 Framework", draft-ietf-fecframe-sdp-elements-11 (work in progress), October 2010. [MBMSTS] 3GPP, "Multimedia Broadcast/Multicast Service (MBMS); Protocols and codecs", 3GPP TS 26.346, April 2005. Watson & Begen Expires May 24, 2011 [Page 37] Internet-Draft FEC Framework November 2010 Authors' Addresses Mark Watson Netflix, Inc. 100 Winchester Circle Los Gatos, CA 95032 USA Email: watsonm@netflix.com Ali Begen Cisco 181 Bay Street Toronto, ON M5J 2T3 Canada Email: abegen@cisco.com Watson & Begen Expires May 24, 2011 [Page 38]