| Internet-Draft | Coding and congestion | February 2021 |
| Kuhn, et al. | Expires 20 August 2021 | [Page] |
Forward Erasure Correction (FEC) is a reliability mechanism that is distinct and separate from the retransmission logic in reliable transfer protocols such as TCP. Using FEC coding can help deal with losses at the end of transfers or with networks having non-congestion losses. However, FEC coding mechanisms should not hide congestion signals. This memo offers a discussion of how FEC coding and congestion control can coexist. Another objective is to encourage the research community to also consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems.¶
This document is the product of the Coding for Efficient Network Communications Research Group (NWCRG). The scope of the document is end-to-end communications: FEC coding for tunnels is out-of-the scope of the document.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 20 August 2021.¶
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶
There are cases where deploying FEC coding improves the performance of a transmission. As an example, it may take time for the sender to detect transfer tail losses (losses that occur at the end of a transfer, where e.g., TCP obtains no more ACKs to repair the loss via retransmission quickly). Allowing the receiver to recover such losses instead of having to rely on a retransmission could improve the experience of applications using short flows. Another example is a network where non-congestion losses are persistent and prevent a sender from exploiting the link capacity.¶
Coding is a reliability mechanism that is distinct and separate from the loss detection of congestion controls. [RFC5681] defines TCP as a loss-based congestion control; since FEC coding repairs such losses, blindly applying it may easily lead to an implementation that also hides a congestion signal from the sender. It is important to ensure that such information hiding does not occur.¶
FEC coding and congestion control can be seen as two separate channels. In practice, implementations may mix the signals that are exchanged on these channels. This memo offers a discussion of how FEC coding and congestion control coexist. Another objective is to encourage the research community also to consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems. This document does not aim at proposing guidelines for characterizing FEC coding solutions.¶
We consider an end-to-end unicast data transfer with FEC coding at the application (above the transport), within the transport or directly below the transport. The typical application scenario considered in the current version of the document is a client browsing the web or watching a live video.¶
This document represents the collaborative work and consensus of the Coding for Efficient Network Communications Research Group (NWCRG); it is not an IETF product and is not a standard. The document follows the terminology proposed in the taxonomy document [RFC8406].¶
Figure 1 presents the notations that will be used in this document and introduces the Congestion Control (CC) and Forward Erasure Correction (FEC) channels. The Congestion Control channel carries source packets from a sender to a receiver, and packets signaling information about the network (number of packets received vs. lost, ECN marks, etc.) from the receiver to the sender. The Forward Erasure Correction channel carries repair symbols (from the sender to the receiver) and potential information signaling which packets have been repaired (from the receiver to the sender). There are cases where these channels are not separated.¶
SENDER RECEIVER +------+ +------+ | | ----- source packets ---->| | | CC | | CC | | | <--- network information ---| | +------+ +------+ +------+ +------+ | | ----- repair symbols ---->| | | FEC | | FEC | | | <--- info: repaired symbols --| | +------+ +------+
Inside a host, the CC and FEC entities can be regarded as conceptually separate:¶
| ^ | ^ | source | coding |packets | sending | packets | rate |requirements | rate (or v | v | window) +---------------+source +-----------------+ | FEC |and/or | CC | | |repair | |source | |symbols | |packets +---------------+==> +-----------------+==> ^ ^ | signaling about | network | losses and/or | information | repaired symbols
| | | source and/or | packets | repair symbols | v v +---------------+ +-----------------+ | FEC |signaling | CC | | |repaired | |network | |symbols | |information +---------------+==> +-----------------+==>
Figure 2 and Figure 3 provide more details than Figure 1. Some elements are introduced:¶
The inputs to FEC (incoming data packets without repair symbols, and signaling from the receiver about losses and/or repaired symbols) are distinct from the inputs to CC. The latter calculates a sending rate or window from network information, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower rate bounds, periods of quiescence, or a priority. It is not clear that the ACK signals feeding into a congestion control algorithm are useful to FEC in their raw form, and vice versa - information about repaired blocks may be quite irrelevant to a CC algorithm.¶
The choice of the adequate transport layer may be related to application requirements and the services offered by a transport protocol [RFC8095]:¶
The document considers one application layer stream as input packets. The application layer may exploit several paths above both the FEC and transport layers; this case is not further described in the document.¶
The contract between the client and the operator may guarantee a minimum data-rate (e.g. mobile networks). However, for residential accesses, the data-rate can be guaranteed for the customer premises equipment, but not necessarily for the client. The quality of service that guarantees fairness between the different clients can be seen as a policy concern [I-D.briscoe-tsvarea-fair].¶
While flow level fairness does not embody the actual application level fairness, the share of available capacity between single flows can help assess when one flow starves the other. While past efforts have focused on achieving fairness, quantifying and limiting harm caused by new algorithm (or algorithms with coding) is more practical [BEYONDJAIN]. Clients may share a bottleneck that may not be ruled by a quality of service mechanism, e.g. in case of:¶
This document considers fairness as an index to quantify the impact of the addition of coded flows on non-coded flows when they share the same bottleneck. This document does not aim at contributing to the definition of fairness at a wider scale. This document assumes that the non-coded flows respond to congestion signals from the network.¶
| source ^ source
| packets | packets
v |
+-------------+ +-------------+
|FEC | signaling|FEC |
| | repaired| |
| | symbols| |
| | <==| |
+-------------+ +-------------+
| source ^ ^ source
| and/or | sending | and/or
| repair | rate | repair
| symbols | (or window) | symbols
v | |
+-------------+ +-------------+
|Transport |source network|Transport |
|(incl. CC) |and/or information| |
| |repair <==| |
| |packets | |
+-------------+==> +-------------+
SENDER RECEIVER
Figure 4 present an architecture where FEC operates on top of the transport.¶
The advantage of this approach is that the FEC overhead does not contribute to congestion in the network. When congestion control is implemented at the transport layer, the repair symbols are sent following the congestion window. This approach can result in improved quality of experience for latency sensitive applications such as VoIP or any not-fully reliable services.¶
This approach requires that the transport protocol does not implement a fully reliable data transfer service (e.g., based on lost packet retransmission). QUIC with unreliable datagram extension [I-D.ietf-quic-datagram] is an example of a protocol for which this approach is relevant. In cases where QUIC traffic is blocked and a fallback to TCP mechanism is proposed, there is a risk for bad interactions between the TCP reliability mechanisms and coding schemes. For reliable transfers, coding usage does not guarantee better performance and would mainly reduce goodput for large file transfers. Moreover, the recovered symbols may not be known to the transport.¶
The addition of coding within the flow does not impact on the interaction between coded and non-coded flows. This interaction would mainly depend on the congestion controls embedded in each host.¶
The congestion control may not be able to differentiate repair symbols from actual source packets. The relevance of adding coding at the application layer is related to the needs of the application. For real-time applications, this approach may reduce the number of retransmission. The usage of a non-reliable transport is more adequate in this case.¶
The coding rate applied at the application layer mainly depends on the available capacity given by the congestion control underneath. Adapting the coding rate to the minimum required data rate of the application may reduce packet losses and improve the quality of experience.¶
The discussion depends on application needs. The only case where adding useless repair symbols does not obviously result in reduced goodput is when the application needs a limited amount of goodput (e.g., VoIP traffic). In this case, the useless repair symbols would only impact the amount of data generated in the network. Extra data in the network can, however, increase the likelihood of increasing delay and/or packet loss, which could provoke a congestion control reaction that would degrade goodput.¶
Whether the transport protocol includes a reordering mechanism or not, the FEC mechanism does not require to implement a reordering mechanism if the application does not require it. However, if the application requires in-order delivery of packets, a reordering mechanism at the client is required.¶
The application may require partial reliability only. In this case, the coding rates of the FEC mechanisms could be adapted accordingly based on inputs of the application and the trade-off between latency and packet loss.¶
Whether the transport protocol exploits multiple paths or not does not have an impact on the FEC mechanism.¶
| source | sending ^ source
| packets | rate | packets
v v |
+------------+ +------------+
| Transport | | Transport |
| | | |
| +---+ +--+ | signaling| +---+ +--+ |
| |FEC| |CC| | repaired| |FEC| |CC| |
| +---+ +--+ |source symbols| +---+ +--+ |
| |and/or <==| |
| |repair network| |
| |packets information| |
+------------+ ==> <==+------------+
SENDER RECEIVER
Figure 5 presents an architecture where FEC operates within the transport. The repair symbols are sent within what the congestion window allows, such as in [CTCP].¶
The advantage of this approach allows a joint optimization of the CC and the FEC. Moreover, the transmission of repair symbols does not add congestion in potentially congested networks but helps repair lost packets (such as tail losses).¶
For reliable transfers, including redundancy reduces goodput for large file transfers but the amount of repair symbols can be adapted, e.g. depending on the congestion window size. There is a trade-off between the cost of capacity used to transmit source packets and the benefits brought out by transmitting repair symbols (e.g. unlocking the receive buffer if this is limiting). The coding ratio needs to be carefully designed. For small files, sending repair symbols when there is no more data to transmit could help to reduce the transfer time. In general, sending repair symbols could avoid a silent period between the transmission of the last packet in the send buffer and 1) firing the retransmission of lost packets, or 2) the transmission of new packets.¶
The addition of coding within the flow may impact the congestion control mechanism and hide congestion losses. Specific interaction between congestion controls and coding schemes can be proposed (see Section 4.2, Section 4.3 and Section 4.4). If no specific interaction is introduced, the coding scheme may hide congestion losses from the congestion controller and the description of Section 5 may apply.¶
If the FEC and CC channels are decoupled, the endpoint may exploit different protocols for each channel. The channels may be coupled and one single protocol may be exploited. In both cases, the receiver can differentiate source packets and repair symbols. The receiver may indicate both the number of source packets received and repair symbols that were actually useful in the recovery process of packets.¶
There is an important flexibility in the trade-off, inherent to the use of coding, between (1) reducing goodput when useless repair symbols are transmitted and (2) helping to recover sooner from transmission and congestion losses. The receiver may indicate to the sender the number of packets that have been received or recovered. The sender may exploit this information to tune the coding ratio. As one example of flexibility of this case, coupling an increased transmission rate with an increasing or decreasing coding rate could be envisioned. A server may use an increasing coding rate as a probe of the channel capacity and adapt the congestion control transmission rate.¶
The sender may exploit the information given by the receiver to reduce the number of useless repair symbols and the resulting goodput reduction.¶
The application may require in-order delivery of packets. In this case, both FEC and transport layer mechanisms should guarantee that packets are delivered in order. If partial ordering is requested by the application, both the FEC and transport could release the constraints related to in-order delivery of packets : reordering mechanisms at the receiver may not be necessary.¶
The application may require partial reliability. In this case, the transport and FEC mechanisms could be conjointly designed. As one example, the reliability offered by FEC may be sufficient and no retransmission required. This depends on application requirements and the trade-off between latency and loss.¶
The sender may adapt the coding rate of each of the single subpaths, whether the congestion control is coupled or not. There is an important flexibility on how the coding rate is tuned depending on the characteristics of each subpath.¶
| source | sending rate ^ source
| packets | (or window) | packets
v v |
+--------------+ +--------------+
|Transport | network|Transport |
|(including CC)| information| |
| | <==| |
+--------------+ +--------------+
| source packets ^ source packets
v |
+--------------+ +--------------+
| FEC |source | FEC |
| |and/or signaling| |
| |repair repaired| |
| |symbols symbols| |
| |==> <==| |
+--------------+ +--------------+
SENDER RECEIVER
Figure 6 presents an architecture where FEC is applied end-to-end below the transport layer, but above the link layer. Note that it is common to apply FEC at the link layer, in which it contributes to the total capacity that a link exposes to upper layers. This application of FEC is out of scope of this document. In the scenario considered here, the repair symbols are sent on top of what is allowed by the congestion control.¶
Including redundancy adds traffic without reducing goodput but leads to potential fairness issues. The effective bitrate is indeed higher than the CC's computed fair share due to the sending of repair symbols and the losses are hidden from the transport. This may cause a problem for loss-based congestion detection, but it is not a problem for delay-based congestion detection.¶
The advantage of this approach is that it can result in performance gains when there are persistent transmission losses along the path.¶
The drawback of this approach is that it can induce congestion in already congested networks. The coding ratio needs to be carefully designed.¶
Examples of the solution could be adding a given percentage of the congestion window as supplementary symbols or sending a given amount of repair symbols at a given rate. The redundancy flow can be decorrelated from the congestion control that manages source packets: a separate congestion control entity could be introduced to manage the amount of repaired packets to transmit on the FEC channel. The separate congestion control instances could be made to work together while adhering to priorities, as in coupled congestion control for RTP media [RFC8699] in case all traffic can be assumed to take the same path, or otherwise with a multipath congestion window coupling mechanism as in Multipath TCP [RFC6356]. Another possibility would be to exploit a lower than best-effort congestion control [RFC6297] for repair symbols.¶
The coding scheme may hide congestion losses from the congestion controller. There are cases where this can drastically reduce the goodput of non-coded flows. Depending on the congestion control, it may be possible to signal to the congestion control mechanism that there was congestion (loss) even when a packet has been recovered, e.g. using ECN, to reduce the impact on the non-coded flows (see Section 5.2 and [TENTET]).¶
The congestion control may not know what is going on in the network underneath and whether a coding scheme is introduced or not. The congestion control may behave as if no coding scheme is introduced. The only way for a coding channel to indicate that symbols have been recovered is to exploit existing signaling that is understood by the congestion control mechanism. An example would be to indicate to a TCP sender that a packet has been recovered (i.e., congestion has occurred), by using ECN signaling [TENTET].¶
The coding rate can be tuned depending on the number of recovered symbols and the rate at which the sender transmits data. The coding scheme is not aware of the congestion control implementation, making it hard for the coding scheme to apply the relevant coding rate.¶
The useless repair symbols only impact the load of the network without actual gain for the coded flow. That being said, using feedback signaling, FEC mechanisms can measure the actually used symbols and tune the amount of useless repair symbols.¶
The transport above the FEC channel may support out-of-order delivery of packets: reordering mechanisms at the receiver may not be necessary. In cases where the transport requires in-order delivery, the FEC channel may need to implement a reordering mechanism otherwise there may be spurious retransmissions at the transport level.¶
The transport or application layer above the FEC channel may require partial reliability only. In this case, FEC may provide an unnecessary service if it is not aware of the reliability requirements.¶
The transport may exploit multiple paths without the FEC channel being aware of it. This depends on whether FEC is applied to all subflows or each of the subflows individually. When FEC is applied to all the flows, there is a risk for the coding rate to be inadequate for the characteristics of the individual paths.¶
This section provides a short state-of-the art overview of activities related to congestion control and coding. The objective is to identify open research questions and contribute to advice when evaluating coding mechanisms.¶
We map activities related to congestion control and coding with the organization presented in this document:¶
There is a general trade-off, inherent to the use of coding, between (1) reducing goodput when useless repair symbols are transmitted and (2) helping to recover from transmission and congestion losses.¶
For the FEC above transport case, there is a trade-off related to the amount of redundancy to add, as a function of the transport layer protocol and application requirements.¶
For the FEC within transport case, recovering lost symbols may hide congestion losses to the congestion control. Some existing solutions already propose to disambiguate acked packets from rebuilt packets [QUIC-FEC]. New signaling methods and FEC-recovery-aware congestion controls could be proposed.¶
For the FEC below transport case, there are opportunities for introducing interaction between congestion control and coding schemes to improve the quality of experience while guaranteeing fairness with other flows. New signaling methods and FEC-recovery-aware congestion controls could be proposed. An open question also resides in the relevance of FEC when there are multiple streams that exploit the FEC channel.¶
The contribution to research questions should be mapped following the organization of this document. Otherwise, this may lead to wrong assumptions on the validity of the proposal and wrong ideas about the relevance of coding for a given use case.¶
The discussion provided in this document aims at encouraging the research community to also consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems. As one example, this draft proposes discussions on the impact of the proposed FEC solution on congestion control, especially loss-based congestion control mechanisms. When a research work aims at improving throughput by hiding the packet loss signal from congestion control, the authors should 1) discuss the advantages of using the proposed FEC solution compared to replacing the congestion control by one that ignores a portion of the encountered losses, 2) critically discuss the impact of hiding packet loss from the congestion control mechanism.¶
Many thanks to Spencer Dawkins, Dave Oran, Carsten Bormann, Vincent Roca and Marie-Jose Montpetit for their useful comments that helped improve the document.¶
This memo includes no request to IANA.¶
FEC and CC schemes can contribute to DoS attacks. This is not specific to this document.¶
In case of FEC below the transport, the aggregate rate of source and repair packets may exceed the rate at which a congestion control mechanism allows an application to send. This could result in an application obtaining more than its fair share of the network capacity.¶