Internet Engineering Task Force T. Jones Internet-Draft G. Fairhurst Intended status: Informational University of Aberdeen Expires: 26 August 2021 N. Kuhn CNES J. Border Hughes Network Systems, LLC E. Stephan Orange 22 February 2021 Enhancing Transport Protocols over Satellite Networks draft-jones-tsvwg-transport-for-satellite-00 Abstract IETF transport protocols such as TCP, SCTP and QUIC are designed to function correctly over any network path. This includes networks paths that utilise a satellite link or network. While transport protocols function, the characteristics of satellite networks can impact performance when using the defaults in standard mechanisms, due to the specific characteristics of these paths. RFC 2488 and RFC 3135 describe mechanisms that enable TCP to more effectively utilize the available capacity of a network path that includes a satellite system. Since publication, both application and transport layers and satellite systems have evolved. Indeed, the development of encrypted protocols such as QUIC challenges currently deployed solutions, for satellite systems the capacity has increased and commercial systems are now available that use a range of satellite orbital positions. This document describes the current characterises of common satellite paths and describes considerations when implementing and deploying reliable transport protocols that are intended to work efficiently over paths that include a satellite system. It discusses available network mitigations and offers advice to designers of protocols and operators of satellite networks. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute Jones, et al. Expires 26 August 2021 [Page 1] Internet-Draft Internet Transport for Satellite February 2021 working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 26 August 2021. Copyright Notice Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Satellite Systems . . . . . . . . . . . . . . . . . . . . . . 5 2.1. Geosynchronous Earth Orbit (GEO) . . . . . . . . . . . . 6 2.2. Low Earth Orbit (LEO) . . . . . . . . . . . . . . . . . . 7 2.3. Medium Earth Orbit (MEO) . . . . . . . . . . . . . . . . 7 2.4. Hybrid Network Paths . . . . . . . . . . . . . . . . . . 7 2.5. Convergence with Mobile Cellular . . . . . . . . . . . . 8 3. Satellite System Characteristics . . . . . . . . . . . . . . 8 3.1. Impact of Delay . . . . . . . . . . . . . . . . . . . . . 10 3.1.1. Larger Bandwidth Delay Product . . . . . . . . . . . 10 3.1.2. Variable Link Delay . . . . . . . . . . . . . . . . . 10 3.1.3. Impact of delay on protocol feedback . . . . . . . . 10 3.2. Intermittent connectivity . . . . . . . . . . . . . . . . 11 4. On-Path Mitigations . . . . . . . . . . . . . . . . . . . . . 11 4.1. Link-Level Forward Error Correction and ARQ . . . . . . . 11 4.2. PMTU Discovery . . . . . . . . . . . . . . . . . . . . . 11 4.3. Quality of Service (QoS) . . . . . . . . . . . . . . . . 11 4.4. Split-TCP PEP . . . . . . . . . . . . . . . . . . . . . . 11 4.5. Application Proxies . . . . . . . . . . . . . . . . . . . 12 5. Generic Transport Protocol Mechanisms . . . . . . . . . . . . 13 5.1. Getting up to Speed . . . . . . . . . . . . . . . . . . . 14 5.2. Sizing of Maxium Congestion Window . . . . . . . . . . . 14 Jones, et al. Expires 26 August 2021 [Page 2] Internet-Draft Internet Transport for Satellite February 2021 5.3. Reliability (Loss Recovery/Repair) . . . . . . . . . . . 14 5.3.1. Packet Level Forward Error Correction . . . . . . . . 15 5.4. Flow Control . . . . . . . . . . . . . . . . . . . . . . 15 5.5. ACK Traffic Reduction . . . . . . . . . . . . . . . . . . 16 5.6. Multi-Path . . . . . . . . . . . . . . . . . . . . . . . 16 6. Protocol Specific Mechanisms . . . . . . . . . . . . . . . . 16 6.1. TCP Protocol Mechanisms . . . . . . . . . . . . . . . . . 16 6.1.1. Transport Initialization . . . . . . . . . . . . . . 16 6.1.2. Getting Up To Speed . . . . . . . . . . . . . . . . . 17 6.1.3. Size of Windows . . . . . . . . . . . . . . . . . . . 17 6.1.4. Reliability . . . . . . . . . . . . . . . . . . . . . 17 6.1.5. ACK Reduction . . . . . . . . . . . . . . . . . . . . 17 6.2. QUIC Protocol Mechanisms . . . . . . . . . . . . . . . . 17 6.2.1. Transport initialization . . . . . . . . . . . . . . 17 6.2.2. Getting up to Speed . . . . . . . . . . . . . . . . . 17 6.2.3. Size of Windows . . . . . . . . . . . . . . . . . . . 17 6.2.4. Reliability . . . . . . . . . . . . . . . . . . . . . 17 6.2.5. Asymmetry . . . . . . . . . . . . . . . . . . . . . . 17 6.2.6. Packet Level Forward Error Correction . . . . . . . . 18 6.2.7. Split Congestion Control . . . . . . . . . . . . . . 18 7. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.1. Mitigation Summary . . . . . . . . . . . . . . . . . . . 18 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 19 9. Security Considerations . . . . . . . . . . . . . . . . . . . 19 10. Informative References . . . . . . . . . . . . . . . . . . . 19 Appendix A. Example Network Profiles . . . . . . . . . . . . . . 22 A.1. LEO . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.2. MEO . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.3. GEO . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.3.1. Small public satellite broadband access . . . . . . . 23 A.3.2. Medium public satellite broadband access . . . . . . 23 A.3.3. Congested medium public satellite broadband access . 24 A.3.4. Variable medium public satellite broadband access . . 25 A.3.5. Loss-free large public satellite broadband access . . 25 A.3.6. Lossy large public satellite broadband access . . . . 26 Appendix B. Revision Notes . . . . . . . . . . . . . . . . . . . 27 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 1. Introduction Satellite communications (SATCOM) systems have long been used to support point-to-point links and specialised networks. The predominate current use today is to support Internet Protocols. Typical example applications include: use as an access technology for remote locations, backup and rapid deployment of new services, transit networks, backhaul of various types of IP and mobile networks, and service provision to moving terminals (maritime, aircraft, etc.). Jones, et al. Expires 26 August 2021 [Page 3] Internet-Draft Internet Transport for Satellite February 2021 In most scenarios, the satellite IP network segment forms only one part of the end-to-end path used by an Internet transport protocol. This means that user traffic can experience a path that includes a satellite network combined with a wide variety of other network technologies (Ethernet, cable modems, WiFi, cellular, radio links, etc). Although a user can sometimes know the presence of a satellite service, a typical user does not deploy special software or applications when a satellite network is being used. Users can therefore be often unaware of the technologies underpinning the links forming a network path. Satellite path characteristics have an effect on the operation of Internet transport protocols, such as TCP, SCTP or QUIC. Transport Protocol performance can be affected by the magnitude and variability of the network delay. When transport protocols perform poorly the link utilization can be low. Techniques and recommendations have been made that can improve the performance of transport protocols when the path includes as satellite network. The end-to-end performance of an application using an Internet path can be impacted by the path characteristics, such as the Bandwidth- Delay Product (BDP) of the links and network devices forming the path. It can also be impacted by underlying mechanisms used to manage the radio resources. Performance can be impacted at several layers. For instance, the page load time for a complex page can be much larger when a path includes a satellite system. Although mechanisms are designed for use across Internet paths, not all designs are performant when used over the wide diversity of path characteristics that can occur. This document therefore considers the implications of Internet paths that include a satellite system. A significant contribution to the reduced performance can arise from the initialisation and design of transport mechanisms. The analysis and conclusions might also apply to other network systems that also result in characteristics that differ from typical Internet paths. RFC 2488 specifies an Internet Best Current Practices for the Internet Community, relating to use of the standards-track Transmission Control Protocol (TCP) mechanisms over satellite channels [RFC2488]. A separate RFC,[RFC2760], identified research issues and proposed mitigations for satellite paths. Since the publication of these RFCs many TCP mechanisms have become widely used. In particular, this includes a series of mitigation based on Performance Enhancing Proxies (PEPs) [RFC3135] that split the protocol at the transport layer. Although PEPs are now a common component of satellite systems, their use slows the deployment of new Jones, et al. Expires 26 August 2021 [Page 4] Internet-Draft Internet Transport for Satellite February 2021 transport protocols and mechanisms (each of which demands an update to the PEP functionality). This has made it difficult for new protocol extensions to achieve comparable performance over satellite channels. In addition, protocols with strong requirements on authentication and privacy such as QUIC [I-D.ietf-quic-transport] are not able to be split using a PEP and mitigation, and need to therefore use other methods. XXX Authors Note: This document currently focuses on Geosynchronous Earth Orbit (GEO) satellite systems, the authors solicit feedback and experience from users and operators of satellite systems using other orbits. XXX The remainder of this document is divided as follows: * Section 2 identifies common characteristics of a SATCOM network that can impact the operation of the transport protocols. This complements the description of [RFC2488]. * Section 3 discusses specific characteristics that need to be considered when implementing and deploying transport protocols and highlights key changes since the publication of [RFC2488]. * Section 4 outlines existing deployed mitigations that operate below the transport protocol layer. This offers advice to designers and operators of satellite networks. * Section 5 outlines transport protocol mechanisms defined that may benefit with satellite networks specific tuning and optimization. In particular it discusses on end-to-end considerations, and the mechanisms that impact performance of encrypted transports. * Finally, Section 6 provides a summary of the features recommended for modern transport protocols. 2. Satellite Systems This document considers the characteristics of satellite communications systems. Satellite systems are being deployed using many space orbits, including low earth orbit, medium earth orbits, geosynchronous orbits, elliptical orbits and more. * Many communications satellites are located at Geostationary Orbit (GEO) with an altitude of approximately 36,000 km [Sta94]. At this altitude the orbit period is the same as the Earth's rotation period. Therefore, each ground station is always able to "see" the orbiting satellite at the same position in the sky. The propagation time for a radio signal to travel twice that distance Jones, et al. Expires 26 August 2021 [Page 5] Internet-Draft Internet Transport for Satellite February 2021 (corresponding to a ground station directly below the satellite) is 239.6 milliseconds (ms) [Mar78]. For ground stations at the edge of the coverage of a satellite, the distance traveled is 2 x 41,756 km for a total propagation delay of 279.0 ms [Mar78]. These delays are for one ground station-to-satellite-to-ground station route (or "hop"). Therefore, the delay to send a packet and receive the corresponding reply (one round-trip time or RTT) could be at least 558 ms. This RTT is not solely due to satellite signal propagation time and will be increased by other factors, such as the serialisation time, including any FEC encoding/ARQ delay and propagation time of other links along the network path and the queueing delay in network equipment. The delay is also increased when multiple hops are used (i.e. communications is relayed via a gateway) or in systems using inter-satellite links. As satellites become more complex and include on-board processing of signals, additional delay can be added. * Communications satellites can also be built to use a Low Earth Orbit (LEO) [Stu95] [Mon98]. The lower orbits require the use of constellations of satellites for constant coverage. In other words, as one satellite leaves the ground station's sight, another satellite appears on the horizon and the channel is switched to it. The propagation delay to a LEO orbit ranges from several milliseconds when communicating with a satellite directly overhead, to as much as 20 ms when the same satellite is on the horizon. Some LEO systems use inter-satellite links, where the path delay depends on the routing through the network. * Another orbital position use a Medium Earth Orbit (MEO) [Mar78]. These orbits lie between LEO and GEO. 2.1. Geosynchronous Earth Orbit (GEO) The characteristics of systems using Geosynchronous Earth Orbit (GEO) satellites differ from paths only using terrestrial links in their path characteristics: * A large propagation delay of at least 250ms one-way delay; * Use of radio resource management (often using techniques similar to cellular mobile or DOCSIS cable networks, but differ to accommodate the satellite propagation delay); * Links can be highly asymmetric in terms of capacity, the one-way delay and their cost of operation. As an example, many GEO systems are build using the DVB-S2 specifications [EN 302 307-1], published by the European Jones, et al. Expires 26 August 2021 [Page 6] Internet-Draft Internet Transport for Satellite February 2021 Telecommunications Standards Institute (ETSI), where the key concept is to ensure both a good usage of the satellite resource and a Quasi- Error-Free (QEF) link. These systems typically monitor the link quality in real-time, and known symbol sequences, included along with regular packets enable an estimation of the current signal-to-noise ratio, that can fed back allowing the transmitting link to adapt its coding rate and modulation to the current transmission conditions. 2.2. Low Earth Orbit (LEO) There are many designs of LEO systems. Depending on the locations of the gateways on the ground, routing within the constellation can be necessary to forward packets down to a ground terminal. Capacity can vary significantly between systems. Depending on the routes currently available - especially upon whether Inter-Satellite Links (ISL) are used, additional jitter may occur (from 40ms to 140ms with the Iridium constellation). Some systems can also experience either out-of-order delivery of packets or additional delay due to buffering. Other systems have very different designs. XXX The authors solicit feedback and experience from users and operators of satellite systems in LEO orbits. XXX 2.3. Medium Earth Orbit (MEO) MEO systems such as O3B combines advantages and drawbacks from both LEO and GEO systems. MEO systems can have a large coverage and with limited number of satellites required providing a broad service. The usage of powerful satellites enables provision of high data rates. MEO systems have the drawback, from a transport protocol perspective, that the BDP can be very high due to the altitude of such constellations (8 063 km for [O3B]) and there may be delay variations when coverage requires handover to another MEO satellite (e.g. every 45 minutes with O3B). This can be mitigated by diversity techniques (e.g. double antennas at terminals). XXX The authors solicit feedback and experience from users and operators of satellite systems in MEO orbits. XXX 2.4. Hybrid Network Paths XXX The authors solicit feedback and experience from users and operators of satellite systems in hybrid network scenarios. XXX Jones, et al. Expires 26 August 2021 [Page 7] Internet-Draft Internet Transport for Satellite February 2021 2.5. Convergence with Mobile Cellular XXX This section should look at IP convergence with 5G systems and emerging specs 3GPP non terrestrial networks (NTN). XXX 3. Satellite System Characteristics There is an inherent delay in the delivery of a packet over a satellite system due to the finite speed of light and the altitude of communications satellites. Satellite links are dominated by two fundamental characteristics, as described below: * Packet Loss: The strength of any radio signal falls in proportion to the square of the distance traveled. For a satellite link the square of the distance traveled. Is large and so the signal becomes weak before reaching its destination. This results in a low signal-to-noise ratio. Some frequencies are particularly susceptible to atmospheric effects such as rain attenuation. For applications with moving terminals, satellite channels are especially susceptible to multi-path distortion and shadowing (e.g., blockage by buildings). A typical modern satellite link can have a bit error ratio (BER) of the order of 1 error per 10 million bits (1 x 10^-7) or less frequent. Advanced error control coding (e.g., Reed Solomon or LDPC) can be added to existing satellite services and is currently being used by many services. Satellite performance approaching fiber will become more common using advanced error control coding in new systems. However, many legacy satellite systems will continue to exhibit higher physical layer BER than newer satellite systems. TCP uses all packet drops as signals of network congestion and reduces its window size in an attempt to alleviate the congestion. In the absence of knowledge about why a packet was dropped (congestion or corruption), TCP must assume the drop was due to network congestion to avoid congestion collapse [Jac88] [FF98]. Therefore, packets dropped due to corruption cause TCP to reduce the size of its sliding window, even though these packet drops do not signal congestion in the network. * Bandwidth: The radio spectrum is a limited natural resource, there is a restricted amount of bandwidth available to satellite systems, which is regulated by ITU-R and usually controlled by licenses. This scarcity makes it difficult to increase bandwidth to solve other design problems. Satellite-based radio repeaters are known as transponders. Traditional C-band transponder bandwidth is typically 36 MHz to accommodate one color television channel (or 1200 voice channels). Ku-band transponders are Jones, et al. Expires 26 August 2021 [Page 8] Internet-Draft Internet Transport for Satellite February 2021 typically around 50 MHz. Furthermore, one satellite may carry a few dozen transponders. Not only is bandwidth limited by nature, but the allocations for commercial communications are limited by international agreements so that this scarce resource can be used fairly by many different communications applications. Typical carrier frequencies for current, point- to-point, commercial, satellite services are 6 GHz (uplink) and 4 GHz (downlink), also known as C-band, and 14/12 GHz (Ku band). Services also utilise higher bands, including 30/20 GHz (Ka-band). XXX JB: I think we need add Ka-band details. You cannot get 250 Mbps out of a C-band or Ku-band transponder. Outbound Ka-band transponders range from 100 to 500 MHz. Inbound Ka-band transponders range from 50 to 250 MHz.XXX * Link Design: It is common to consider a satellite network segment as composed of a forward link and a return link. The two links usually have different capacities and employ different technologies to carry IP packets. On the forward link, a satellite gateway often manages all the available capacity, possibly with several carriers, to communicate with a set of remote terminals. A carrier is a single Time-Division- Multiplexing (TDM) channel that multiplexes packets addressed to specific terminals. There are trade-offs in terms of overall system efficiency and performance observed by a user. Most systems incur additional delay to ensure overall system performance. On the return link, satellite resource is typically dynamically shared among the terminals. * Shared Medium Access: In common with other radio media, satellite capacity can be assigned for use by a link for a period of time, for the duration of communication, for a per-packet or per burst of packets, or accessed using contention mechanisms. Packets sent over a shared radio channels need to be sent in frames that need to be allocated resources (bandwidth, power, time) for their transmission. This results in a range of characteristics that are very different to a permanently assigned medium (such as an Ethernet link using an optical fibre). Two access methods can be distinguished: on-demand access or contention access. In the former, a terminal receives dedicated transmission resources (usually to send to the gateway). In the latter, some resources are reserved for contention access, where a set of terminals are allowed to compete to obtain transmission resource. Dynamic access is more common in currently deployed systems and can be through a Demand Assigned Multiple Access (DAMA) mechanism, while contention access techniques are usually based on Slotted Aloha (SA) and its numerous derivatives. More information on satellite links characteristics can be found in [RFC2488] [IJSCN17]. Jones, et al. Expires 26 August 2021 [Page 9] Internet-Draft Internet Transport for Satellite February 2021 Satellite systems have several characteristics that differ from most terrestrial channels. These characteristics may degrade the performance of TCP. These characteristics include: 3.1. Impact of Delay Even for characteristics shared with terrestrial paths, the impact on a satellite link could be amplified by the path RTT. For example, paths using a satellite system can also exhibit a high loss-rate (e.g., a mobile user or a user behind a Wi-Fi link), where the additional delay can impact transport mechanisms. 3.1.1. Larger Bandwidth Delay Product Although capacity is often less than in many terrestrial systems, the bandwidth delay product (BDP) defines the amount of data that a protocol is permitted to have "in flight" (data transmitted, but not yet acknowledged) at any one time to fully utilize the available capacity. The delay used in this equation is the path RTT and the bandwidth is the capacity of the bottleneck link along the network path. Because the delay in some satellite environments is larger, protocols need to keep a larger number of packets "in flight" (that is, sent but not yet acknowledged). This also impacts the size of window/credit needed to avoid flow control mechanisms throttling the sender rate. 3.1.2. Variable Link Delay In some satellite environments, such as some Low Earth Orbit (LEO) constellations, the propagation delay to and from the satellite varies over time. Even when the propagation delay varies only very slightly, the effects of medium access methods can result in significant variation in the link delay. Whether or not this will have an impact on performance of a well-designed transport is currently an open question. 3.1.3. Impact of delay on protocol feedback The link delay of some satellite systems may require more time for a transport sender to determine whether or not a packet has been successfully received at the final destination. This delay impacts interactive applications as well as loss recovery, congestion control, flow control, and other algorithms (see Section 5). Jones, et al. Expires 26 August 2021 [Page 10] Internet-Draft Internet Transport for Satellite February 2021 3.2. Intermittent connectivity For systems using non-GEO satellites, from time to time Internet connections need to be transferred from one satellite to another or from one ground station to another. This hand-over can be made without interrupting the service, but in some system designs might cause packet loss or reordering. 4. On-Path Mitigations This section describes mitigations that operate on the path, rather than with the transport endpoints. 4.1. Link-Level Forward Error Correction and ARQ XXX Common. This includes Adaptive Coding and Modulation (ACM) and sometimes link ARQ - which can reduce the loss at the expense of decreasing the available capacity. XXX 4.2. PMTU Discovery XXX Packet size can impact performance and mitigations (such as PEP/ Application Proxy) can interact with end-to-end PMTUD. XXX 4.3. Quality of Service (QoS) Links were packets are sent over radio channels exhibit various trade-offs in the way the signal is sent on the communications channel. These trade-offs are not necessarily the same for all packets, and network traffic flows can be optimised by mapping these onto different types of lower layer treatment (packet queues, resource management requests, resource usage, and adaption to the channel using FEC, ARQ, etc). Many systems differentiate classes of traffic to mange these QoS trade-offs. 4.4. Split-TCP PEP High BDP networks commonly break the TCP end-to-end paradigm to adapt the transport protocol. Splitting a TCP connection allows adaptation for a specific use-case and to address the issues discussed in Section 2. Satellite communications commonly deploy Performance Enhancing Proxy (PEP) for compression, caching and TCP acceleration services [RFC3135] . Their deployment can result in significant performance improvement (e.g., a 50% page load time reduction in a SATCOM use-case [ICCRG100] . [NCT13] and [RFC3135] describe the main functions of a SATCOM TCP split solution. For traffic originated at a gateway to an endpoint Jones, et al. Expires 26 August 2021 [Page 11] Internet-Draft Internet Transport for Satellite February 2021 connected via a satellite terminal, the TCP split proxy intercepts TCP SYN packets, acting on behalf of the endpoint and adapts the sending rate to the SATCOM scenario. The split solution can specifically tune TCP parameters to the satellite link (latency, available capacity). When a proxy is used on each side of the satellite link, the transport protocol can be replaced by a protocol other than TCP, optimized for the satellite link. This can be tuned using a priori information about the satellite system and/or by measuring the properties of the network segment that includes the satellite system. Split connections can also recover from packet loss that is local to the part of the connection on which the packet losses occur. This eliminates the need for end-to-end recovery of lost packets. One important advantage of a TCP split solution is that it does not require any end-to-end modification and is independent of both the client and server sides. This also comes with a drawback: split-TCP PEPs can ossify the protocol stack being used because they are often unable to track improvements in end-to-end protocol mechanisms (e.g., RACK, ECN, TCP Fast Open). The set of methods configured in a split proxy usually continue to be used, until the split solution is finally updated. This can delay/negate the benefit of any end-to-end improvements. 4.5. Application Proxies Authenticated proxies: * The existence of Application Proxies requires a discovery device, which might vary by user - by service - etc.; * Application Proxies can split key functions, but this requires agreement between endpoints and the proxy on the formats/semantics of the protocol info that is to be changed; * With the common use of security functions (such as TLS), there also needs to be a trust relationship - a proxy needs to be authenticated; * A proxy needs to remain on the path, which can place constraints on the routing infrastructure - handover between proxies is possible, but is generally complex. Jones, et al. Expires 26 August 2021 [Page 12] Internet-Draft Internet Transport for Satellite February 2021 5. Generic Transport Protocol Mechanisms This section outlines transport protocol mechanisms that may be necessary to tune or optimize in satellite or hybrid satellite/ terrestrial networks to better utilize the available capacity of the link. These mechanisms may also be needed to fully utilize fast terrestrial channels. Furthermore, these mechanisms do not fundamentally hurt performance in a shared terrestrial network. Each of the following sections outlines one mechanism and why that mechanism may be needed. * Transport initialization: the connection handshake (in TCP the 3-way exchange) takes a longer time to complete, delaying the time to send data (several transport protocol exchanges may be needed, such as TLS); * Size of congestion window required: to fully exploit the bottleneck capacity, a high BDP requires a larger number of in- flight packets; * Size of receiver (flow control) window required: to fully exploit the bottleneck capacity, a high BDP requires a larger number of in-flight packets; * Reliability: transport layer loss detection and repair can incur a single or multiple RTTs (the performance of end-to-end retransmission is also impacted when using a high RTT path); * Getting up to speed: many congestion control methods employ an exponential increase in the sending rate during slow start (for path capacity probing), a high RTT will increase the time to reach a specific rate; * Asymmetry: when the links are asymmetric the return path may modify the rate and/timing of transport acknowledgment traffic, potentially changing behaviour (e.g., limiting the forward sending rate). Jones, et al. Expires 26 August 2021 [Page 13] Internet-Draft Internet Transport for Satellite February 2021 5.1. Getting up to Speed Many transport protocols now deploy 0-RTT mechanisms [REF] to reduce the number of RTTs required to establish a connection. QUIC has an advantage that the TLS and TCP negotiations can be completed during the transport connection handshake. This can reduce the time to transmit the first data. Results of [IJSCN19] illustrate that it can still take many RTTs for a CC to increase the sending rate to fill the bottleneck capacity. The delay in getting up to speed can dominate performance for a path with a large RTT, and requires the congestion and flow controls to accommodate the impact of path delay. One relevant solution is tuning of the initial window described in [I-D.irtf-iccrg-sallantin-initial-spreading], which has been shown to improve performance both for high BDP and more common BDP [CONEXT15] [ICC16]. Such a solution requires using sender pacing to avoid generating bursts of packets in a network. 5.2. Sizing of Maxium Congestion Window Size of windows required: to fully exploit the bottleneck capacity, a high BDP requires a larger number of in-flight packets. The number of in-flight packets required to fill a bottleneck capacity, is dependent on the BDP. Default values of maximum windows might be unsuitable in a SATCOM context. Such as presented in [PANRG105] , only increasing the initial congestion window is not the only way that can improve QUIC performance in a SATCOM context: increasing maximum congestion windows can also result in much better performance. Other protocol mechanisms also need to be considered, such as flow control at the stream level in QUIC. 5.3. Reliability (Loss Recovery/Repair) The time for end systems to perform packet loss detection and recovery/repair is a function of the path RTT. The RTT also determines the time needed by a server to react to a congestion event. Both can impact the user experience. For example, when a user uses a Wi-Fi link to access the Internet via SATCOM terminal. End-to-end packet Forward Error Correction (FEC) offers an alternative to retransmission with different trade offs in terms of utilised capacity and repair capability. Jones, et al. Expires 26 August 2021 [Page 14] Internet-Draft Internet Transport for Satellite February 2021 Network coding as proposed in [I-D.swett-nwcrg-coding-for-quic] and [I-D.roca-nwcrg-rlc-fec-scheme-for-quic] could help QUIC recover from link or congestion loss. Another approach could utilise QUIC tunnels [I-D.schinazi-masque] to apply FEC to all or a part of the end-to-end path. The benefits of introducing FEC need to weighed against the additional capacity introduced by end-to-end FEC and the opportunity to use link-local ARQ and/or link-adaptive FEC. A transport connections can suffer link-related losses from a particular link (e.g., Wi-Fi), but also congestion loss (e.g. router buffer overflow in a satellite operator ground segment or along an Internet path). Mechanisms have been proposed in [I-D.ferrieux-hamchaoui-quic-lossbits] , to identify congestion losses in the ground segment. 5.3.1. Packet Level Forward Error Correction XXX Packet level FEC can mitigate loss/re-ordering, with a trade-off in capacity. XXX 5.4. Flow Control Flow Control mechanisms allow the receiver to control the amount of data a send can have in flight at any time. Flow Control allows the receiver to allocate the smallest buffer sizes possible improving memory usage on receipt. The sizing of initial receive buffers requires a balance between keeping receive memory allocation small while allowing the send window to grow quickly to help ensure high utilization. The size of receive windows and their growth can govern the performance of the protocol if updates are not timely. Many TCP implementations deploy Auto-scaling mechanisms to increase the size of the largest receive window over time. If these increases are not timely then sender traffic can stall while waiting to be notified of an increase in receive window size. XXX QUIC? XXX Multi-streaming Protocols such as QUIC implement Flow Control using credit-based mechanisms that allow the receiver to prioritise which stream is able to send and when. Credit-based systems, when flow credit allocations are not timely, can stall sending when credit is exhausted. Jones, et al. Expires 26 August 2021 [Page 15] Internet-Draft Internet Transport for Satellite February 2021 5.5. ACK Traffic Reduction When the links are asymmetric, for various reasons, the return path may modify the rate and/timing of transport acknowledgment traffic, potentially changing behaviour (e.g., limiting the forward sending rate). Asymmetry in capacity (or in the way capacity is granted to a flow) can lead to cases where the transmission in one direction of communication is restricted by the transmission of the acknowledgment traffic flowing in the opposite direction. A network segment could present limitations in the volume of acknowledgment traffic (e.g., limited available return path capacity) or in the number of acknowledgment packets (e.g., when a radio-resource management system has to track channel usage), or both. TCP Performance Implications of Network Path Asymmetry [RFC3449] describes a range of mechanisms that have been used to mitigate the impact of path asymmetry, primarily targeting operation of TCP. Many mitigations have been deployed in satellite systems, often as a mechanism within a PEP. Despite their benefits over paths with high asymmetry, most mechanisms rely on being able to inspect and/or modify the transport layer header information of TCP ACK packets. This is not possible when the transport layer information is encrypted (e.g., using an IP VPN). One simple mitigation is for the remote endpoint to send compound acknowledgments less frequently. A rate of one ACK for every RTT/4 can significantly reduce this traffic. The QUIC transport specification may evolve to allow the ACK Ratio to be adjusted. 5.6. Multi-Path XXX This includes between different satellite systems and between satellite and terrestrial paths XXX 6. Protocol Specific Mechanisms 6.1. TCP Protocol Mechanisms 6.1.1. Transport Initialization Jones, et al. Expires 26 August 2021 [Page 16] Internet-Draft Internet Transport for Satellite February 2021 6.1.2. Getting Up To Speed One relevant solution is tuning of the initial window described in [I-D.irtf-iccrg-sallantin-initial-spreading][RFC6928], which has been shown to improve performance both for high BDP and more common BDP [CONEXT15] [ICC16]. This requires sender pacing to avoid generating bursts of packets to the network. 6.1.3. Size of Windows 6.1.4. Reliability 6.1.5. ACK Reduction Mechanisms are being proposed in TCPM for TCP [REF]. 6.2. QUIC Protocol Mechanisms 6.2.1. Transport initialization QUIC has an advantage that the TLS and TCP negotiations can be completed during the transport connection handshake. This can reduce the time to transmit the first data. Moreover, using 0-RTT may further reduce the connection time for users reconnecting to a server. 6.2.2. Getting up to Speed Getting up to speed may be easier with the usage of the 0-RTT-BDP extension proposed in [I-D.kuhn-quic-0rtt-bdp]. 6.2.3. Size of Windows 6.2.4. Reliability Mechanisms have been proposed in [I-D.ferrieux-hamchaoui-quic-lossbits] , to identify congestion losses in the ground segment. 6.2.5. Asymmetry The QUIC transport specification may evolve to allow the ACK Ratio to be adjusted. Default could be adapted following [I-D.fairhurst-quic-ack-scaling] or using extensions to tune acknowledgement strategies [I-D.iyengar-quic-delayed-ack]. Jones, et al. Expires 26 August 2021 [Page 17] Internet-Draft Internet Transport for Satellite February 2021 6.2.6. Packet Level Forward Error Correction Network coding as proposed in [I-D.swett-nwcrg-coding-for-quic] and [I-D.roca-nwcrg-rlc-fec-scheme-for-quic] could help QUIC recover from link or congestion loss. Another approach could utilise QUIC tunnels [I-D.schinazi-masque] to apply packet FEC to all or a part of the end-to-end path or enable local retransmissions. 6.2.7. Split Congestion Control Splitting the congestion control requires the deployment of application proxies. 7. Discussion Many of the issues identified for high BDP paths already exist when using an encrypted transport service over a path that employs encryption at the IP layer. This includes endpoints that utilise IPsec at the network layer, or use VPN technology over a satellite network segment. Users are unable to benefit from enhancement within the satellite network segment, and often the user is unaware of the presence of the satellite link on their path, except through observing the impact it has on the performance they experience. One solution would be to provide PEP functions at the termination of the security association (e.g., in a VPN client). Another solution could be to fall-back to using TCP (possibly with TLS or similar methods being used on the transport payload). A different solution could be to deploy and maintain a bespoke protocol tailored to high BDP environments. In the future, we anticipate that fall-back to TCP will become less desirable, and methods that rely upon bespoke configurations or protocols will be unattractive. In parallel, new methods such as QUIC will become widely deployed. The opportunity therefore exists to ensure that the new generation of protocols offer acceptable performance over high BDP paths without requiring operating tuning or specific updates by users. 7.1. Mitigation Summary XXX A Table will be inserted here XXX Jones, et al. Expires 26 August 2021 [Page 18] Internet-Draft Internet Transport for Satellite February 2021 8. Acknowledgments The authors would like to thank Mark Allman, Daniel R. Glover and Luis A. Sanchez the authors of RFC2488 from which the format and descriptions of satellite systems in this document have taken inspiration. The authors would like to thank Christian Huitema, Igor Lubashev, Alexandre Ferrieux, Francois Michel, Emmanuel Lochin and the participants of the IETF106 side-meeting on QUIC for high BDP for their useful feedback. 9. Security Considerations This document does not propose changes to the security functions provided by the QUIC protocol. QUIC uses TLS encryption to protect the transport header and its payload. Security is considered in the "Security Considerations" of cited IETF documents. 10. Informative References [CONEXT15] Li, Q., Dong, M., and P B. Godfrey, "Halfback: Running Short Flows Quickly and Safely", ACM CoNEXT , 2015. [FF98] Floyd, S. and K. Fall, "Promoting the Use of End-to-End Congestion Control in the Internet. IEEE Transactions on Networking". [I-D.fairhurst-quic-ack-scaling] Fairhurst, G., Custura, A., and T. Jones, "Changing the Default QUIC ACK Policy", Work in Progress, Internet- Draft, draft-fairhurst-quic-ack-scaling-03, 14 September 2020, . [I-D.ferrieux-hamchaoui-quic-lossbits] Ferrieux, A. and I. Hamchaoui, "The QUIC Loss Bits", Work in Progress, Internet-Draft, draft-ferrieux-hamchaoui- quic-lossbits-00, 9 April 2019, . [I-D.ietf-quic-recovery] Iyengar, J. and I. Swett, "QUIC Loss Detection and Congestion Control", Work in Progress, Internet-Draft, draft-ietf-quic-recovery-34, 14 January 2021, . Jones, et al. Expires 26 August 2021 [Page 19] Internet-Draft Internet Transport for Satellite February 2021 [I-D.ietf-quic-transport] Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed and Secure Transport", Work in Progress, Internet-Draft, draft-ietf-quic-transport-34, 14 January 2021, . [I-D.irtf-iccrg-sallantin-initial-spreading] Sallantin, R., Baudoin, C., Arnal, F., Dubois, E., Chaput, E., and A. Beylot, "Safe increase of the TCP's Initial Window Using Initial Spreading", Work in Progress, Internet-Draft, draft-irtf-iccrg-sallantin-initial- spreading-00, 15 January 2014, . [I-D.iyengar-quic-delayed-ack] Iyengar, J. and I. Swett, "Sender Control of Acknowledgement Delays in QUIC", Work in Progress, Internet-Draft, draft-iyengar-quic-delayed-ack-02, 2 November 2020, . [I-D.kuhn-quic-0rtt-bdp] Kuhn, N., Emile, S., Fairhurst, G., and T. Jones, "Transport parameters for 0-RTT connections", Work in Progress, Internet-Draft, draft-kuhn-quic-0rtt-bdp-07, 18 May 2020, . [I-D.roca-nwcrg-rlc-fec-scheme-for-quic] Roca, V., Michel, F., Swett, I., and M. Montpetit, "Sliding Window Random Linear Code (RLC) Forward Erasure Correction (FEC) Schemes for QUIC", Work in Progress, Internet-Draft, draft-roca-nwcrg-rlc-fec-scheme-for-quic- 03, 9 March 2020, . [I-D.schinazi-masque] Schinazi, D., "The MASQUE Protocol", Work in Progress, Internet-Draft, draft-schinazi-masque-02, 8 January 2020, . [I-D.swett-nwcrg-coding-for-quic] Swett, I., Montpetit, M., Roca, V., and F. Michel, "Coding for QUIC", Work in Progress, Internet-Draft, draft-swett- nwcrg-coding-for-quic-04, 9 March 2020, Jones, et al. Expires 26 August 2021 [Page 20] Internet-Draft Internet Transport for Satellite February 2021 . [ICC16] Sallantin, R., Baudoin, C., Chaput, E., Arnal, F., Dubois, E., and A-L. Beylot, "Reducing web latency through TCP IW: Be smart", IEEE ICC , 2016. [ICCRG100] Kuhn, N., "MPTCP and BBR performance over Internet satellite paths", IETF ICCRG 100, 2017. [IJSCN17] Ahmed, T., Dubois, E., Dupe, JB., Ferrus, R., Gelard, P., and N. Kuhn, "Software-defined satellite cloud RAN", International Journal of Satellite Communications and Networking , 2017. [IJSCN19] Thomas, L., Dubois, E., Kuhn, N., and E. Lochin, "Google QUIC performance over a public SATCOM access", International Journal of Satellite Communications and Networking , 2019. [Jac88] Jacobson, V., "Congestion Avoidance and Control. In ACM SIGCOMM, 1988". [Mar78] Martin, J., "Communications Satellite Systems. Prentice Hall, 1978.". [Mon98] Montpetit, M.J., "TELEDESIC: Enabling The Global Community Interaccess. In Proc. of the International Wireless Symposium, May 1998". [NCT13] Pirovano, A. and F. Garcia, "A new survey on improving TCP performances over geostationary satellite link", Network and Communication Technologies , 2013. [PANRG105] Kuhn, N., Stephan, E., Border, J., and G. Fairhurst, "QUIC Over In-sequence Paths with Different Characteristics", IRTF PANRG 105, 2019. [RFC2488] Allman, M., Glover, D., and L. Sanchez, "Enhancing TCP Over Satellite Channels using Standard Mechanisms", BCP 28, RFC 2488, DOI 10.17487/RFC2488, January 1999, . [RFC2760] Allman, M., Ed., Dawkins, S., Glover, D., Griner, J., Tran, D., Henderson, T., Heidemann, J., Touch, J., Kruse, H., Ostermann, S., Scott, K., and J. Semke, "Ongoing TCP Research Related to Satellites", RFC 2760, Jones, et al. Expires 26 August 2021 [Page 21] Internet-Draft Internet Transport for Satellite February 2021 DOI 10.17487/RFC2760, February 2000, . [RFC3135] Border, J., Kojo, M., Griner, J., Montenegro, G., and Z. Shelby, "Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations", RFC 3135, DOI 10.17487/RFC3135, June 2001, . [RFC3449] Balakrishnan, H., Padmanabhan, V., Fairhurst, G., and M. Sooriyabandara, "TCP Performance Implications of Network Path Asymmetry", BCP 69, RFC 3449, DOI 10.17487/RFC3449, December 2002, . [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, "Increasing TCP's Initial Window", RFC 6928, DOI 10.17487/RFC6928, April 2013, . [Sta94] Stallings, W., "Data and Computer Communications. MacMillian, 4th edition, 1994.". [Stu95] Sturza, M.A., "Architecture of the TELEDESIC Satellite System. In Proceedings of the International Mobile Satellite Conference, 1995". Appendix A. Example Network Profiles This proposes sampler profiles and a set of regression tests to evaluate transport protocols over SATCOM links and discusses how to ensure acceptable protocol performance. XXX These test profiles currently focus on the measuring performance and testing for regressions in the QUIC protocol. The authors solicit input to adapt these tests to apply to more transport protocols. XXX A.1. LEO A.2. MEO A.3. GEO This section proposes a set of regression tests for QUIC that consider high BDP scenarios. We define by: * Download path: from Internet to the client endpoint; Jones, et al. Expires 26 August 2021 [Page 22] Internet-Draft Internet Transport for Satellite February 2021 * Upload path: from the client endpoint to a server (e.g., in the Internet). A.3.1. Small public satellite broadband access The tested scenario has the following path characteristics: * Satellite downlink path: 10 Mbps * Satellite uplink path: 2 Mbps * No emulated packet loss * RTT: 650 ms * Buffer size : BDP During the transmission of 100 MB on both download and upload paths, the test should report the upload and download time of 2 MB, 10 MB and 100 MB. Initial thoughts of the performance objectives for QUIC are the following: * 3 s for downloading 2 MB * 10 s for downloading 10 MB * 85 s for downloading 100 MB * 10 s for uploading 2 MB * 50 s for uploading 10 MB * 420 s for uploading 100 MB A.3.2. Medium public satellite broadband access The tested scenario has the following path characteristics: * Satellite downlink path: 50 Mbps * Satellite uplink path: 10 Mbps * No emulated packet loss * RTT: 650 ms Jones, et al. Expires 26 August 2021 [Page 23] Internet-Draft Internet Transport for Satellite February 2021 * Buffer size : BDP During the transmission of 100 MB on the download path, the test should report the download time for 2 MB, 10 MB and 100 MB. Then, to assess the performance of QUIC with the 0-RTT extension and its variants, after 10 seconds, repeat the transmission of 100 MB on the download path where the download time for 2 MB, 10 MB and 100 MB is recorded. Initial thoughts of the performance objectives for QUIC are the following: * 3 s for the first downloading 2 MB * 5 s for the first downloading 10 MB * 20 s for the first downloading 100 MB * TBD s for the second downloading 2 MB * TBD s for the second downloading 10 MB * TBD s for the second downloading 100 MB A.3.3. Congested medium public satellite broadband access There are cases where the uplink path is congested or where the capacity of the uplink path is not guaranteed. The tested scenario has the following path characteristics: * Satellite downlink path: 50 Mbps * Satellite uplink path: 0.5 Mbps * No emulated packet loss * RTT: 650 ms * Buffer size : BDP During the transmission of 100 MB on the download path, the test should report the download time for 2 MB, 10 MB and 100 MB. Initial thoughts of the performance objectives for QUIC are the following: * 3 s for downloading 2 MB Jones, et al. Expires 26 August 2021 [Page 24] Internet-Draft Internet Transport for Satellite February 2021 * 5 s for downloading 10 MB * 20 s for downloading 100 MB A.3.4. Variable medium public satellite broadband access There are cases where the downlink path is congested or where, due to link layer adaptations to rain fading, the capacity of the downlink path is variable. The tested scenario has the following path characteristics: * Satellite downlink path: 50 Mbps - wait 5s - 10 Mbps * Satellite uplink path: 10 Mbps * No emulated packet loss * RTT: 650 ms * Buffer size : BDP During the transmission of 100 MB on the download path, the test should report the download time for 2 MB, 10 MB and 100 MB. Initial thoughts of the performance objectives for QUIC are the following: * TBD s for downloading 2 MB * TBD s for downloading 10 MB * TBD s for downloading 100 MB A.3.5. Loss-free large public satellite broadband access The tested scenario has the following path characteristics: * Satellite downlink path: 250 Mbps * Satellite uplink path: 6 Mbps * No emulated packet loss * RTT: 650 ms * Buffer size : BDP Jones, et al. Expires 26 August 2021 [Page 25] Internet-Draft Internet Transport for Satellite February 2021 During the transmission of 100 MB on the download path, the test should report the download time for 2 MB, 10 MB and 100 MB. Then, to assess the performance of QUIC with the 0-RTT extension and its variants, after 10 seconds, repeat the transmission of 100 MB on the download path where the download time for 2 MB, 10 MB and 100 MB is recorded. Initial thoughts of the performance objectives for QUIC are the following: * 3 s for the first downloading 2 MB * 5 s for the first downloading 10 MB * 8 s for the first downloading 100 MB * TBD s for the second downloading 2 MB * TBD s for the second downloading 10 MB * TBD s for the second downloading 100 MB A.3.6. Lossy large public satellite broadband access The tested scenario has the following path characteristics: * Satellite downlink path: 250 Mbps * Satellite uplink path: 6 Mbps * Emulated packet loss on both downlink and uplink paths: - Uniform random transmission link losses: 1% * RTT: 650 ms * Buffer size : BDP During the transmission of 100 MB on the download path, the test should report the download time for 2 MB, 10 MB and 100 MB. Initial thoughts of the performance objectives for QUIC are the following: * 3 s for downloading 2 MB (uniform transmission link losses) * 6 s for downloading 10 MB (uniform transmission link losses) Jones, et al. Expires 26 August 2021 [Page 26] Internet-Draft Internet Transport for Satellite February 2021 * 10 s for downloading 100 MB (uniform transmission link losses) Appendix B. Revision Notes Note to RFC-Editor: please remove this entire section prior to publication. Individual draft -00: * Comments and corrections are welcome directly to the authors or via the https://github.com/uoaerg/draft-jones-transport-for- satellite github repo in the form of pull requests and issues. Authors' Addresses Tom Jones University of Aberdeen Email: tom@erg.abdn.ac.uk Godred Fairhurst University of Aberdeen Email: gorry@erg.abdn.ac.uk Nicolas Kuhn CNES Email: nicolas.kuhn@cnes.fr John Border Hughes Network Systems, LLC Email: border@hns.com Emile Stephan Orange Email: emile.stephan@orange.com Jones, et al. Expires 26 August 2021 [Page 27]