QUIC Working Group Q. De Coninck Internet-Draft O. Bonaventure Intended status: Standards Track UCLouvain Expires: September 8, 2019 March 7, 2019 Multipath Extensions for QUIC (MP-QUIC) draft-deconinck-quic-multipath-02 Abstract This document specifies extensions to the QUIC protocol to enable, other than connection migration, simultaneous usage of multiple paths for a single connection. Those extensions remain compliant with the current single-path QUIC design. They allow devices to benefit from multiple network paths while preserving the privacy features of QUIC. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on September 8, 2019. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. De Coninck & BonaventureExpires September 8, 2019 [Page 1] Internet-Draft MP-QUIC March 2019 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3 2.1. Notational Conventions . . . . . . . . . . . . . . . . . 4 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. What is a Path? . . . . . . . . . . . . . . . . . . . . . 4 3.2. Beyond Connection Migration . . . . . . . . . . . . . . . 5 3.3. Establishment of a Multipath QUIC Connection . . . . . . 6 3.4. Architecture of Multipath QUIC . . . . . . . . . . . . . 7 3.5. Path Establishment . . . . . . . . . . . . . . . . . . . 8 3.6. Exchanging Data over Multiple Paths . . . . . . . . . . . 9 3.7. Exchanging Addresses . . . . . . . . . . . . . . . . . . 10 3.8. Coping with Address Removals . . . . . . . . . . . . . . 11 3.9. Path Migration . . . . . . . . . . . . . . . . . . . . . 11 3.10. Sharing Path Policies . . . . . . . . . . . . . . . . . . 12 3.11. Congestion Control . . . . . . . . . . . . . . . . . . . 12 4. Mapping Path ID to Connection IDs . . . . . . . . . . . . . . 13 5. Using Multiple Paths . . . . . . . . . . . . . . . . . . . . 13 5.1. Multipath Negotiation . . . . . . . . . . . . . . . . . . 13 5.1.1. Transport Parameter Definition . . . . . . . . . . . 14 5.2. Coping with Additional Remote Addresses . . . . . . . . . 14 5.3. Path State . . . . . . . . . . . . . . . . . . . . . . . 14 5.4. Dynamic Management of Paths . . . . . . . . . . . . . . . 17 5.5. Losing Addresses . . . . . . . . . . . . . . . . . . . . 17 5.6. Scheduling Strategies . . . . . . . . . . . . . . . . . . 18 6. Modifications to QUIC frames . . . . . . . . . . . . . . . . 18 6.1. NEW_CONNECTION_ID Frame . . . . . . . . . . . . . . . . . 19 6.2. RETIRE_CONNECTION_ID Frame . . . . . . . . . . . . . . . 20 6.3. ACK Frame . . . . . . . . . . . . . . . . . . . . . . . . 20 7. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 21 7.1. MAX_PATHS Frame . . . . . . . . . . . . . . . . . . . . . 21 7.2. PATH_UPDATE Frame . . . . . . . . . . . . . . . . . . . . 22 7.3. ADD_ADDRESS Frame . . . . . . . . . . . . . . . . . . . . 23 7.4. REMOVE_ADDRESS Frame . . . . . . . . . . . . . . . . . . 24 7.5. PATHS Frame . . . . . . . . . . . . . . . . . . . . . . . 24 8. Security Considerations . . . . . . . . . . . . . . . . . . . 26 8.1. Nonce Computation . . . . . . . . . . . . . . . . . . . . 26 8.2. Validation of Exchanged Addresses . . . . . . . . . . . . 26 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 9.1. QUIC Transport Parameter Registry . . . . . . . . . . . . 27 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 27 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 27 11.1. Normative References . . . . . . . . . . . . . . . . . . 27 11.2. Informative References . . . . . . . . . . . . . . . . . 28 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 29 A.1. Since draft-deconinck-quic-multipath-01 . . . . . . . . . 29 A.2. Since draft-deconinck-quic-multipath-00 . . . . . . . . . 29 De Coninck & BonaventureExpires September 8, 2019 [Page 2] Internet-Draft MP-QUIC March 2019 A.3. Since draft-deconinck-multipath-quic-00 . . . . . . . . . 30 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30 1. Introduction Endhosts have evolved. Today's endhosts are equipped with several network interfaces and users expect to be able to seamlessly switch from one to another or use them simultaneously to aggregate or grab bandwidth whenever needed. During the last years, several multipath extensions to transport protocols have been proposed [RFC6824],[MPRTP], [SCTPCMT]. Multipath TCP [RFC6824] is the most mature one. It is already deployed on popular smartphones, but also for other use cases [RFC8041]. With regular TCP and UDP, all the packets that belong to a given flow share the same 5-tuple that acts as an identifier for this flow. Such characterization prevents these flows from using multiple paths. QUIC [I-D.ietf-quic-transport] does not use the 5-tuple as an implicit connection identifier. A QUIC flow is identified by two Connection IDs (Source and Destination). This enables QUIC flows to cope with events affecting the 5-tuple, such as NAT rebinding or IP address changes. The QUIC connection migration feature, described in more details in [I-D.ietf-quic-transport], is key to migrate a flow from one 5-tuple to another one. This migration feature offers the opportunity for QUIC to sustain a connection over multiple paths, but still there is a void to specify simultaneous usage of available paths for a single connection. Use cases such as bandwidth aggregation or seamless network handovers would be applicable to QUIC, as they are now with Multipath TCP. An early performance evaluation of such use cases and a comparison between Multipath QUIC and Multipath TCP may be found in [MPQUIC]. In this document, we leverage many of the lessons learned from the design of Multipath TCP and the comments received on the first versions of this draft to propose extensions to the current QUIC design to enable it to simultaneously use several paths. This document focuses mainly on paths that are locally visible to an endpoint. This document is organized as follows. It first provides in Section 3 an overview of the operation of Multipath QUIC. It then states changes required in the current QUIC design [I-D.ietf-quic-transport] and specifies in Section 5 the usage of multiple paths. Finally, it discusses some security considerations. 2. Conventions and Definitions The words "MUST", "MUST NOT", "SHOULD", and "MAY" are used in this document. It's not shouting; when they are capitalized, they have the special meaning defined in [RFC2119]. De Coninck & BonaventureExpires September 8, 2019 [Page 3] Internet-Draft MP-QUIC March 2019 We assume that the reader is familiar with the terminology used in [I-D.ietf-quic-transport]. In addition, we define the following terms: o Path: A logical association within a QUIC connection between two hosts over which packets can be sent. It is typically characterized by (Source IP Address, Source Port Number, Destination IP Address, Destination Port Number). A path is internally identified by endpoints using a Path ID and uses a potentially changing Connection ID identifying the parent connection in packets exchanged on that path. o Initial Path: The path used for the establishment of the QUIC connection. The cryptographic handshake is done on this path. It is identified by Path ID 0. 2.1. Notational Conventions Packet and frame diagrams use the format described in Section 2.1 of [I-D.ietf-quic-transport]. 3. Overview The current design of QUIC [I-D.ietf-quic-transport] provides reliable transport with multiplexing and security. A wide range of devices on today's Internet are multihomed. Examples include smartphones equipped with both WLAN and cellular interfaces, but also regular dual-stack hosts that use both IPv4 and IPv6. Experience with Multipath TCP has shown that the ability to combine different paths during the lifetime of a connection provides various benefits including bandwidth aggregation or seamless handovers [RFC8041],[IETFJ]. The current design of QUIC does not enable multihomed devices to efficiently use different paths simultaneously. We first explain why a multipath extension would be beneficial to QUIC and then describe it at a high level. 3.1. What is a Path? Before going into details, let us first define what is called a "path". A path is a UDP flow between two hosts denoted by a 4-tuple (source IP address, destination IP address, source port, destination port). Considering a smartphone interacting with a single-homed server, the smartphone might want to use one path over the WLAN network and another over the cellular one. Those paths are not necessarily disjoint. For example, when interacting with a dual- De Coninck & BonaventureExpires September 8, 2019 [Page 4] Internet-Draft MP-QUIC March 2019 stack server, a smartphone may create two paths over the Wi-Fi network, one over IPv4 and the other over IPv6. 3.2. Beyond Connection Migration Unlike TCP [RFC0793], QUIC is not bound to a particular 4-tuple during the lifetime of a connection. A QUIC connection is identified by a Connection ID, placed in the public header of each QUIC packet. This enables hosts to continue the connection even if the 4-tuple changes due to, e.g., NAT rebinding. This ability to shift a connection from one 4-tuple to another is called Connection Migration. One of its use cases is fail-over when the IP address in use fails but another one is available. A device losing the WLAN connectivity can then continue the connection over its cellular interface, for instance. A QUIC connection can thus start on a given path, denoted as the initial path, and end on another one. However, the current QUIC design [I-D.ietf-quic-transport] assumes that only one path is in use for a given connection. The specification does not support means to distinguish path migration from simultaneous usage of available paths for a given connection. This document fills that void. Concretely, instead of switching the 4-tuple for the whole connection, this draft first proposes mechanisms to communicate endhost addresses to the peer. It then leverages the address validation process with the PATH_CHALLENGE and PATH_RESPONSE frames proposed in [I-D.ietf-quic-transport] to verify if additional addresses advertised by the communicating host are available and actually belong to it. In this case, those addresses can be used to create new paths to spread packets over several networks following a traffic distribution policy that is out of scope of this document. When multiple paths are available, different delays may be experienced as a function of the initial path selected for the establishment of the QUIC connection. The first version of the specification does not discuss considerations related to the selection of the initial path to place the connection. The example of Figure 1 shows a possible data exchange between a dual-homed client performing a request fitting in two packets and a single-homed server. De Coninck & BonaventureExpires September 8, 2019 [Page 5] Internet-Draft MP-QUIC March 2019 Server Phone Server via WLAN via LTE ------- ------- ----- | Pkt(DCID=B,PN=1,frames=[ | | | STREAM("Request (1/2)")]) | Pkt(DCID=D,PN=1,frames=[ | |<-------------------------------| STREAM("Request (2/2)")]) | | Pkt(DCID=A,PN=1,frames=[ |-------- | | ACK(PID=1,LargestAcked=1)]) | |---------- | |------------------------------->| |---------- | | Pkt(DCID=B,PN=2,frames=[ | |--->| | STREAM("Response 1")]) | Pkt(DCID=C,PN=1,frames=[ | |------------------------------->| ACK(PID=2,LargestAcked=1), | | | STREAM("Response 2")]) -----| | Pkt(DCID=A,PN=2,frames=[ | ----------| | | ACK(PID=1, LargestAcked=2), | ----------| | | ACK(PID=2, LargestAcked=1)]) |<------| | |<-------------------------------| | | Pkt(DCID=B,PN=3,frames=[ | Pkt(DCID=D,PN=2,frames=[ | | STREAM("Response 3")]) | STREAM("Response 4")]) | |------------------------------->| ----| | | ---------| | | ... | ... <---------| | Figure 1: Dataflow with Multipath QUIC The remaining of this section focuses on providing a high-level overview of the multipath operations in QUIC. 3.3. Establishment of a Multipath QUIC Connection A Multipath QUIC connection starts like a regular QUIC connection. A cryptographic handshake takes place with CRYPTO frames and follows the classical process [I-D.ietf-quic-transport] [I-D.ietf-quic-tls]. It is during that process that the multipath capability is negotiated between hosts. This is performed using the max_paths transport parameter, where both hosts advertise their support for the multipath extension, by indicating how many paths can be simultaneously in use. Any value different from 0 indicates that the host wants to support multipath over the connection. If one of the hosts does not advertise the max_paths transport parameter, the negotiated value is 0, meaning that the QUIC connection will not use the multipath extensions presented in this document. The handshake is performed on a given path. This path is called the Initial path and is identified by Path ID 0. Because attaching to new networks may be volatile and an endpoint does not have full visibility on multiple paths that may be available De Coninck & BonaventureExpires September 8, 2019 [Page 6] Internet-Draft MP-QUIC March 2019 (e.g., hosts connected to a CPE), an MP-QUIC capable endhost SHOULD set max_paths to at least 4. 3.4. Architecture of Multipath QUIC Once established, a Multipath QUIC connection is composed of one or more paths. Each path is associated with a different four-tuple and identified by a Path ID, as shown in Figure 2. +-----------------------------------------------+ | Connection (MSCID, MDCID) | | +---------+ +---------+ ... +------------+ | | | Path 0 | | Path 1 | | Path N - 1 | | | | Tuple | | Tuple' | ... | Tuple" | | | | PSCID | | PSCID' | | PSCID" | | | | PDCID | | PDCID' | ... | PDCID" | | | | PN | | PN' | | PN" | | | +---------+ +---------+ ... +------------+ | +-----------------------------------------------+ Figure 2: Architectural view of Multipath QUIC As described before, a Multipath QUIC connection starts on the Initial Path, identified by Path ID 0. For Multipath QUIC, this document proposes two levels of asymmetric Connection IDs. The first ones are the Main (or Primary) Connection IDs (MCIDs). Both the Main Source Connection ID (MSCID) and the Main Destination Connection ID (MDCID) uniquely identify the connection, as with the current QUIC design. The second ones are the Path Connection IDs (PCIDs). Packets belonging to a given path share the same Path Source Connection ID (PSCID) and Path Destination Connection ID (PDCID) written in the Connection ID field of the public header. The PCIDs act as the path identifiers for packets. Preventing the linkability of different paths is an important requirement for the multipath extension [I-D.huitema-quic-mpath-req]. Using PCIDs as implicit path identifier makes this linkability harder than having explicit signaling as in the early version of this draft and does not require public header change to keep invariants [I-D.ietf-quic-invariants]. The MCIDs of a connection will be the PCIDs of the Initial Path. In the example of Figure 1, if the connection started using WLAN, then the Source Connection ID A is both the PSCID of the WLAN path and the MSCID of the connection. In addition to the PCIDs, some additional information is kept for each path. The Path ID identifies the path at the frame level and ensures uniqueness of the nonce (see Section 8.1 for details). A congestion window is maintained for each path. Hosts can also De Coninck & BonaventureExpires September 8, 2019 [Page 7] Internet-Draft MP-QUIC March 2019 collect network measurements on a per-path basis, such as round-trip- time measurements and lost packets. 3.5. Path Establishment The max_paths transport parameter exchanged during the cryptographic handshake determines how many paths can be simultaneously used. The lowest advertised value is the negotiated one. Then, hosts must agree on which paths can be used, and with which Path Connection IDs. Unlike Multipath TCP [RFC6824], both hosts dynamically control how many paths can currently be in use, i.e., the handshake only defines the initial value. This can be done using a new MAX_PATHS frame indicating how many paths can be simultaneously in use. MAX_PATHS frames can be exchanged during the lifetime of a connection. Hosts propose new paths with an extended version of the NEW_CONNECTION_ID frame (see Section 6.1). This frame proposes a PDCID for a given path. Once both hosts proposed a PDCID to their peer and received the acknowledgment for the related NEW_CONNECTION_ID, the Path ID can be used to send packets. Both hosts store the NEW_CONNECTION_ID information in order to cope with the remote trying to use a new path. As frames are encrypted, adding new paths does not leak cleartext identifiers [I-D.huitema-quic-mpath-req]. A server might provide more Path IDs with NEW_CONNECTION_ID frames than the value advertised in the MAX_PATHS frame. This can be useful to cope with migration cases, as described in Section 3.9. In such cases, hosts can only use a subset of the proposed Path IDs. Multipath QUIC is fully symmetrical. Both the client and the server can start using new paths once their corresponding PCIDs have been negotiated. It might happen that both hosts try to create paths with different Path IDs, such that there are more paths than the advertised number in MAX_PATHS frame. In that case, the paths with the lowest Path IDs will be used while the others will stop. Hosts are not able to create new paths as long as the peer does not send NEW_CONNECTION_ID and MAX_PATHS frames. To limit the latency of the path handshake, hosts should send those frames as soon as possible, i.e., just after the 0-RTT handshake packet. Although a path can be first used by any host, it might not be practical for one of the peers to send an initial packet on new paths. A possible cause is when a server wants to initiate a new path to a client behind a NAT. The client would possibly never receive this packet, leading to connectivity issues on that path. To avoid such issues, a remote address MUST have been validated as described in [I-D.ietf-quic-transport] before sending packets on a De Coninck & BonaventureExpires September 8, 2019 [Page 8] Internet-Draft MP-QUIC March 2019 path using it. A host MUST be prepared to receive packets on paths it advertised. Peers MUST NOT require to first receive data from the peer which advertised a path to make use of an available path. 3.6. Exchanging Data over Multiple Paths A QUIC packet acts as a container for one of more frames. Multipath QUIC uses the same STREAM frames as QUIC to carry data. A byte offset is associated to the data payload. One of the key design decision of (Multipath) QUIC is that frames are independent of the packets carrying them. This implies that a frame transmitted over one path could be retransmitted later on another path without any change. The path on which data is sent is a packet-level information. This means a frame can be sent regardless of the path of the packet carrying it. Furthermore, because the data offset is a frame-level information, there is no need to define additional sequence numbers to cope with reordering across paths, unlike Multipath TCP [RFC6824] that uses a Data Sequence Number at the MPTCP level. Other flow control considerations like the stream receive window advertised by the MAX_STREAM_DATA frame remain unchanged when there are multiple paths. However, Multipath QUIC might face reordering at packet-level when using paths having different latencies. The presence of different Path Connection IDs ensures that the packets sent over a given path will contain monotonically increasing packet numbers. To ensure more flexibility and potentially to reduce the ACK block section of the ACK frame when aggregating bandwidth of paths exhibiting different network characteristics, each path keeps its own monotonically increasing Packet Number space. This potentially allows sending up to 2^62 * 2^62 packets on a QUIC connection since each path has its own packet number space. The ACK frame is also modified to allow per-path packet acknowledgments. This remains compliant with the independence between packets and frames while providing more flexibility to hosts to decide on which path they want to send path acknowledgments. Looking again at Figure 1, packets that were sent over a given path (e.g., the response2 packet on path 2 with DCID C) can be acknowledged on another path (here, path 1 with DCID A) to limit the latency due to ACK transmissions on high-latency paths. Such scheduling decision would not have been possible in Multipath TCP [RFC6824] which must acknowledge data on the path it was received on. De Coninck & BonaventureExpires September 8, 2019 [Page 9] Internet-Draft MP-QUIC March 2019 3.7. Exchanging Addresses When a multi-homed mobile device connects to a dual-stacked server using its IPv4 address, it is aware of its local addresses (e.g., the Wi-Fi and the cellular ones) and the IPv4 remote address used to establish the QUIC connection. If the client wants to create new paths over IPv6, it needs to learn the other addresses of the remote peer. This is possible with the ADD_ADDRESS frames that are sent by a Multipath QUIC host to advertise its current addresses. Each advertised address is identified by an Address ID. The addresses attached to a host can vary during the lifetime of a Multipath QUIC connection. A new ADD_ADDRESS frame is transmitted when a host has a new address. This ADD_ADDRESS frame is protected as other QUIC control frames, which implies that it cannot be spoofed by attackers. The communicated address is first validated by the receiving host before it starts using it. This ensures that the address actually belongs to the peer and that the peer can send and receive packets on that address. It also prevents hosts from launching amplification attacks to a victim address. If the client is behind a NAT, it could announce a private address in an ADD_ADDRESS frame. In such situations, the server would not be able to validate the communicated address. The client might still use its NATed address to start a new path. To enable the server to make the link between the private and the public addresses, Multipath QUIC provides the PATHS frame that lists current active Path IDs of the sending host. Likewise, the client may be located behind a NAT64. As such it may announce an IPv6 address in an ADD_ADDRESS frame, that will be received over IPv4 by an IPv4-only server. The server should not discard that address, even if it is not IPv6-capable. An IPv6-only client may also receive from the server an ADD_ADDRESS frame which may contain an IPv4 address. The client should rely on means, such as [RFC7050] or [RFC7225], to learn the IPv6 prefix to build an IPv4-converted IPv6 address. A path is called active when it was created over a validated 4-tuple and is still in use. The frame indicates the local Address ID that the path uses. With this information, the server can then validate the public address and associate the advertised with the perceived addresses. Hosts that are connected behind an address sharing mechanism may collect the external IP address and port numbers assigned to the De Coninck & BonaventureExpires September 8, 2019 [Page 10] Internet-Draft MP-QUIC March 2019 hosts and then use there addresses in the ADD_ADDRESS. Means to gather such information include, but not limited to, UPnP IGD, PCP, or STUN. 3.8. Coping with Address Removals During the lifetime of a QUIC connection, a host might lose some of its addresses. A concrete example is a smartphone going out of reachability of a Wi-Fi network or shutting off one of its network interfaces. Such address removals are advertised by using REMOVE_ADDRESS frames. The REMOVE_ADDRESS frame contains the Address ID of the lost address previously communicated through ADD_ADDRESS. 3.9. Path Migration At a given time, a Multipath QUIC endpoint gathers a set of paths, each using a 4-tuple. To address privacy issues due to the linkability of addresses with Connection IDs, hosts should avoid changing the 4-tuple used by a path. There remain situations where this change is unavoidable. These can be categorized into two groups: host-aware changes (e.g., network handover from Wi-Fi to cellular) and host-unaware changes (e.g., NAT rebinding). For the host-aware case, let us consider the case of a Multipath QUIC connection where the client is a smartphone with both Wi-Fi and cellular. It advertised both addresses and the server currently enables only one path, the initial one. The Initial Path uses the Wi-Fi address. Then, for some reason, the Wi-Fi address becomes unusable. To preserve connectivity, the client might then decide to use the cellular address for the Initial Path. It thus sends a REMOVE_ADDRESS announcing the loss of the Wi-Fi address and a PATHS frame to inform that the Initial Path is now using the cellular address. If the cellular address validation succeeds (which could have been done as soon as the cellular address was advertised), the server can continue exchanging data through the cellular address. However, both server and client might want to change the path used on the cellular address for privacy concerns. If the server provides an additional path (e.g., Path ID 42) through NEW_CONNECTION_ID frame at the beginning of the connection, the client can perform the path change directly and avoid using the Initial Path Connection ID on the cellular network. This can be done using the PATH_UPDATE frame. It can indicate that the host stopped to use the Initial Path to use Path ID 42 instead. This frame is placed in the first packet sent to the new path with its corresponding PCID. The client can then send the REMOVE_ADDRESS and PATHS frames on this new path. Compared to the previous case, it is harder to link the paths with the IP De Coninck & BonaventureExpires September 8, 2019 [Page 11] Internet-Draft MP-QUIC March 2019 addresses to observe that they belong to the same Multipath QUIC connection. For the host-unaware case, the situation is similar. In case of NAT rebinding, the server will observe a change in the 2-tuple (source IP, source port) of the packet. The server first validates that the 2-tuple actually belongs to the client [I-D.ietf-quic-transport]. If it is the case, the server can send a PATH_UPDATE frame on a previously communicated but unused Path ID. The client might have sent some packets with a given PCID on a different 4-tuple, but the server did not use the given PCID on that 4-tuple. Because some on- path devices may rewrite the source IP address to forward packets via the available network attachments (e.g., an host located behind a multi-homed CPE), the server may inadvertently conclude that a path is not anymore valid leading thus to frequently sending PATH_UPDATE frames as a function of the traffic distribution scheme enforced by the on-path device. To prevent such behavior, the server SHOULD wait for at least X seconds to ensure this is about a connection migration and not a side effect of an on-path multi-interfaced device. 3.10. Sharing Path Policies Some access networks are subject to a volume quota. To prevent a peer from aggressively using a given path while available resources can be freely grabbed using existing paths, it is desirable to support a signal to indicate to a remote peer how it must place data into available paths. An approach may consist in indicating in an ADD_ADDRESS the type of the interface (e.g., cellular, WLAN, fixed): interface-type. The remote peer may rely on interface-type to select the path to be used for sending data. For example, fixed interfaces will be preferred over WLAN and cellular interfaces, and WLAN interface will be preferred over cellular interface. This information might also be used to avoid draining battery for some devices. 3.11. Congestion Control The QUIC congestion control scheme is defined in [I-D.ietf-quic-recovery]. This congestion control scheme is not suitable when several paths are active. Using the congestion control scheme defined in [I-D.ietf-quic-recovery] with Multipath QUIC would result in unfairness. Each path of a Multipath QUIC connection MUST have its own congestion window. The windows of the different paths MUST be coupled together. Multipath TCP uses the LIA congestion control scheme specified in [RFC6356]. This scheme can immediately be adapted to Multipath QUIC. Other coupled congestion control schemes have been proposed for Multipath TCP such as [OLIA]. De Coninck & BonaventureExpires September 8, 2019 [Page 12] Internet-Draft MP-QUIC March 2019 4. Mapping Path ID to Connection IDs As described in the overview section, hosts need to identify on which path packets are sent. The Path ID must then be inferred from the public header. This is done by using Path Connection IDs in addition to Main Connection IDs. The Master Connection IDs is determined during the cryptographic handshake and actually corresponds to both Connection IDs in the current QUIC design [I-D.ietf-quic-transport]. The Path Connection IDs of the Initial Path (with Path ID 0) are equal to the Main Connection IDs. The Path Connection IDs of the other paths are determined when the NEW_CONNECTION_ID frames are exchanged. The server MUST ensure that all advertised Path Connection IDs are available for the whole connection lifetime. Once it sends a NEW_CONNECTION_ID frame containing a PCID, the server can start receiving packets with the advertised Connection ID as belonging to the corresponding path. The server MUST wait until the reception of the frame acknowledgment before starting to send packets on that path. Upon reception of the NEW_CONNECTION_ID frame, the client MUST acknowledge it and MUST store the advertised Destination Path Connection ID and the Path ID of the proposed path. It MUST ensure that it can receive packets coming from the server using the PCID and associate it with the corresponding Path ID. 5. Using Multiple Paths This section describes in details the multipath QUIC operations. 5.1. Multipath Negotiation The Multipath Negotiation takes place during the cryptographic handshake with the max_paths transport parameter. A QUIC connection is initially single-path in QUIC, and all packets prior to handshake completion MUST be exchanged over the Initial Path. During this process, hosts advertise their support for multipath operations. Any value different from 0 indicates that the host supports multipath operations over the connection, i.e., the extensions defined in this document (not to be mixed with the availability of local multiple paths). If a host does not send the max_paths transport parameter during the cryptographic handshake, the remote MUST assume a value of 0, leading to a single-path connection over the Initial Path. If both hosts advertise their support of the multipath extensions, NEW_CONNECTION_ID frames can later propose Path IDs that can then be used for the connection. De Coninck & BonaventureExpires September 8, 2019 [Page 13] Internet-Draft MP-QUIC March 2019 5.1.1. Transport Parameter Definition An endhost MAY use the following transport parameter: max_paths (0x0020): The maximum number of paths that can be simultaneously active, encoded as an unsigned 16-bit integer. If absent, the value for this parameter is 0, meaning that a host omitting this transport parameter does not agree to use the multipath extension over the connection. 5.2. Coping with Additional Remote Addresses The usage of multiple networks paths is often done using multiple IP addresses. For instance, a smartphone willing to use both Wi-Fi and LTE will use the corresponding addresses assigned by these networks. It can then safely send packets to a previously-used IP address of a server. The server can receive packets sourced with different IP addresses, but it MUST first validate the new remote IP addresses before starting sending packets to those addresses. Similarly, additional addresses could be communicated using ADD_ADDRESS frames. Such addresses MUST be validated before starting to send packets to them. This requirement is explained in Section 8.2. The validation of an address could be performed with PATH_CHALLENGE and PATH_RESPONSE frames as described in [I-D.ietf-quic-transport]. A validated address MAY be cached for a given host for a limited amount of time. 5.3. Path State During the Multipath QUIC connection, hosts maintain some state for paths. Information about the path that hosts are required to store depends on its state. The possible path states are depicted in Figure 3. De Coninck & BonaventureExpires September 8, 2019 [Page 14] Internet-Draft MP-QUIC March 2019 +----------+ | PROPOSED | +----------+ | both hosts exchanged PCIDs for the path v +----------+ | READY | +----------+ | | path usage | | restricted by negotiated MAX_PATHS v or affected by REMOVE_ADDRESS +--------+ --------------------------> +----------+ | ACTIVE | reusing path | UNUSABLE | +--------+ <-------------------------- +----------+ | | | PATH_UPDATE | +---------------------------------------+ | v +--------+ | CLOSED | +--------+ Figure 3: Finite-State Machine of paths Once a path has been proposed in a NEW_CONNECTION_ID frame, either sent or received, it is in the PROPOSED state. In this situation, hosts MUST keep the following path information: o Path ID: encoded as a 4-byte integer. It uniquely identifies the path in the connection. This value is immutable. o Main Connection IDs: they make the link between the path and the QUIC connection it belongs to. These values are immutable. o Path Connection IDs: they make the link between the packet's Connection ID field and the path. This value can be updated with subsequent NEW_CONNECTION_ID frames. o Path State: the current state of the path, one of the values presented in Figure 3. Once both hosts exchanged both asymmetric PCIDs for a given Path ID, the path enters the READY state. In this state, hosts MUST ensure that they can associate a packet with the PCIDs and its corresponding De Coninck & BonaventureExpires September 8, 2019 [Page 15] Internet-Draft MP-QUIC March 2019 path at any time, as they must be ready to figure out the path used by the remote host. Either the host received a packet with the corresponding PCIDs coming from a validated address or it wants to start using the path to a validated address, the path goes to the ACTIVE state. This is the state where a path can be used to send and receive packets. In addition to the fields required in the PROPOSED state, the following elements MUST be tracked: o Packet Number Space: each path is associated with its own monotonically increasing packet number space. Each endpoint maintains a separate packet number for sending and receiving packets. Packet number considerations described in [I-D.ietf-quic-transport] apply within a given path. o Current 4-tuple: the tuple (sIP, dIP, sport, dport) used to send or receive packets over this path. This value is mutable, either because the host decides to change its local address and/or port, or because it receives a packet with a different validated remote address and/or port than the one currently recorded. A host that changes the 4-tuple of a path SHOULD migrate it. o Current (local Address ID, remote Address ID) tuple: those identifiers come from the ADD_ADDRESS sent (local) and received (remote). This enables a host to mark a path as UNUSABLE when, e.g., the remote Address ID is declared as lost in a REMOVE_ADDRESS. The addresses on which the connection was established have Address ID 0. The reception of PATHS frames helps hosts to associate the remote Address ID used by the path. o Performance metrics: basic statistics such as round-trip-time or number of packets sent and received can be collected on a per-path basis. This information can be useful when a host needs to perform packet scheduling decisions and flow control management. It might happen that a path is temporarily unavailable, because one of the endpoint's addresses is no more available or because the server decided to decrease the number of active paths. In such cases, the path is in UNUSABLE state and the host MUST NOT send packets on it. The host keeps the same information as in the ACTIVE state for the path. It might happen that packets are received on that path. If the path is in UNUSABLE state because of a decrease of the active paths, packets MUST be discarded silently. If it is because a REMOVE_ADDRESS was received for the remote Address ID of the path, the host SHOULD validate the remote address first before reusing it. If this validation succeeds, or the number of paths De Coninck & BonaventureExpires September 8, 2019 [Page 16] Internet-Draft MP-QUIC March 2019 increased again, a path in the UNUSABLE state can go into ACTIVE state, but its congestion window MUST be restarted and its performance metrics SHOULD be reset. Eventually, a path may either be migrated to another one or closed. This is signaled by the PATH_UPDATE frame. In that case, the path is in CLOSED state. In that state, packets MUST NOT be sent or received over it. A host MUST keep the same path state field as in the PROPOSED state, to avoid ambiguities and CLOSED path reuse. 5.4. Dynamic Management of Paths Both hosts determine how many paths can be used over a Multipath QUIC connection. It is their responsibility to propose Path IDs with corresponding PCIDs that could be used by the peer. In addition, they dynamically control the number of active paths that can be used for the connection, initially determined during the handshake with the max_paths transport parameter. This is performed by sending MAX_PATHS frames that set an upper limit on the number of active paths. At any time, the maximum number of active paths that could be present over a connection is the minimum value advertised by peers. A host can propose more Path IDs with NEW_CONNECTION_ID frames than the number of paths it agrees to use simultaneously. This can be useful in migration scenarios, where the host wants to share a pool of Path IDs that can be directly used to migrate paths. A host can also decrease the number of paths that can be used over a connection. If this value decreases below the current number of active paths, hosts MUST put in UNUSABLE state the paths in the ACTIVE state with the highest Path IDs. Server and client MUST stop sending packets over UNUSABLE paths once the MAX_PATHS has been sent or received. The host restricting the number of paths SHOULD allow receiving packets on newly UNUSABLE paths for one round-trip-time. After this delay, hosts SHOULD silently discard received packets. When the server proposes more Path IDs than the negotiated maximum number of ACTIVE paths, it might happen that both hosts decide at the same time to create paths with different Path IDs, such as there are more created paths than the negotiated maximum value. In this case, the hosts MUST only keep as ACTIVE the paths with the lowest Path IDs. Paths with the highest Path IDs are in the UNUSABLE state and packets SHOULD be silently discarded. 5.5. Losing Addresses During the lifetime of a connection, a host might lose addresses, e.g., a network interface that was shut down. All the ACTIVE paths that were using that local address MUST be set in the UNUSABLE state. De Coninck & BonaventureExpires September 8, 2019 [Page 17] Internet-Draft MP-QUIC March 2019 To advertise the address loss to the peer, the host MUST send a REMOVE_ADDRESS frame indicating which Address IDs has been lost. The host MUST also send a PATHS frame indicating the status of the remaining ACTIVE paths. Upon reception of the REMOVE_ADDRESS, the receiving host MUST set the ACTIVE paths affected by the address removal into the UNUSABLE state. The host that locally lost the address MAY reuse one of these paths by changing the assigned 4-tuple. In this case, it MUST send a PATHS frame describing that change. 5.6. Scheduling Strategies The current QUIC design [I-D.ietf-quic-transport] offers a two- dimensional scheduling space, i.e., which frames will be packed inside a given packet. With the use of multiple paths, a third dimension is added, i.e., the path on which the packet will be sent. This dimension can have a non negligible impact on the operations of Multipath QUIC, especially if the available paths exhibit very different network characteristics. The progression of the data flow depends on the reception of the MAX_DATA and MAX_STREAM_DATA frames. Those frames SHOULD be duplicated on several or all paths in use. This would limit the head-of-line blocking issue due to the transmission of the frames over a slow path. The path on which ACK frames are sent impacts the peer. The ACK frame is notably used to determine the latency of a path. If the ACK frame is sent on the path it acknowledges, then the peer can compute the round-trip-time of that path. Otherwise, the peer would compute the latency as the sum of the forward delay of the acknowledged path and the return delay of the path used to send the ACK frame. Choosing between acknowledging packets on the same path or on a specific path is up to the implementation. However, hosts SHOULD keep a consistent acknowledgement strategy. Selecting a random path to acknowledge packets will possibly increase the variability of the latency estimation, especially if paths exhibit very different network characteristics. Unlike MAX_DATA and MAX_STREAM_DATA, ACK frames SHOULD NOT be systematically duplicated on several paths as they can induce a large network overhead. 6. Modifications to QUIC frames The multipath extension allows hosts to send packets over multiple paths. Since nearly all QUIC frames are independent of packets, no change is required for most of them. The only exceptions are the De Coninck & BonaventureExpires September 8, 2019 [Page 18] Internet-Draft MP-QUIC March 2019 NEW_CONNECTION_ID and the ACK frames. The NEW_CONNECTION_ID is modified to provide Path Connection ID negotiation for each path. The ACK frame contains packet-level information with the Largest Acknowledged field. Since the Packet Numbers are now associated to paths, the ACK frame must contain the Path ID it acknowledges. 6.1. NEW_CONNECTION_ID Frame The NEW_CONNECTION_ID frame (type=0x0b) as defined by [I-D.ietf-quic-transport] keeps its ability to provide the client with alternative connection IDs that can be used to break linkability when migrating connections. It also allows the server to indicate which connection IDs the client must use to take advantage of multiple paths. The NEW_CONNECTION_ID is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path ID (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length (8) | | +-+-+-+-+-+-+-+-+ Connection ID (32..144) + | ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Stateless Reset Token (128) + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: NEW_CONNECTION_ID frame adapted to Multipath QUIC Compared to the frame specified in [I-D.ietf-quic-transport], a Path ID field of variable size is prefixed to associate the Path ID with the Connection ID. If the multipath extension was not negotiated during the connection establishment, the NEW_CONNECTION_ID frame is the same as the one presented in [I-D.ietf-quic-transport]. This frame can be sent by both hosts. Upon reception of the frame with a specified path ID, the peer can update the related path state either to PROPOSED or READY and store the communicated Connection ID as the Destination Connection ID of the path. De Coninck & BonaventureExpires September 8, 2019 [Page 19] Internet-Draft MP-QUIC March 2019 A host MUST NOT start using a path as long as both peers have not proposed their Destination Connection ID with NEW_CONNECTION_ID frames. To limit the delay of the multipath usage upon handshake completion, hosts SHOULD send NEW_CONNECTION_ID frames for paths they allow using as soon the connection establishment completes. To cope with privacy issues, it should be hard to make the link between two different connections or two different paths of a same connection by just looking at the Connection ID contained in packets. Therefore, Path Connection IDs chosen by hosts MUST be random. 6.2. RETIRE_CONNECTION_ID Frame The RETIRE_CONNECTION_ID frame (type=0x19) as defined by [I-D.ietf-quic-transport] still allows an endpoint to indicate that it will no longer use a connection ID that was issued by its peer. The multipath extensions link Connection IDs to paths. Therefore, this frame should contains the Path ID on which it applies. The format of the adapted RETIRE_CONNECTION_ID is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path ID (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5: RETIRE_CONNECTION_ID frame adapted to Multipath QUIC The frame is handled as described in [I-D.ietf-quic-transport] on a path basis. 6.3. ACK Frame The format of the modified ACK frame is shown below. De Coninck & BonaventureExpires September 8, 2019 [Page 20] Internet-Draft MP-QUIC March 2019 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path ID (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Largest Acknowledged (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ACK Delay (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ACK Range Count (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | First ACK Range (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ACK Ranges (*) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [ECN Counts] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 6: ACK frame adapted to Multipath Compared to the ACK frame in the current QUIC design [I-D.ietf-quic-transport], the ACK frame is prefixed by a variable size Path ID field indicating to which path the acknowledged PSNs relate to. Notice that if the multipath extension was not negotiated during the connection handshake, the ACK frame is the same as the one presented in [I-D.ietf-quic-transport]. Since frames are independent of packets, and the path notion relates to the packets, the ACK frames can be sent on any path, unlike Multipath TCP [RFC6824] which is constrained to send ACKs on the same path. 7. New Frames To support the multipath operations, new frames have been defined to coordinate hosts. This draft uses a type field containing 0x20 to indicate that the frame is related to multipath operations. 7.1. MAX_PATHS Frame The MAX_PATHS frame is used by hosts to control the number of paths that can be simultaneously in the ACTIVE state on a given connection. The proposed type for the MAX_PATHS frame is 0x20. A MAX_PATHS frame is shown below. De Coninck & BonaventureExpires September 8, 2019 [Page 21] Internet-Draft MP-QUIC March 2019 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Num Additional Active Paths (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 7: MAX_PATHS Frame The MAX_PATHS frame contains the following fields: o Sequence: A variable-length integer. This value starts at 0 and increases by 1 for each change of number of active paths that is provided by the host. o Num Additional Active Paths: A variable-length integer indicating how many additional paths can be used over the connection. The number of paths that can be in ACTIVE state is thus (Num Additional Active Paths + 1). Once the connection is established, hosts MUST assume that the server sent a MAX_PATHS frame with sequence -1 and number of additional active paths equal to 0 and the client sent a MAX_PATHS frame with sequence -1 and number of additional active paths equal to the negotiated max_path_id value. 7.2. PATH_UPDATE Frame The PATH_UPDATE frame is used by a host either to migrate a path or to close it. This indicates to the remote that the closed path MUST NOT be used anymore and it can use the proposed one instead, if any. The proposed type for the PATH_UPDATE is 0x21. A PATH_UPDATE frame is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Closed Path ID (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Proposed Path ID (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 8: PATH_UPDATE Frame The PATH_UPDATE frame contains the following fields: De Coninck & BonaventureExpires September 8, 2019 [Page 22] Internet-Draft MP-QUIC March 2019 o Closed Path ID: A variable-length integer corresponding to the Path ID of the path that is closed. o Proposed Path ID: A variable-length integer corresponding to the Path ID of the path that substitutes the closed path. If the value is 0, no path is proposed. Upon the transmission or the reception of the PATH_UPDATE frame, the path with the Path ID referenced in Closed Path ID MUST be in the CLOSED state. If the proposed Path ID is different of 0, the path MUST have been either in the PROPOSED or in the UNUSABLE state and MUST now be considered as ACTIVE. 7.3. ADD_ADDRESS Frame The ADD_ADDRESS frame is used by a host to advertise its currently reachable addresses. The proposed type for the ADD_ADDRESS frame is 0x22. An ADD_ADDRESS frame is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0|0|P|IPVers.|Address ID (8) |Interface T.(8)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP Address (32/128) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Port (16)] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 9: ADD_ADDRESS Frame The ADD_ADDRESS frame contains the following fields: o Reserved bits: the three most-significant bits of the first byte are set to 0, and are reserved for future use. o P bit: the fourth most-significant bit of the first byte indicates, if set, the presence of the Port field. o IPVers.: the remaining four least-significant bits of the first byte contain the version of the IP address contained in the IP Address field. o Address ID: an unique identifier for the advertised address for tracking and removal purposes. This is needed when, e.g., a NAT changes the IP address such that both hosts see different IP addresses for a same path endpoint. De Coninck & BonaventureExpires September 8, 2019 [Page 23] Internet-Draft MP-QUIC March 2019 o Interface Type: used to provide an indication about the interface type to which this address is bound. The following values are defined: o 0: fixed. Used as default value. o 1: WLAN o 2: cellular o IP Address: the advertised IP address, in network order. o Port: this optional field indicates the port number related to the advertised IP address. When this field is present, it indicates that a path can use the 2-tuple (IP addr, port). Upon reception of an ADD_ADDRESS frame, the receiver SHOULD store the communicated address for future use. The receiver MUST NOT send packets others than validation ones to the communicated address without having validated it. ADD_ADDRESS frames SHOULD contain globally reachable addresses. Link-local and possibly private addresses SHOULD NOT be exchanged. 7.4. REMOVE_ADDRESS Frame The REMOVE_ADDRESS frame is used by a host to signal that a previously announced address was lost. The proposed type for the REMOVE_ADDRESS frame is 0x23. A REMOVE_ADDRESS frame is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+ |Address ID (8) | +-+-+-+-+-+-+-+-+ Figure 10: REMOVE_ADDRESS Frame The frame contains only one field, Address ID, being the identifier of the address to remove. A host SHOULD stop using paths using the removed address and set them in the UNUSABLE state. If the REMOVE_ADDRESS contains an Address ID that was not previously announced, the receiver MUST silently ignore the frame. 7.5. PATHS Frame The PATHS frame communicates the paths state of the sending host to the peer. It allows the sender to communicate its active paths to the peer in order to detect potential connectivity issue over paths. Its proposed type is 0x24. The format of the PATHS frame is shown below. De Coninck & BonaventureExpires September 8, 2019 [Page 24] Internet-Draft MP-QUIC March 2019 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ActivePaths (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path Info Section (*) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 11: PATHS Frame The PATHS frame contains the following fields: o Sequence: A variable-length integer. This value starts at 0 and increases by 1 for each PATHS frame sent by the host. It allows identifying the most recent PATHS frame. o ActivePaths: the current number of additional paths considered as being active from the sender point of view, i.e., (the number of active paths - 1). ActivePaths MUST be smaller or equal to the last value advertised by MAX_PATHS frame. o Path Info Section: contains information about all the active paths (i.e., there are ActivePaths + 1 entries). The format of this section is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path ID 0 (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |LocAddrID 0 (8)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path ID N (i) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |LocAddrID N (8)| +-+-+-+-+-+-+-+-+ Figure 12: Path Info Section The fields in the Path Info Section are: o Path ID: the Path ID of the active path the sending host provides information about. De Coninck & BonaventureExpires September 8, 2019 [Page 25] Internet-Draft MP-QUIC March 2019 o LocAddrID: the local Address ID of the address currently used by the path. The Path Info section only contains the Local Address ID so far, but this section can be extended later with other potentially useful information. 8. Security Considerations 8.1. Nonce Computation With Multipath QUIC, each path has its own packet number space. With the current nonce computation [I-D.ietf-quic-tls], using twice the same packet number over two different paths leads to the same cryptographic nonce. Depending on the size of the Initial Value (and hence the nonce), there are two ways to mitigate this issue. If the Initial Value has a length of 8 bytes, then a packet number used on a given path MUST NOT be reused on another path of the connection, and therefore at most 2^64 packets can be sent on a QUIC connection. This means there will be packet number skipping at path level, but the packet number will remain monotonically increasing on each path. If the Initial Value has a length of 9 or more, then the cryptographic nonce computation is now performed as follows. The nonce, N, is formed by combining the packet protection IV (either client_pp_iv_n or server_pp_iv_n) with the Path ID and the packet number. The 64 bits of the reconstructed QUIC packet number in network byte order is left-padded with zeros to the size of the IV. The Path ID encoded in its variable-length format is right-padded with zeros to the size of the IV. The Path IV is computed as the exclusive OR of the padded Path ID and the IV. The exclusive OR of the padded packet number and the Path IV forms the AEAD nonce. 8.2. Validation of Exchanged Addresses To use addresses communicated by the peer through ADD_ADDRESS frames, hosts are required to validate them before using paths to these addresses. The main reason for this validation is that the remote host might have sent, purposely or not, a packet with a source IP that does not correspond to the IP of the remote interface. This could lead to amplification attacks where the client start using a new path with a source IP corresponding to the victim's one. Without validation, the server might then flood the victim. Similarly for ADD_ADDRESS frames, a malicious server might advertise the IP address of a victim, hoping that the client will use it without validating it before. De Coninck & BonaventureExpires September 8, 2019 [Page 26] Internet-Draft MP-QUIC March 2019 9. IANA Considerations 9.1. QUIC Transport Parameter Registry This document defines a new transport parameter for the negotiation of multiple paths. The following entry in Table 1 should be added to the "QUIC Transport Parameters" registry under the "QUIC Protocol" heading. +--------+----------------+---------------+ | Value | Parameter Name | Specification | +--------+----------------+---------------+ | 0x0020 | max_paths | Section 5.1.1 | +--------+----------------+---------------+ Table 1: Addition to QUIC Transport Parameters Entries 10. Acknowledgements We would like to thanks Masahiro Kozuka and Kazuho Oku for their numerous comments on the first version of this draft. We also thanks Philipp Tiesel for his early comments that led to the current design, and Ian Swett for later feedbacks. We also want to thank Christian Huitema for his draft about multipath requirements to identify critical elements about the multipath feature. Mohamed Boucadair provided lot of useful inputs on the second version of this document. 11. References 11.1. Normative References [I-D.ietf-quic-invariants] Thomson, M., "Version-Independent Properties of QUIC", draft-ietf-quic-invariants-03 (work in progress), October 2018. [I-D.ietf-quic-recovery] Iyengar, J. and I. Swett, "QUIC Loss Detection and Congestion Control", draft-ietf-quic-recovery-18 (work in progress), January 2019. [I-D.ietf-quic-tls] Thomson, M. and S. Turner, "Using TLS to Secure QUIC", draft-ietf-quic-tls-18 (work in progress), January 2019. De Coninck & BonaventureExpires September 8, 2019 [Page 27] Internet-Draft MP-QUIC March 2019 [I-D.ietf-quic-transport] Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed and Secure Transport", draft-ietf-quic-transport-18 (work in progress), January 2019. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . 11.2. Informative References [Cellnet] Paasch, C., Detal, G., Duchene, F., Raiciu, C., and O. Bonaventure, "Exploring Mobile/Wi-Fi Handover with Multipath TCP", ACM SIGCOMM workshop on Cellular Networks (Cellnet'12) , 2012. [I-D.huitema-quic-mpath-req] Huitema, C., "QUIC Multipath Requirements", draft-huitema- quic-mpath-req-01 (work in progress), January 2018. [IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments", IETF Journal , November 2016. [MPQUIC] De Coninck, Q. and O. Bonaventure, "Multipath QUIC: Design and Evaluation", 13th International Conference on emerging Networking EXperiments and Technologies (CoNEXT 2017). http://multipath-quic.org , December 2017. [MPRTP] Singh, V., Ahsan, S., and J. Ott, "MPRTP: Multipath considerations for real-time media", Proceedings of the 4th ACM Multimedia Systems Conference , 2013. [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J. Le Boudec, "MPTCP is not pareto-optimal: performance issues and a possible solution", Proceedings of the 8th international conference on Emerging networking experiments and technologies, ACM , 2012. [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, DOI 10.17487/RFC0793, September 1981, . [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled Congestion Control for Multipath Transport Protocols", RFC 6356, DOI 10.17487/RFC6356, October 2011, . De Coninck & BonaventureExpires September 8, 2019 [Page 28] Internet-Draft MP-QUIC March 2019 [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, . [RFC7050] Savolainen, T., Korhonen, J., and D. Wing, "Discovery of the IPv6 Prefix Used for IPv6 Address Synthesis", RFC 7050, DOI 10.17487/RFC7050, November 2013, . [RFC7225] Boucadair, M., "Discovering NAT64 IPv6 Prefixes Using the Port Control Protocol (PCP)", RFC 7225, DOI 10.17487/RFC7225, May 2014, . [RFC8041] Bonaventure, O., Paasch, C., and G. Detal, "Use Cases and Operational Experience with Multipath TCP", RFC 8041, DOI 10.17487/RFC8041, January 2017, . [SCTPCMT] Iyengar, J., Amer, P., and R. Stewart, "Concurrent multipath transfer using SCTP multihoming over independent end-to-end paths", IEEE/ACM Transactions on networking, Vol. 14, no 5 , 2006. Appendix A. Change Log A.1. Since draft-deconinck-quic-multipath-01 o Include path policies considerations o Add practical considerations thanks to Mohamed Boucadair inputs o Adapt the RETIRE_CONNECTION_ID frame o Updated text to match draft-ietf-quic-transport-18 A.2. Since draft-deconinck-quic-multipath-00 o Comply with asymmetric Connection IDs o Add max_paths transport parameter to negotiate initial number of active paths o Path ID as a regular varint o Remove max_path_id transport parameter De Coninck & BonaventureExpires September 8, 2019 [Page 29] Internet-Draft MP-QUIC March 2019 o Updated text to match draft-ietf-quic-transport-14 A.3. Since draft-deconinck-multipath-quic-00 o Added PATH_UPDATE frame o Added MAX_PATHS frame o No more packet header change o Implicit Path ID notification using Connection ID and NEW_CONNECTION_ID frames o Variable-length encoding for Path ID o Updated text to match draft-ietf-quic-transport-10 o Fixed various typos Authors' Addresses Quentin De Coninck UCLouvain Email: quentin.deconinck@uclouvain.be Olivier Bonaventure UCLouvain Email: Olivier.Bonaventure@uclouvain.be De Coninck & BonaventureExpires September 8, 2019 [Page 30]