Network Working Group F. L. Templin, Ed. Internet-Draft Boeing Research & Technology Updates: RFC2675 (if approved) 24 January 2023 Intended status: Standards Track Expires: 28 July 2023 IP Parcels draft-templin-intarea-parcels-30 Abstract IP packets (both IPv4 and IPv6) contain a single unit of upper layer protocol data which becomes the retransmission unit in case of loss. Upper layer protocols including the Transmission Control Protocol (TCP) and transports over the User Datagram Protocol (UDP) prepare data units known as "segments", with traditional arrangements including a single segment per IP packet. This document presents a new construct known as the "IP Parcel" which permits a single packet to carry multiple upper layer protocol segments, essentially creating a "packet-of-packets". IP parcels provide an essential building block for improved performance, efficiency and integrity while encouraging larger Maximum Transmission Units (MTUs) in the Internet. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 28 July 2023. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. Templin Expires 28 July 2023 [Page 1] Internet-Draft IP Parcels January 2023 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Background and Motivation . . . . . . . . . . . . . . . . . . 6 4. IP Parcel Formation . . . . . . . . . . . . . . . . . . . . . 7 5. TCP Parcels . . . . . . . . . . . . . . . . . . . . . . . . . 12 6. UDP Parcels . . . . . . . . . . . . . . . . . . . . . . . . . 13 7. Transmission of IP Parcels . . . . . . . . . . . . . . . . . 14 7.1. Singleton IP Packets over Non-Parcel Links . . . . . . . 15 7.2. Sub-Parcels over Parcel-capable Links . . . . . . . . . . 16 7.3. Parcels/Sub-Parcels over OMNI Interfaces . . . . . . . . 17 7.4. Final Destination Reassembly . . . . . . . . . . . . . . 19 8. Parcel Path Qualification . . . . . . . . . . . . . . . . . . 20 9. IP Jumbograms . . . . . . . . . . . . . . . . . . . . . . . . 24 10. Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . 26 11. Implementation Status . . . . . . . . . . . . . . . . . . . . 29 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30 13. Security Considerations . . . . . . . . . . . . . . . . . . . 30 14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 30 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 30 15.1. Normative References . . . . . . . . . . . . . . . . . . 31 15.2. Informative References . . . . . . . . . . . . . . . . . 31 Appendix A. IP Parcel Futures . . . . . . . . . . . . . . . . . 34 Appendix B. Change Log . . . . . . . . . . . . . . . . . . . . . 35 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 35 1. Introduction IP packets (both IPv4 [RFC0791] and IPv6 [RFC8200]) contain a single unit of upper layer protocol data which becomes the retransmission unit in case of loss. Upper layer protocols such as the Transmission Control Protocol (TCP) [RFC9293] and transports over the User Datagram Protocol (UDP) [RFC0768] (including QUIC [RFC9000], LTP [RFC5326] and others) prepare data units known as "segments", with traditional arrangements including a single segment per IP packet. This document presents a new construct known as the "IP Parcel" which permits a single packet to carry multiple upper layer protocol segments. This essentially creates a "packet-of-packets" with the IP Templin Expires 28 July 2023 [Page 2] Internet-Draft IP Parcels January 2023 layer and full {TCP,UDP} headers appearing only once but with possibly more than one segment included. Parcels are formed when an upper layer protocol entity identified by the "5-tuple" (source address, destination address, source port, destination port, protocol number) prepares a data buffer beginning with an Integrity Block of up to 256 2-octet Checksums followed by their corresponding upper layer protocol segments that can be broken out into smaller sub-parcels and/or individual packets if necessary. All segments except the final one must be equal in length and no larger than 65535 octets (minus headers), while the final segment must not be larger than the others but may be smaller. The upper layer protocol entity then delivers the buffer, number of segments and non-final segment size to lower layers which append a {TCP,UDP} header and an IP header plus extensions that identify this as a parcel and not an ordinary packet. Parcels can be forwarded over consecutive parcel-capable links in a path until arriving at a router where the next hop is via a link that does not support parcels, a parcel-capable link with a size restriction, or an ingress middlebox Overlay Multilink Network (OMNI) Interface [I-D.templin-intarea-omni] that spans intermediate Internetworks using adaptation layer encapsulation and fragmentation. In the first case, the router breaks the parcel into individual IP packets and forwards them via the next hop link. In the second case, the router breaks the parcel into smaller sub-parcels and forwards them via the next hop link. In the final case, the OMNI interface breaks the parcel into smaller sub-parcels if necessary then applies adaptation layer encapsulation and fragmentation if necessary. These OMNI interface sub-parcels may then be recombined into one or more larger parcels by an egress middlebox OMNI interface which either delivers them locally or forwards them over additional parcel- capable links on the path to the final destination. The final destination can then further re-combine sub-parcels of the same original parcel so as to present the largest possible data unit to upper layers. Reordering and even loss or damage of individual segments within the network is therefore possible, but what matters is that the parcels delivered to the final destination should be the largest practical size for best performance and that loss or receipt of individual segments (and not parcel size) determines the retransmission unit. Templin Expires 28 July 2023 [Page 3] Internet-Draft IP Parcels January 2023 The following sections discuss rationale for creating and shipping IP parcels as well as the actual protocol constructs and procedures involved. IP parcels provide an essential building block for improved performance, efficiency and integrity while encouraging larger Maximum Transmission Units (MTUs) in the Internet. It is further expected that the parcel concept will drive future innovation in applications, operating systems, network equipment and data links. 2. Terminology The Oxford Languages dictionary defines a "parcel" as "a thing or collection of things wrapped in paper in order to be carried or sent by mail". Indeed, there are many examples of parcel delivery services worldwide that provide an essential transit backbone for efficient business and consumer transactions. In this same spirit, an "IP parcel" is simply a collection of up to 256 upper layer protocol segments wrapped in an efficient package for transmission and delivery (i.e., a "packet-of-packets") while a "singleton IP parcel" is simply a parcel that contains a single segment. IP parcels are distinguished from ordinary packets through the special header constructions discussed in this document. The IP parcel construct is defined for both IPv4 and IPv6. Where the document refers to "IPv4 header length", it means the total length of the base IPv4 header plus all included options, i.e., as determined by consulting the Internet Header Length (IHL) field. Where the document refers to "IPv6 header length", however, it means only the length of the base IPv6 header (i.e., 40 octets), while the length of any extension headers is referred to separately as the "IPv6 extension header length". Finally, the term "IP header plus extensions" refers generically to an IPv4 header plus all included options or an IPv6 header plus all included extension headers. Where the document refers to "{TCP, UDP} header length", it means the length of either the TCP header plus options (20 or more octets) or the UDP header (8 octets). It is important to note that only a single IP header and a single full upper layer header appears in each parcel regardless of the number of segments included. This distinction often provides a significant savings in overhead made possible only by IP parcels. Where the document refers to checksum calculations, it means the standard Internet checksum unless otherwise specified. The same as for TCP [RFC9293], UDP [RFC0768] and IPv4 [RFC0791], the standard Internet checksum is defined as (sic) "the 16-bit one's complement of the one's complement sum of all (pseudo-)headers plus data, padded with zero octets at the end (if necessary) to make a multiple of two Templin Expires 28 July 2023 [Page 4] Internet-Draft IP Parcels January 2023 octets". A notional Internet checksum algorithm can be found in [RFC1071], while practical implementations require special attention to byte ordering "endianness" to ensure interoperability between diverse architectures. The Automatic Extended Route Optimization (AERO) [I-D.templin-intarea-aero] and Overlay Multilink Network Interface (OMNI) [I-D.templin-intarea-omni] technologies provide an ideal architectural framework for transmission of IP parcels. AERO/OMNI are expected to provide an operational environment for IP parcels beginning from the earliest deployment phases and extending to accommodate continuous growth. As more and more parcel-capable links begin to emerge, e.g., in data centers, edge networks, space-domain links, etc., AERO/OMNI will provide an essential transit service for true IP parcel Internetworking. The term "parcel-capable link" refers to any data link medium (physical or virtual) capable of transiting a {TCP,UDP}/IP packet that employs the parcel-specific constructions specified in this document. The link MUST be capable of forwarding all parcels with segment lengths no larger than the minimum of the link Maximum Transmission Unit (MTU) and 65535, while first applying parcel subdivision if necessary (see: Section 7). Currently, only the OMNI link satisfies these properties, but new and existing link types are encouraged to incorporate parcel support in their designs. The term "Maximum Transmission Unit (MTU)" is widely understood in Internetworking terminology to mean the largest packet size that can traverse a single link ("link MTU") or an entire path ("path MTU") without requiring IP layer fragmentation. If the MTU value returned during parcel path qualification is larger than 65535, it determines the maximum parcel size with unrestricted segment size that a router can forward over the path/link without requiring a router to perform subdivision; otherwise, it determines both the maximum parcel and segment sizes (see: Section 8). The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119][RFC8174] when, and only when, they appear in all capitals, as shown here. Templin Expires 28 July 2023 [Page 5] Internet-Draft IP Parcels January 2023 3. Background and Motivation Studies have shown that applications can improve their performance by sending and receiving larger packets due to reduced numbers of system calls and interrupts as well as larger atomic data copies between kernel and user space. Larger packets also result in reduced numbers of network device interrupts and better network utilization (e.g., due to header overhead reduction) in comparison with smaller packets. A first study [QUIC] involved performance enhancement of the QUIC protocol [RFC9000] using the linux Generic Segment/Receive Offload (GSO/GRO) facility. GSO/GRO provides a robust (but non-standard) service similar in nature to the IP parcel service described here, and its application has shown significant performance increases due to the increased transfer unit size between the operating system kernel and QUIC applications. Unlike IP parcels, however, GSO/GRO perform fragmentation and reassembly at the transport layer with the transport protocol segment size limited by the path MTU (typically 1500 octets or smaller in today's Internet). A second study [I-D.templin-dtn-ltpfrag] showed that GSO/GRO also improves performance for the Licklider Transmission Protocol (LTP) [RFC5326] used for the Delay Tolerant Networking (DTN) Bundle Protocol [RFC9171] for segments larger than the actual path MTU through the use of OMNI interface encapsulation and fragmentation. Historically, the NFS protocol also saw significant performance increases using larger (single-segment) UDP datagrams even when IP fragmentation is invoked, and LTP still follows this profile today. Moreover, LTP shows this (single-segment) performance increase profile extending to the largest possible segment size which suggests that additional performance gains are possible using (multi-segment) IP parcels that approach or even exceed 65535 octets. TCP also benefits from larger packet sizes and efforts have investigated TCP performance using jumbograms internally with changes to the linux GSO/GRO facilities [BIG-TCP]. The idea is to use the Jumbo Payload option internally and to allow GSO/GRO to use buffer sizes larger than 65535 octets, but with the understanding that links that support jumbos natively are not yet widely available. Hence, IP parcels provide a packaging that can be considered in the near term under current deployment limitations. Templin Expires 28 July 2023 [Page 6] Internet-Draft IP Parcels January 2023 A limiting consideration for sending large packets is that they are often lost at links with MTU restrictions, and the resulting Packet Too Big (PTB) message [RFC1191][RFC8201] may be lost somewhere in the return path to the original source. This "Path MTU black hole" condition can degrade performance unless robust path probing techniques are used, however the best case performance always occurs when loss of packets due to size restrictions is minimized. These considerations therefore motivate a design where transport protocols should employ a maximum segment size no larger than 65535 octets (minus headers), while parcels that carry multiple segments may themselves be significantly larger. Then, even if the network needs to sub-divide the parcels into smaller sub-parcels for further forwarding toward the final destination, an important performance optimization for the original source, final destination and network path as a whole can be realized. This performance advantage is accompanied by an overall improvement in integrity and efficiency. An analogy: when a consumer orders 50 small items from a major online retailer, the retailer does not ship the order in 50 separate small boxes. Instead, the retailer packs as many of the small items as possible into one or a few larger boxes (i.e., parcels) then places the parcels on a semi-truck or airplane. The parcels may then pass through one or more regional distribution centers where they may be repackaged into different parcel configurations and forwarded further until they are finally delivered to the consumer. But most often, the consumer will only find one or a few parcels at their doorstep and not 50 separate small boxes. This flexible parcel delivery service greatly reduces shipping and handling cost for all including the retailer, regional distribution centers and finally the consumer. 4. IP Parcel Formation An upper layer protocol entity (identified by the 5-tuple as above) forms an IP parcel when it prepares a data buffer containing the concatenation of an Integrity Block of up to 256 2-octet Checksums followed by their corresponding upper layer protocol segments (with each TCP non-first segment preceded by a 4-octet Sequence Number). All non-final segments MUST be equal in length while the final segment MUST NOT be larger and MAY be smaller. Each non-final segment MUST NOT be larger than the minimum of 65535 octets and the path MTU, minus the length of the {TCP,UDP} header, minus the length of the IP header (plus options/extensions), minus 2 octets for the per-segment Checksum. (Note that this also satisfies the case of ingress middlebox OMNI interfaces in the path that would process the headers as upper layer protocol payload during IPv6 encapsulation/ fragmentation.) Templin Expires 28 July 2023 [Page 7] Internet-Draft IP Parcels January 2023 The upper layer protocol entity then presents the buffer and non- final segment size L to lower layers (noting that the buffer may be larger than 65535 octets if it includes sufficient segments of a large enough size to exceed that value). If the buffer plus headers would together be no larger than the first hop link MTU or path MTU, the lower layer then appends a single full {TCP,UDP} header (plus options) followed by a single IP header (plus options/extensions). If the buffer would cause a single parcel to exceed the link/path MTU, the lower layer instead breaks the buffer up into multiple smaller buffers (each with an integral number of segments) and appends separate {TCP,UDP}/IP headers for each as separate parcels. (Note: if the first hop link MTU is larger than the path MTU, the lower layer can either restrict the size of the parcels it sends to the path MTU or send parcels up to as large as the link MTU with the understanding that a router in the path will subdivide it into smaller sub-parcels.) The IP layer then presents each parcel to a network interface attachment to either an ordinary parcel-capable link or an OMNI link that performs adaptation layer encapsulation and fragmentation (see: Section 7). The IP layer includes a special coding of the Jumbo Payload option in the IP header formed as shown in Figure 1: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option Type | Opt Data Len |S| Code | Check | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Nsegs | Jumbo Payload Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+- Identification -+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: Jumbo Payload Option Format Templin Expires 28 July 2023 [Page 8] Internet-Draft IP Parcels January 2023 For IPv4, the Jumbo Payload option format is similar to [RFC2675] except that the IP layer sets option type to '00001011' and option length to '00010000' noting that the length distinguishes this type from its obsoleted use as the "IPv4 Probe MTU" option [RFC1063]. The IP layer then sets the "(S)ub-parcel" flag to '0', sets Code to 127, sets Check to the same value that will appear in the TTL of the outgoing IPv4 header and sets Identification as discussed in Section 7. The IP layer also interprets the most significant option data octet as an "Nsegs" field that encodes a value J between 0 and 255 and sets the Jumbo Payload Length field to a 3-octet value M that encodes the length of the IPv4 header plus the length of the {TCP,UDP} header plus the combined length of the Integrity Block plus all concatenated segments. The IP layer next sets the IPv4 header DF bit to 1 and Total Length field to the non-final segment size L. For IPv6, the IP layer includes a Jumbo Payload option in an IPv6 Hop-by-Hop Options extension header formatted the same as for IPv4 above, but with option type set to '11001110' and option length set to '00001110'. The IP layer then sets the "(S)ub-parcel" flag to '0', sets Code to 127, sets Check to the same value that will appear in the Hop Limit of the outgoing IPv6 header and sets Identification as discussed in Section 7. The IP layer then sets the option data "Nsegs" field to a 1-octet value J between 0 and 255 and sets the Jumbo Payload Length field to a 3-octet value M that encodes the lengths of all IPv6 extension headers present plus the length of the {TCP,UDP} header plus the combined length of the Integrity Block plus all concatenated segments. The IP layer next sets the IPv6 header Payload Length field to L. The IP layer then prepares the rest of the {TCP,UDP}/IP parcel according to the formats shown in Figure 2: Templin Expires 28 July 2023 [Page 9] Internet-Draft IP Parcels January 2023 TCP/IP Parcel Structure UDP/IP Parcel Structure +------------------------------+ +------------------------------+ |IP Hdr plus options/extensions| |IP Hdr plus options/extensions| ~ {Total, Payload} Length = L ~ ~ {Total, Payload} Length = L ~ | Nsegs = J; Jumbo Length = M | | Nsegs = J; Jumbo Length = M | +------------------------------+ +------------------------------+ | | | | ~ TCP header (plus options) ~ ~ UDP header ~ | (Includes Sequence Number 0) | | | +------------------------------+ +------------------------------+ | | | | ~ Integrity Block ~ ~ Integrity Block ~ | | | | +------------------------------+ +------------------------------+ ~ ~ ~ ~ ~ Segment 0 (L-4 octets) ~ ~ Segment 0 (L octets) ~ +------------------------------+ +------------------------------+ ~ Sequence Number 1 followed ~ ~ ~ ~ by Segment 1 (L octets) ~ ~ Segment 1 (L octets) ~ +------------------------------+ +------------------------------+ ~ Sequence Number 2 followed ~ ~ ~ ~ by Segment 2 (L octets) ~ ~ Segment 2 (L octets) ~ +------------------------------+ +------------------------------+ ~ ... ~ ~ ... ~ ~ ... ~ ~ ... ~ +------------------------------+ +------------------------------+ ~ Sequence Number J followed ~ ~ ~ ~ by Segment J (K octets) ~ ~ Segment J (K octets) ~ +------------------------------+ +------------------------------+ Figure 2: {TCP,UDP}/IP Parcel Structure where the total number of segments is (J + 1), L is the length of each non-final segment which MUST NOT be larger than 65535 octets (minus headers) and K is the length of the final segment which MUST NOT be larger than L. (Note that when J is 0, K and L are one and the same value.) The {TCP,UDP} header is then immediately followed by an Integrity Block containing (J + 1) 2-octet Checksums concatenated in numerical order as shown in Figure 3: Templin Expires 28 July 2023 [Page 10] Internet-Draft IP Parcels January 2023 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (0) | Checksum (1) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (2) | ... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... ~ ~ ... ... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (J-1) | Checksum (J) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: Integrity Block Format The Integrity Block is then followed by (J + 1) upper layer protocol segments. For TCP, the TCP header Sequence Number field encodes a 4-octet starting sequence number for the first segment only, while each additional segment is preceded by its own 4-octet Sequence Number field. For this reason, the length of the first segment is only (L-4) octets since the 4-octet TCP header Sequence Number field applies to that segment. (All non-first TCP segments instead begin with their own Sequence Numbers, with the 4-octet length included in L and K.) Following parcel construction, the Nsegs value unambiguously determines the number of 2-octet Checksums present in the Integrity Block and (together with the IP {Total, Payload} length and Jumbo Payload Length) also determines the number of parcel data segments present. Receiving nodes that process IP parcels therefore observe the following requirements: * if the Jumbo Payload Length indicates insufficient space for the full Integrity Block plus at least one data segment of length K, the receiver discards the parcel. * if the length of the payload following the Integrity Block is (J * L) or less, the receiver processes all initial Checksums along with their corresponding segments up to the end of the payload and ignores any remaining Checksums. * if the length of the payload following the Integrity Block is greater than ((J + 1) * L) the receiver processes all Checksums with their corresponding segments and ignores any remaining payload beyond the end of the final segment. Note: per-segment Checksums appear in a contiguous Integrity Block immediately following the {TCP,UDP}/IP headers instead of inline with the parcel segments to greatly increase the probability that they will appear in the contiguous head of a kernel receive buffer even if the parcel was subject to OMNI interface IPv6 fragmentation. This Templin Expires 28 July 2023 [Page 11] Internet-Draft IP Parcels January 2023 condition may not always hold if the IPv6 fragments also incur IPv4 encapsulation and fragmentation over paths that traverse slow IPv4 links with small MTUs. In that case, performance is bounded by the unavoidable slow link traversal and not the overhead for pulling a fragmented Integrity Block into the contiguous head of a kernel receive buffer. 5. TCP Parcels A TCP Parcel is an IP Parcel that includes an IP header plus extensions with a Jumbo Payload option formed as shown in Figure 1 with Nsegs/J encoding one less than the number of segments and Jumbo Payload length encoding a value up to 16,777,215 (2**24 - 1). The IP header plus extensions is then followed by a TCP header plus options (20 or more octets), which is then followed by an Integrity Block with (J + 1) consecutive 2-octet Checksums. The Integrity Block is then followed by (J + 1) consecutive segments, where the first segment is (L-4) octets in length and uses the 4-octet sequence number found in the TCP header, each intermediate segment is L octets in length (including its own 4-octet Sequence Number segment header) and the final segment is K octets in length (including its own 4-octet Sequence Number segment header). The value L is encoded in the IP header {Total, Payload} Length field while J is encoded in the Nsegs octet. The overall length of the parcel as well as final segment length K are determined by the Jumbo Payload length M as discussed above. The source prepares TCP Parcels in a similar fashion as for simple TCP jumbograms [RFC2675]. The source calculates a checksum of the TCP header plus IP pseudo-header only (see: Section 10), but with the TCP header Sequence Number field temporarily set to 0 during the calculation since the true sequence number will be included as a pseudo header for the first segment. The source then writes the calculated value in the TCP header Checksum field as-is (i.e., without converting calculated '0' values to 'ffff') and finally re- writes the actual sequence number back into the Sequence Number field. (Nodes that verify the header checksum first perform the same operation of temporarily setting the Sequence Number field to 0 and then resetting to the actual value following checksum verification.) Templin Expires 28 July 2023 [Page 12] Internet-Draft IP Parcels January 2023 The source then calculates the checksum of the first segment beginning with the sequence number found in the full TCP header as a 4-octet pseudo-header then extending over the remaining (L-4) octet length of the segment. The source next calculates the checksum for each L octet intermediate segment independently over the length of the segment (beginning with its sequence number), then finally calculates the checksum of the K octet final segment (beginning with its sequence number). As the source calculates each segment(i) checksum (for i = 0 thru J), it writes the value into the corresponding Integrity Block Checksum(i) field as-is. See: Section 10 for further discussion. 6. UDP Parcels A UDP Parcel is an IP Parcel that includes an IP header plus extensions with a Jumbo Payload option formed as shown in Figure 1 with Nsegs/J encoding one less than the number of segments and Jumbo Payload length encoding a value up to 16,777,215 (2**24 - 1). The IP header plus extensions is then followed by an 8-octet UDP header followed by an Integrity Block with (J + 1) consecutive 2-octet Checksums followed by (J + 1) upper layer protocol segments. Each segment must begin with a transport-specific start delimiter (e.g., a segment identifier) included by the transport layer user of UDP. The length of the first segment L is encoded in the IP {Total, Payload} Length field while J is encoded in the Nsegs octet. The overall length of the parcel as well as the final segment length are determined by the Jumbo Payload length M as discussed above. The source prepares UDP Parcels in a similar fashion as for simple UDP jumbograms [RFC2675] and therefore MUST set the UDP header length field to 0. The source then calculates the checksum of the UDP header plus IP pseudo-header (see: Section 10) and writes the calculated value in the UDP header Checksum field as-is (i.e., without converting calculated '0' values to 'ffff'). The source then calculates a separate checksum for each segment for which checksums are enabled independently over the length of the segment. As the source calculates each segment(i) checksum (for i = 0 thru J), it writes the value into the corresponding Integrity Block Checksum(i) field with calculated '0' values converted to 'ffff'; for segments with checksums disabled, the source instead writes the value '0'. See: Section 10 for further discussion. Templin Expires 28 July 2023 [Page 13] Internet-Draft IP Parcels January 2023 7. Transmission of IP Parcels The IP layer of the source maintains a randomly-initialized 64 bit cached value to assign as parcel Identifications. For each parcel transmission, the IP layer sets the Identification field in the Jumbo Payload to the current cached value then increments the cached value by 1 (modulo 2**64). The IP layer can subsequently reset the cached value to a new random value at any time, e.g., to maintain an unpredictable profile (noting that resetting too frequently may limit opportunistic reassembly at destinations). The IP layer of the source next presents each parcel to a network interface for transmission. For ordinary IP interface attachments to parcel-capable links, the interface simply admits each parcel into the link the same as for any IP packet after which it may then be forwarded by any number of routers over additional consecutive parcel-capable links possibly even traversing the entire forward path to the final destination. If any router in the path does not recognize the parcel construct, it may drop the parcel and return an ICMP "Parameter Problem" message. Each parcel serves as an implicit probe to test the forward path's ability to pass parcels. When a router that observes this specification receives an IP Parcel it first compares Code with 127 and Check with the IP header TTL/Hop Limit; if either value differs, the router drops the parcel and return a negative Parcel Reply (see Section 8). Otherwise, the router compares the value L with the next hop link MTU. If the next hop link MTU is too small to pass either a singleton parcel or an individual IP packet with segment of length L the router discards the parcel and returns a positive Parcel Reply with MTU set to the next hop link MTU. Otherwise, if the next hop link is parcel capable the router MUST reset Check to the same value that would appear in the TTL/Hop Limit of the outgoing IP header for forwarding the parcel to the next hop. If the router recognizes parcels but the next hop link in the path does not, or if the entire parcel would exceed the next hop link MTU, the router instead opens the parcel. The router then forwards each enclosed segment in singleton IP packets or in a set of smaller sub- parcels that each contain a subset of the original parcel's segments. If the next hop link is via an OMNI interface, the router instead proceeds according to OMNI Adaptation Layer procedures. These considerations are discussed in detail in the following sections. Templin Expires 28 July 2023 [Page 14] Internet-Draft IP Parcels January 2023 7.1. Singleton IP Packets over Non-Parcel Links For transmission of singleton IP packets over links that do not support parcels, the router first determines whether a singleton parcel with segment of length L can fit within the next-hop link MTU. If not, the router returns a positive Parcel Reply message with MTU set to the next-hop link MTU and containing the leading portion of the parcel beginning with the IP header, then drops the parcel. Otherwise, the router removes the Jumbo Payload option, sets aside and remembers the Integrity Block (and for TCP also truncates the Sequence Number headers of each non-first segment while remembering their values) then copies the {TCP,UDP}/IP headers (but with the Jumbo Payload option removed) followed by segment(i) (for i= 0 thru J) into individual singleton(i) IP packets. The router then sets IP {Total, Payload} length for each singleton(i) based on the length of segment(i) according to the standards [RFC0791] [RFC8200]. The router then processes each singleton(i) according to upper layer protocol conventions. For TCP, the router clears the SYN/ACK flags in all except singleton(0) then calculates the checksum for singleton(0)'s TCP/IP headers only according to [RFC9293] but with the Sequence Number value saved and the field set to 0. The router then adds Integrity Block Checksum(0) to the calculated value and writes the sum into singleton(0)'s TCP checksum field. The router then resets the Sequence Number field to singleton(0)'s saved sequence number and forwards singleton(0) to the next hop. The router next calculates the checksum of singleton(1)'s TCP/IP headers with the Sequence Number field set to 0 and saves the calculated value. In each non- first singleton(i) (for i = 1 thru J), the router then adds the saved value to Integrity Block Checksum(i), writes the sum into singleton(i)'s TCP checksum field, sets the TCP Sequence Number field to singleton(i)'s sequence number then forwards singleton(i) to the next hop. Templin Expires 28 July 2023 [Page 15] Internet-Draft IP Parcels January 2023 For UDP, the router sets the UDP length field according to [RFC0768] in each singleton(i) (for i= 0 thru J). If Integrity Block Checksum(i) is 0, the router then sets the UDP header checksum to 0, forwards singleton(i) to the next hop and continues to the next. The router next calculates the checksum over singleton(i)'s UDP/IP headers only according to [RFC0768]. If Integrity Block Checksum(i) is not 'ffff', the router then adds the value to the header checksum; otherwise, the router re-calculates the checksum for segment(i). If the re-calculated segment(i) checksum value is 'ffff' or '0' the router adds the value to the header checksum; otherwise, it continues to the next singleton(i) (see note). The router finally writes the total checksum value into the UDP checksum field for singleton(i) (or writes 'ffff' if the total was '0') and forwards singleton(i) to the next hop. Note: for each UDP singleton(i), the router must recalculate the segment checksum if Checksum(i) is 'ffff', since that value is shared by both '0' and 'ffff' calculated checksums. If recalculating the checksum produces an incorrect value, segment(i) is considered errored and the router can optionally drop or forward (noting that the forwarded singleton would simply be discarded as an error by the final destination). Note: for each {TCP,UDP} singleton(i), the router can optionally re- calculate and verify the segment checksum unconditionally before forwarding, but this may introduce undesirable extra delay and processing overhead. 7.2. Sub-Parcels over Parcel-capable Links For transmission of smaller sub-parcels over parcel-capable links, the router first determines whether a single segment of length L can fit within the next-hop link MTU if packaged as a (singleton) sub- parcel. If not, the router returns a positive Parcel Reply message with MTU set to the next-hop link MTU and containing the leading portion of the parcel beginning with the IP header, then drops the parcel. Otherwise, the router breaks the original parcel into smaller groups of segments that would fit within the path MTU by determining the number of segments of length L that can fit into each sub-parcel under the size constraints. For example, if the router determines that a sub-parcel can contain 3 segments of length L, it creates sub-parcels with the first containing Integrity Block Checksums/Segments 0-2, the second containing Checksums/Segments 3-5, etc., and with the final containing any remaining Checksums/Segments. When the router breaks an original parcel into sub-parcels, it first checks the "(S)ub-parcel" flag in the Jumbo Header. If the S flag is '0', the router sets S to '1' in all resulting sub-parcels except the Templin Expires 28 July 2023 [Page 16] Internet-Draft IP Parcels January 2023 final one (i.e., the one containing the final segment of length K, which may be shorter than L) for which it sets S to '0'. If the S flag is '1', the router instead sets S to '1' in all resulting sub- parcels including the final one. The router then appends identical {TCP,UDP}/IP headers (including the Jumbo Payload option and any other extensions) to each sub-parcel while resetting L and M in each according to the above equations with Nsegs/J set to 2 for each intermediate sub-parcel and with Nsegs/J set to one less than the remaining number of segments for the final sub-parcel. For TCP, the router then sets the TCP Sequence Number field to the value that appears in the first sub-parcel segment while removing the first segment Sequence Number field (if present) and also clears the SYN/ACK flags in all sub-parcels except the first. For both TCP and UDP, the router finally resets the {TCP,UDP} header checksum according to ordinary parcel formation procedures (see above) then forwards each (sub-)parcel over the outgoing parcel- capable link. Note: sub-dividing a larger parcel into two or more sub-parcels entails replication of the {TCP,UDP}/IP headers (including the Jumbo Payload option and any other extensions). For TCP, the process entails copying the full TCP/IP header from the original parcel while writing the sequence number of the first sub-parcel segment into the TCP Sequence Number field, clearing the SYN/ACK flags if necessary as discussed above and truncating the (new) first segment Sequence Number field. For UDP, the process entails copying the full UDP/IP header from the original parcel into each sub-parcel. For both TCP and UDP, the process finally includes recalculating and resetting Nsegs and Jumbo Payload Length then recalculating the {TCP,UDP} header checksum. Note that the per-segment Integrity Block Checksum values in the sub-parcel segments themselves are still valid and need not be recalculated. 7.3. Parcels/Sub-Parcels over OMNI Interfaces For transmission of original parcels or sub-parcels over OMNI interfaces, all parcels are admitted into the OMNI interface unconditionally since the OMNI interface MTU is unrestricted. The OMNI Adaptation Layer (OAL) of this First Hop Segment (FHS) OAL source node then forwards the parcel to the next OAL hop which may be either an OAL intermediate node or a Last Hop Segment (LHS) OAL destination. OMNI interface upper layer protocol processing procedures are specified in detail in the remainder of this section, while lower layer encapsulation and fragmentation procedures are specified in detail in [I-D.templin-intarea-omni]. Templin Expires 28 July 2023 [Page 17] Internet-Draft IP Parcels January 2023 When the OAL source forwards a parcel or sub-parcel (whether generated by a local application or forwarded by other nodes over one or more parcel-capable links), it first assigns a monotonically- incrementing (modulo 255) "Parcel ID" for adaptation layer processing. If the parcel is larger than the OAL maximum segment size of 65535 octets, the OAL source then subdivides the parcel into sub-parcels the same as for the IP layer procedures discussed above. The OAL source next assigns a different monotonically-incrementing adaptation layer Identification value for each sub-parcel of the same "Parcel ID" then performs adaptation layer encapsulation and fragmentation and finally forwards each fragment to the next OAL hop which forwards them further toward the OAL destination as necessary. (During encapsulation, the OAL source examines the Jumbo Payload option S flag to determine the setting for the adaptation layer fragment header S flag according to the same rules specified in Section 7.2.) When the sub-parcels arrive at the OAL destination, the node can optionally retain them along with their Parcel ID and Identifications for a brief time to support re-combining with peer sub-parcels of the same original parcel identified by the adaptation layer 4-tuple (source, destination, Identification, Parcel ID). This re-combining entails the concatenation of Checksums/Segments included in sub- parcels with the same Parcel ID and with Identification values within 255 of one another to create a larger sub-parcel possibly even as large as the entire original parcel. Order of concatenation need not be strictly enforced, with the exception that the sub-parcel with S flag set to '0' must occur as a final concatenation and not as an intermediate. The recombined (sub)parcel then sets the S flag to '0' if and only if one of its recombined elements also had the S flag set to '0'; otherwise, it sets the S flag to '1'. The OAL destination then appends a common {TCP,UDP}/IP header plus extensions to each re-combined sub-parcel while resetting J, K, L and M in each according to the above equations. For TCP, if any sub- parcels have the SYN/ACK flags set the OAL destination also sets the SYN/ACK flags in the re-combined sub-parcel TCP header. The OAL destination then resets the {TCP,UDP}/IP header checksum for each re- combined sub-parcel. If the OAL destination is also the final destination, it then delivers the sub-parcels to the IP layer which processes them according to the 5-tuple information supplied by the original source. Otherwise, the OAL destination forwards each sub- parcel toward the final destination the same as for an ordinary IP packet as discussed above. Note: sub-dividing of IP parcels over OMNI links occurs only at an OAL ingress node while re-combining of IP parcels occurs only at an OAL egress node. Therefore, intermediate OAL nodes do not Templin Expires 28 July 2023 [Page 18] Internet-Draft IP Parcels January 2023 participate in the sub-dividing or recombining processes. For TCP, the SYN/ACK flags must be managed as specified above to avoid confusing receivers with gratuitous duplicate ACKs. Note: re-combining two or more sub-parcels into a larger parcel entails a process in which the {TCP,UDP}/IP headers of non-first sub- parcels are discarded and their included segments concatenated following those of a first sub-parcel. For TCP, the process includes setting the SYN/ACK flags in the TCP header only if SYN/ACK were set in any of the original sub-parcels. For both TCP and UDP, the process finally includes recalculating and resetting Nsegs and Jumbo Payload Length then recalculating the {TCP,UDP} header checksum as discussed above (the per-segment Integrity Block Checksums need not be recalculated). The OAL destination can instead avoid this process if it would negatively impact performance, noting that forwarding individual sub-parcels without delay and without re-combining is always acceptable. Note: sub-dividing and re-combining of IP parcels over OMNI links occurs as an adaptation layer and is based on the adaptation layer 4-tuple and not the network layer 5-tuple. The OAL must adhere to this discipline even if 5-tuple information is available in the clear, since some sub-parcels of the same original parcel may be forwarded over different network paths. 7.4. Final Destination Reassembly When a large parcel transits a path that includes links with restrictive MTUs, the final destination may receive multiple sub- parcels having the same 5-tuple and Identification value. The final destination should hold the sub-parcels in a reassembly buffer for a short time or until a sub-parcel with the S flag set to '0' arrives. The final destination then concatenates the segments of all non-final sub-parcels and finally concatenates the segments of the final sub- parcel then passes the reassembled parcel to upper layers. Due to the possibility of network loss and/or reordering, it will often be the case that a sub-parcel with S set to '0' arrives before all other sub-parcels of the same original parcel have arrived. This condition does not constitute an error, but may in some cases result in delivery of sub-parcels to upper layers that are smaller than the original parcel. Upper Layers will then process any segments received even if there may be some segment reordering, and will request retransmission of any segments that were lost and/or damaged. Note: if the final destination's reassembly buffer holds sub-parcels of "adjacent" parcels (i.e., those with identical 5-tuples, L values, and with Identification values in close proximity) the destination Templin Expires 28 July 2023 [Page 19] Internet-Draft IP Parcels January 2023 can optionally recombine sub-parcels of adjacent parcels to deliver to upper layers. If so, however, the destination must avoid recombining sub-parcels containing final segments of multiple original parcels. 8. Parcel Path Qualification To determine whether parcels are supported over at least an initial portion of the forward path toward the final destination, the original source can send IP parcels that contain Jumbo Payload options formatted as either ordinary parcels or as special-purpose "Parcel Probes". The probe will elicit a "Parcel Reply" and possibly also an upper layer protocol-specific probe reply from the final destination. If the original source receives a positive Parcel Reply, it marks the path as "parcels supported" and ignores any ordinary ICMP [RFC0792][RFC4443] and/or Packet Too Big (PTB) messages [RFC1191][RFC8201] concerning the probe. If the original source instead receives a negative Parcel Reply or no reply, it marks the path as "parcels not supported" and may regard any ordinary ICMP and/ or PTB messages concerning the probe (or its contents) as indications of a possible path limitation. The original source can therefore send Parcel Probes in parallel with sending real data as ordinary IP packets/parcels. The probes will traverse parcel-capable links joined by routers on the forward path possibly extending all the way to the destination. If the original source receives a positive Parcel Reply, it can continue using IP parcels. Parcel Probes include the same Jumbo Payload option type used for ordinary parcels (see: Section 4) but set a different option length and include a trailing 4-octet "Path MTU" field into which conformant routers write the minimum link MTU observed in a similar fashion as described in [RFC1063] and [I-D.ietf-6man-mtu-option]. Parcel Probes include one or more upper layer protocol segments corresponding to the 5-tuple for the flow, which may also include {TCP,UDP} segment size probes used for packetization layer path MTU discovery [RFC4821] [RFC8899]. The original source sends Parcel Probes unidirectionally in the forward path toward the final destination to elicit a Parcel Reply, since it will often be the case that IP parcels are supported only in the forward path and not in the return path. Parcel Probes may be dropped in the forward path by any node that does not recognize IP parcels, but Parcel Replys must be packaged to avoid filtering since parcels may not be recognized along portions of the return path. For Templin Expires 28 July 2023 [Page 20] Internet-Draft IP Parcels January 2023 this reason, the Jumbo Payload options included in Parcel Probes are always packaged as IPv4 header options or IPv6 Hop-by-Hop options while Parcel Replys are returned as UDP/IP encapsulated ICMPv6 PTB messages with a "Parcel Reply" Code value (see: [I-D.templin-intarea-omni]). Original sources send Parcel Probes that include a Jumbo Payload option coded in an alternate format as shown in Figure 4: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option Type | Opt Data Len |S| Code | Check | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Nsegs | Jumbo Payload Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+- Identification -+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Path MTU (PMTU) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: Parcel Probe Jumbo Payload Option Format For IPv4, the original source includes the option as an IPv4 header option with Type set to '00001011' the same as for an ordinary IPv4 parcel (see: Section 4) but with Length set to '00010100' to distinguish this as a probe. The original source sets S to '0', sets Code to 127, sets Check to the same value that will appear in the TTL of the outgoing IPv4 header, sets Identification as specified in Section 7 and sets PMTU to the MTU of the outgoing IPv4 interface. The source next includes a {TCP,UDP} header followed by an Integrity Block with Checksums followed by their upper layer protocol Segments in the same format as for an ordinary parcel. (The source can also form a NULL probe by setting Protocol to "No Next Header (59)" and including an Integrity Block with Checksum fields set to '0' followed by NULL segments with zero, random and/or other disposable payloads.) The source then sets {Nsegs, Jumbo Payload Length, IPv4 Total Length} and calculates the header and per-segment checksums the same as for an ordinary parcel. The source finally sends the Parcel Probe via the outbound IPv4 interface. According to [RFC7126], middleboxes (i.e., routers, security gateways, firewalls, etc.) that do not observe this specification SHOULD drop IP packets that contain option type '00001011' ("IPv4 Probe MTU") but some might instead either attempt to implement [RFC1063] or ignore the option altogether. IPv4 middleboxes that observe this specification instead MUST process the option as a Parcel Probe as specified below. Templin Expires 28 July 2023 [Page 21] Internet-Draft IP Parcels January 2023 For IPv6, the original source includes the probe option as an IPv6 Hop-by-Hop option with Type set to '11000010' the same as for an ordinary IPv6 parcel (see: Section 4) but with Length set to '00010010' to distinguish this as a probe. The original source sets S to '0', sets Code to 127, sets Check to the same value that will appear in the Hop Limit of the outgoing IPv6 header, sets Identification as specified in Section 7 and sets PMTU to the MTU of the outgoing IPv6 interface. The source next includes a {TCP,UDP} header followed by upper layer protocol Segments along with their Integrity Block Checksums in the same format as for an ordinary parcel. (The source can also form a NULL probe by setting Next Header to "No Next Header (59)" and including an Integrity Block with Checksum fields set to '0' followed by NULL segments with zero, random and/or other disposable payloads.) The source then sets {Nsegs, Jumbo Payload Length, IPv6 Payload Length} and calculates the header and per-segment checksums the same as for an ordinary parcel. The source finally sends the Parcel Probe via the outbound IPv6 interface. According to [RFC2675], middleboxes (i.e., routers, security gateways, firewalls, etc.) that recognize the IPv6 Jumbo Payload option but do not observe this specification SHOULD return an ICMPv6 Parameter Problem message (and presumably also drop the packet) due to the different option length. IPv6 middleboxes that observe this specification instead MUST process the option as a Parcel Probe as specified below. When a router that observes this specification receives an IP Parcel Probe it first compares Code with 127 and Check with the IP header TTL/Hop Limit; if either value differs, the router MUST drop the probe and return a negative Parcel Reply (see below). Otherwise, if the next hop link is non-parcel-capable the router compares the PMTU value with the MTU of the inbound link for the probe and MUST (re)set PMTU to the lower MTU. The router then MUST return a positive Parcel Reply (see below) and convert the probe into singleton IP packet(s) the same as was described previously for routers forwarding to non- parcel-capable links. If the next hop IP link configures a sufficiently large MTU to pass the packet(s), the router converts the probe and MUST forward each singleton packet to the next hop; otherwise, it drops the probe. If the next hop IP link both supports parcels and configures an MTU that is large enough to pass the probe, the router instead compares the probe PMTU value with the MTUs of both the inbound and outbound links for the probe and MUST (re)set PMTU to the lower MTU. The router then MUST reset Check to the same value that will appear in the TTL/Hop Limit of the outgoing IP header, and MUST forward the Parcel Probe to the next hop. If the next hop IP link supports parcels but configures an MTU that is too small to pass the probe, it resets PMTU and Check the same as above then subdivides the probe into multiple smaller probes small enough to traverse the link. Templin Expires 28 July 2023 [Page 22] Internet-Draft IP Parcels January 2023 The final destination may therefore receive either one or more ordinary IP packets or intact Parcel Probes. If the final destination receives ordinary IP packets, it performs any necessary integrity checks then delivers the packets to upper layers which will return an upper layer probe response if necessary. If the final destination receives a Parcel Probe, it first compares Code with 127 and Check with the IP header TTL/Hop Limit; if either value differs, the final destination MUST drop the probe and return a negative Parcel Reply. Otherwise, the final destination compares the probe PMTU value with the MTU of the inbound link and MUST (re)set PMTU to the lower MTU. The final destination then MUST return a positive Parcel Reply and deliver the probe contents to upper layers the same as for an ordinary IP parcel. When a router or final destination returns a Parcel Reply, it prepares an ICMPv6 PTB message [RFC4443] with Code set to "Parcel Reply" (see: [I-D.templin-intarea-omni]) and with MTU set to either the PMTU value reported in the Parcel Probe for a positive reply or to the value '0' for a negative reply. The node then writes its own IP address as the Parcel Reply source and writes the source of the Parcel Probe as the Parcel Reply destination (for IPv4 Parcel Probes, the node writes the Parcel Reply addresses as IPv4-Compatible IPv6 addresses [RFC4291]). The node next copies as much of the leading portion of the Parcel Probe (beginning with the IP header) as possible into the "packet in error" field without causing the Parcel Reply to exceed 512 octets in length, then calculates the ICMPv6 header checksum. Since IPv6 packets cannot traverse IPv4 paths, and since middleboxes often filter ICMPv6 messages as they traverse IPv6 paths, the node next wraps the Parcel Reply in UDP/IP headers of the correct IP version with the IP source and destination addresses copied from the Parcel Reply and with UDP port numbers set to the UDP port number for OMNI [I-D.templin-intarea-omni]. In the process, the node either calculates or omits the UDP checksum as appropriate and (for IPv4) clears the DF bit. The node finally sends the prepared Parcel Reply to the original source of the probe. After sending a Parcel Probe the original source may therefore receive a UDP/IP encapsulated Parcel Reply (see above) and/or an upper layer protocol probe reply. If the source receives a Parcel Reply, it first verifies the checksum then matches the enclosed PTB message with the original Parcel Probe by examining the Identification field echoed in the ICMPv6 "packet in error" field containing the leading portion of the probe. If PTB does not match, the source discards the Parcel Reply; otherwise, it continues to process. If the Parcel Reply MTU is '0', the source marks the path as "parcels not supported"; otherwise, it marks the path as "parcels supported" and also records the MTU value as the MTU for the parcel path (i.e., the portion of the path up to and including the node that Templin Expires 28 July 2023 [Page 23] Internet-Draft IP Parcels January 2023 returned the Parcel Reply). If the MTU value is 65535 or larger, the MTU determines the largest whole parcel size that can traverse the parcel path without subdivision while using any segment size up to and including the maximum. If the MTU value is smaller than 65535, the MTU represents both the largest whole parcel size and a maximum segment size limitation. In both cases, the maximum segment size that can traverse the parcel path may be larger than the maximum segment size that can continue to traverse the remaining path to the final destination, which can only be determined through upper layer protocol probes (i.e., either as individual probe packets or as payloads of the Parcel Probes). Note: The original source includes Code and Check fields as the first 2 octets of both ordinary parcels and Parcel Probes in case a router on the path overwrites the values in a wayward attempt to implement [RFC1063]. Parcel Probe recipients should therefore regard a Code value other than 127 as an indication that the field was either intentionally or accidentally altered by a previous hop node. Note: If a router or final destination receives a Parcel Probe but does not recognize the parcel construct, it drops the probe without further processing (and may return an ICMP error). The original source will then consider the probe as lost, but may attempt to probe again later. Note: On links that include a forward error correction capability, in-transit damage to the Parcel Probe headers may be corrected as a lower-layer function of the receiver before the headers are examined by the network layer. 9. IP Jumbograms True IPv6 jumbograms are distinguished from IPv6 parcels by including a zero IPv6 Payload Length and an IPv6 Hop-by-Hop Option with type '11001110' and length '00000100'. The jumbo payload option format and all aspects of IPv6 jumbogram processing are exactly as specified in [RFC2675]. True IPv4 jumbograms are distinguished from IPv4 parcels by including a zero IPv4 Total Length and an IPv4 option with type '00001011' and length '00000110'. The jumbo payload option format and all aspects of IPv4 jumbogram processing are exactly the same as for IPv6 jumbograms. This specification augments IP jumbograms by also providing a Jumbo Path Qualification function using the mechanisms specified in Section 8. The function employs a "Jumbo Probe" formed exactly the same as for Parcel Probes (see: Figure 4), but with Nsegs/Jumbo Templin Expires 28 July 2023 [Page 24] Internet-Draft IP Parcels January 2023 Payload Length as well as IP {Total, Payload} length all set to '0', with {Protocol, Next Header} set to "No Next Header (59)" and with no octets included beyond the IP header. The purpose of the Jumbo Probe is to determine whether the entire path from the source to the destination is jumbo-capable (i.e., one in which all links recognize jumbograms and configure an MTU larger than 65535 octets) as well as to determine the jumbo path MTU. The source sets the Jumbo Probe PMTU to the MTU of the (jumbo- capable) outgoing link, sets Code to 127, sets Check to the next hop TTL/Hop Limit, then sends the probe via the link toward the final destination. At each forwarding hop, the router examines Code and Check and returns a negative "Jumbo Reply" (i.e., prepared the same as a Parcel Reply) if either value is incorrect. Otherwise, if the next hop link MTU is jumbo-capable the router sets PMTU, sets Check to the next hop TTL/Hop Limit and silently forwards the probe to the next hop. If the next hop link is not jumbo-capable, the router instead drops the probe and returns a negative Jumbo Reply. If the Jumbo Probe encounters an OMNI link, the OAL source can either drop the probe and return a negative Jumbo Reply or forward the probe further toward the OAL destination using adaptation layer encapsulation. In a first option, if the OAL source has a table of known PMTUs for selected OAL destinations it can encapsulate and forward the Jumbo Probe based on the known PMTU value. In a second option, the OAL source can encapsulate the Jumbo Probe in the adaptation layer IPv6 header with a jumbo payload option and with (PMTU - headers) NULL padding octets added beyond the end of the encapsulated Jumbo Probe to form an actual adaptation layer probe. The OAL source then forwards the probe via the path toward the OAL destination, where it may be lost due to a link restriction. If the probe somehow traverses the path, the OAL destination then removes the adaptation layer encapsulation, discards the trailing padding, resets PMTU and Check and forwards the original Jumbo Probe further toward the final destination. If the Jumbo Probe reaches the final destination, the final destination returns a positive Jumbo Reply with the PMTU set to the maximum-sized jumbogram that can transit the path. (Note that the jumbo probing process is conducted independently of any parcel probing, and that the two processes could very possibly yield very different results.) Templin Expires 28 July 2023 [Page 25] Internet-Draft IP Parcels January 2023 10. Integrity The {TCP,UDP}/IP header plus each segment of a (multi-segment) IP parcel includes its own integrity check. This means that IP parcels can support stronger and more discrete integrity checks for the same amount of upper layer protocol data compared to an ordinary IP packet or Jumbogram. The {TCP/UDP} header integrity checks can be verified at each hop to ensure that parcels with errored headers are detected. The per-segment Integrity Block Checksums are set by the source and verified by the final destination, noting that TCP parcels must honor the sequence number discipline discussed in Section 5. IP parcels can range in length from as small as only the {TCP,UDP}/IP headers plus a single Integrity Block Checksum with a non-zero length segment to as large as the headers plus (256 * (65535 minus headers)) octets. Although 32-bit link layer integrity checks provide sufficient protection for contiguous data blocks up to approximately 9KB, reliance on link-layer integrity checks may be inadvisable for links with significantly larger MTUs and may not be possible at all for links such as tunnels over IPv4 that invoke fragmentation. Moreover, the segment contents of a received parcel may arrive in an incomplete and/or rearranged order with respect to their original packaging. Lower layer protocol entities calculate and verify {TCP,UDP}/IP parcel header Checksums at their layer, since an errored header could result in mis-delivery to the wrong upper layer protocol entity. If a lower layer protocol entity on the path detects an incorrect {TCP,UDP}/IP Checksum it discards the entire IP parcel unless the header(s) can somehow be repaired. To support the parcel header checksum calculation, lower layer protocol entities use modified versions of the {TCP,UDP}/IPv4 "pseudo-header" found in [RFC0768][RFC9293], or the {TCP,UDP}/IPv6 "pseudo-header" found in Section 8.1 of [RFC8200]. Note that while the contents of the two IP protocol version-specific pseudo-headers beyond the address fields are the same, the order in which the contents are arranged differs and must be honored according to the specific IP protocol version as shown in Figure 5. This allows for maximum reuse of widely deployed code while ensuring interoperability. Templin Expires 28 July 2023 [Page 26] Internet-Draft IP Parcels January 2023 IPv4 Parcel Pseudo-Header +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | zero | Next Header | Segment Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Nsegs | Upper-Layer Packet Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ IPv6 Parcel Pseudo-Header +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ IPv6 Source Address ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ IPv6 Destination Address ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Nsegs | Upper-Layer Packet Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment Length | zero | Next Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5: {TCP,UDP}/IP Parcel Pseudo-Header Formats where the following fields appear in both pseudo-headers but with different ordering: * Source Address is the 4-octet IPv4 or 16-octet IPv6 source address of the prepared parcel. * Destination Address is the 4-octet IPv4 or 16-octet IPv6 destination address of the prepared parcel. * zero encodes the constant value '0'. * Next Header is the IP protocol number corresponding to the upper layer protocol, i.e., TCP or UDP. * Segment Length is the value that appears in the IPv4 Total Length or IPv6 Payload Length field of the prepared parcel. Templin Expires 28 July 2023 [Page 27] Internet-Draft IP Parcels January 2023 * Nsegs is a 1-octet value one less than the number of segments included, and must contain a number between 0 and 255 (this is the same value that appears in the Jumbo Payload Option Nsegs field). * Upper-Layer Packet Length is the 3-octet length of the {TCP,UDP} header plus data (this value can be derived from the Jumbo Payload Length by subtracting the IPv4 header length for IPv4 or IPv6 extension header length for IPv6). Upper layer protocol entities use socket options to coordinate per- segment checksum processing with lower layers. If the upper layer sets a SO_NO_CHECK(TX) socket option, the upper layer is responsible for supplying per-segment checksums on transmission and the lower layer forwards the IP parcel to the next hop without further processing; otherwise, the lower layer supplies the per-segment checksums before forwarding. If the upper layer sets a SO_NO_CHECK(RX) socket option, the upper layer is responsible for verifying per-segment checksums on reception and the lower layer delivers each received parcel body to the upper layer without further processing; otherwise, the lower layer verifies the per-segment parcel checksums before delivering. When the upper layer protocol entity of the source sends a parcel body to lower layers, it prepends an Integrity Block of (J + 1) 2-octet Checksum fields and includes a 4-octet Sequence Number field with each TCP non-first segment. If the SO_NO_CHECK(TX) socket option is set, the upper layer protocol either calculates each segment checksum and writes the value into the corresponding Checksum field (and for UDP with '0' values written as 'ffff') or writes the value '0' to disable checksums for specific segments (for UDP only). If the SO_NO_CHECK(TX) socket options is clear, for UDP the upper layer instead writes the value '0' to disable or any non-zero value to enable checksums for specific segments (for TCP, the upper layer instead writes any value). When the lower layer protocol entity of the source receives the parcel body from upper layers, if the SO_NO_CHECK(TX) socket option is set the lower layer appends the {TCP,UDP}/IP headers and forwards the parcel to the next hop without further processing. If the SO_NO_CHECK(TX) socket option is clear, the lower layer instead calculates the checksum for each TCP segment (or each UDP segment with a non-zero value in the corresponding Integrity Block Checksum field) and overwrites the calculated value into the Checksum field (and for UDP with '0' values written as 'ffff'). When the lower layer protocol entity of the destination receives a parcel from the source, if the SO_NO_CHECK(RX) socket option is set the lower layer delivers the parcel body to the upper layer without Templin Expires 28 July 2023 [Page 28] Internet-Draft IP Parcels January 2023 further processing, and the upper layer is responsible for per- segment checksum verification. If the SO_NO_CHECK(RX) socket option is clear, the lower layer instead verifies the checksum for each TCP segment (or each UDP segment with a non-zero value in the corresponding Integrity Block Checksum field) and marks a corresponding field for the segment in an ancillary data structure as either "correct" or "incorrect". (For UDP, if the Checksum is '0' the lower layer protocol unconditionally marks the segment as "correct".) The lower layer then delivers both the parcel body (beginning with the Integrity block) and ancillary data to the upper layer which can then determine which segments have correct/incorrect checksums. Note: The Integrity Block itself is intentionally omitted from the IP Parcel {TCP,UDP} header checksum calculation. This permits destinations to accept as many intact segments as possible from received parcels with checksum block bit errors, whereas the entire parcel would need to be discarded if the header checksum also covered the Integrity Block. Note: IP parcels that set {Protocol, Next Header} to "No Next Header (59)" do not include a {TCP,UDP} Checksum field and therefore do not include a header checksum. Intermediate nodes simply forward these NULL parcels without verifying a header checksum, while destination nodes simply discard them after returning a Parcel Reply, if necessary. 11. Implementation Status Common widely-deployed implementations include services such as TCP Segmentation Offload (TSO) and Generic Segmentation/Receive Offload (GSO/GRO). These services support a robust (but non-standard) service that has been shown to improve performance in many instances. UDP/IPv4 parcels have been implemented in the linux-5.10.67 kernel and ION-DTN ion-open-source-4.1.0 source distributions. Patch distribution found at: "https://github.com/fltemplin/ip-parcels.git". Performance analysis with a single-threaded receiver has shown that including increasing numbers of segments in a single parcel produces measurable performance gains over fewer numbers of segments due to more efficient packaging and reduced system calls/interrupts. For example, sending parcels with 30 2000-octet segments shows a 48% performance increase in comparison with ordinary IP packets with a single 2000-octet segment. Templin Expires 28 July 2023 [Page 29] Internet-Draft IP Parcels January 2023 Since performance is strongly bounded by single-segment receiver processing time (with larger segments producing dramatic performance increases), it is expected that parcels with increasing numbers of segments will provide a performance multiplier on multi-threaded receivers in parallel processing environments. 12. IANA Considerations The IANA is instructed to change the "MTUP - MTU Probe" entry in the 'ip option numbers' registry to the "JUMBO - IPv4 Jumbo Payload" option. The Copy and Class fields must both be set to 0, and the Number and Value fields must both be set to '11'. The reference must be changed to this document [RFCXXXX]. 13. Security Considerations In the control plane, original sources match the Identification values in received Parcel Replys with their corresponding Parcels or Parcel Probes. If the values match, the reply is likely authentic. In environments where stronger authentication is necessary, nodes that send Parcel Replys can apply the message authentication services specified for AERO/OMNI. In the data plane, multi-layer security solutions may be needed to ensure confidentiality, integrity and availability. Since parcels are defined only for TCP and UDP, IP layer securing services such as IPsec-AH/ESP [RFC4301] cannot be applied directly to parcels, although they can certainly be used at lower layers such as for transmission of parcels over VPNs and/or OMNI link secured spanning trees. Since the IP layer does not manipulate segments exchanged with upper layers, parcels do not interfere with transport- or higher-layer security services such as (D)TLS/SSL [RFC8446] which may provide greater flexibility in some environments. Further security considerations related to IP parcels are found in the AERO/OMNI specifications. 14. Acknowledgements This work was inspired by ongoing AERO/OMNI/DTN investigations. The concepts were further motivated through discussions on the IETF intarea and 6man lists as well as with Boeing colleagues. A considerable body of work over recent years has produced useful "segmentation offload" facilities available in widely-deployed implementations. 15. References Templin Expires 28 July 2023 [Page 30] Internet-Draft IP Parcels January 2023 15.1. Normative References [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, DOI 10.17487/RFC0768, August 1980, . [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, DOI 10.17487/RFC0791, September 1981, . [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, RFC 792, DOI 10.17487/RFC0792, September 1981, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms", RFC 2675, DOI 10.17487/RFC2675, August 1999, . [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, DOI 10.17487/RFC4291, February 2006, . [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification", STD 89, RFC 4443, DOI 10.17487/RFC4443, March 2006, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017, . [RFC9293] Eddy, W., Ed., "Transmission Control Protocol (TCP)", STD 7, RFC 9293, DOI 10.17487/RFC9293, August 2022, . 15.2. Informative References Templin Expires 28 July 2023 [Page 31] Internet-Draft IP Parcels January 2023 [BIG-TCP] Dumazet, E., "BIG TCP, Netdev 0x15 Conference (virtual), https://netdevconf.info/0x15/session.html?BIG-TCP", 31 August 2021. [I-D.ietf-6man-mtu-option] Hinden, R. M. and G. Fairhurst, "IPv6 Minimum Path MTU Hop-by-Hop Option", Work in Progress, Internet-Draft, draft-ietf-6man-mtu-option-15, 10 May 2022, . [I-D.templin-dtn-ltpfrag] Templin, F., "LTP Fragmentation", Work in Progress, Internet-Draft, draft-templin-dtn-ltpfrag-09, 25 July 2022, . [I-D.templin-intarea-aero] Templin, F., "Automatic Extended Route Optimization (AERO)", Work in Progress, Internet-Draft, draft-templin- intarea-aero-14, 10 January 2023, . [I-D.templin-intarea-omni] Templin, F. L., "Transmission of IP Packets over Overlay Multilink Network (OMNI) Interfaces", Work in Progress, Internet-Draft, draft-templin-intarea-omni-15, 23 January 2023, . [QUIC] Ghedini, A., "Accelerating UDP packet transmission for QUIC, https://blog.cloudflare.com/accelerating-udp-packet- transmission-for-quic/", 8 January 2020. [RFC1063] Mogul, J., Kent, C., Partridge, C., and K. McCloghrie, "IP MTU discovery options", RFC 1063, DOI 10.17487/RFC1063, July 1988, . [RFC1071] Braden, R., Borman, D., and C. Partridge, "Computing the Internet checksum", RFC 1071, DOI 10.17487/RFC1071, September 1988, . [RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191, DOI 10.17487/RFC1191, November 1990, . Templin Expires 28 July 2023 [Page 32] Internet-Draft IP Parcels January 2023 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, December 2005, . [RFC4821] Mathis, M. and J. Heffner, "Packetization Layer Path MTU Discovery", RFC 4821, DOI 10.17487/RFC4821, March 2007, . [RFC5326] Ramadas, M., Burleigh, S., and S. Farrell, "Licklider Transmission Protocol - Specification", RFC 5326, DOI 10.17487/RFC5326, September 2008, . [RFC7126] Gont, F., Atkinson, R., and C. Pignataro, "Recommendations on Filtering of IPv4 Packets Containing IPv4 Options", BCP 186, RFC 7126, DOI 10.17487/RFC7126, February 2014, . [RFC8201] McCann, J., Deering, S., Mogul, J., and R. Hinden, Ed., "Path MTU Discovery for IP version 6", STD 87, RFC 8201, DOI 10.17487/RFC8201, July 2017, . [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, . [RFC8899] Fairhurst, G., Jones, T., Tüxen, M., Rüngeler, I., and T. Völker, "Packetization Layer Path MTU Discovery for Datagram Transports", RFC 8899, DOI 10.17487/RFC8899, September 2020, . [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, May 2021, . [RFC9171] Burleigh, S., Fall, K., and E. Birrane, III, "Bundle Protocol Version 7", RFC 9171, DOI 10.17487/RFC9171, January 2022, . Templin Expires 28 July 2023 [Page 33] Internet-Draft IP Parcels January 2023 Appendix A. IP Parcel Futures Both historic and modern-day data links configure Maximum Transmission Units (MTUs) that are far smaller than the desired state for IP parcel transmission futures. When the first Ethernet data links were deployed many decades ago, their 1500 octet MTU set a strong precedent that was widely adopted. This same size now appears as the predominant MTU limit for most paths in the Internet today, although modern link deployments with MTUs as large as 9KB have begun to emerge. In the late 1980's, the Fiber Distributed Data Interface (FDDI) standard defined a new link type with MTU slightly larger than 4500 octets. The goal of the larger MTU was to increase performance by a factor of 10 over the ubiquitous 10Mbps and 1500-octet MTU Ethernet technologies of the time. Many factors including a failure to harmonize MTU diversity and an Ethernet performance increase to 100Mbps led to poor FDDI market reception. In the next decade, the 1990's saw new initiatives including ATM/AAL5 (9KB MTU) and HiPPI (64KB MTU) which offered high-speed data link alternatives with larger MTUs but again the inability to harmonize diversity derailed their momentum. By the end of the 1990s and leading into the 2000's, emergence of the 1Gbps, 10Gbps and even faster Ethernet performance levels seen today has obscured the fact that the modern Internet of the 21st century is still operating with 20th century MTUs! To bridge this gap, increased OMNI interface deployment in the near future will provide a virtual link type that can pass IP parcels over paths that traverse traditional data links with small MTUs. Performance analysis has proven that (single-threaded) receive-side performance is bounded by upper layer protocol segment size, with performance increasing in direct proportion with segment size. Experiments have also shown measurable (single-threaded) performance increases by including larger numbers of segments per parcel, with steady increases for including increasing number of segments. However, parallel receive-side processing will provide performance multiplier benefits since the multiple segments that arrive in a single parcel can be processed simultaneously instead of serially. In addition to the clear near-term benefits, IP parcels will increase performance to new levels as future parcel-capable links with very large MTUs begin to emerge. These links will provide MTUs far in excess of 64KB to as large as 16MB. With such large MTUs, the traditional CRC-32 (or even CRC-64) error checking with errored packet discard discipline will no longer apply for large parcels. Instead, parcels larger than a link-specific threshold will include Forward Error Correction (FEC) codes so that errored parcels can be repaired at the receiver's data link layer then delivered to upper Templin Expires 28 July 2023 [Page 34] Internet-Draft IP Parcels January 2023 layers rather than being discarded and triggering retransmission of large amounts of data. Even if the FEC repairs are incomplete or imperfect, all parcels can still be delivered to upper layers where the individual segment checksums will detect and discard any damaged data not repaired by lower layers. These new "super-links" will appear mostly in the network edges (e.g., high-performance data centers) and not as often in the middle of the Internet. (However, some space-domain links that extend over enormous distances may also benefit.) For this reason, a common use case will include parcel-capable super-links in the edge networks of both parties of an end-to-end session with an OMNI link connecting the two over wide area Internetworks. Medium- to moderately large- sized IP parcels over OMNI links will already provide considerable performance benefits for wide-area end-to-end communications while truly large IP parcels over super-links can provide boundless increases for localized bulk transfers in edge networks or for deep space long haul transmissions. The ability to grow and adapt without practical bound enabled by IP parcels will inevitably encourage new data link development leading to future innovations in new markets that will revolutionize the Internet. Until these new links begin to emerge, however, parcels will already provide a tremendous benefit to end systems by allowing applications to send and receive segment buffers larger than 65535 octets in a single system call. By expanding the current operating system call data copy limit from its current 16-bit length to a 32-bit length, applications will be able to send and receive maximum-length parcel buffers even if lower layers need to break them into multiple parcels to fit within the underlying interface MTU. For applications such as the Delay Tolerant Networking (DTN) Bundle Protocol [RFC9171], this will allow applications to send and receive entire large upper layer protocol constructs (such as DTN bundles) in a single system call. Appendix B. Change Log << RFC Editor - remove prior to publication >> Changes from earlier versions: * Submit for Intarea Standards Track RFC Publication. Author's Address Templin Expires 28 July 2023 [Page 35] Internet-Draft IP Parcels January 2023 Fred L. Templin (editor) Boeing Research & Technology P.O. Box 3707 Seattle, WA 98124 United States of America Email: fltemplin@acm.org Templin Expires 28 July 2023 [Page 36]