Internet Research Task Force Patrick Frejborg Internet Draft October 7, 2010 Intended status: Experimental Expires: April 2011 Hierarchical IPv4 Framework draft-frejborg-hipv4-10.txt Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on April 7, 2011. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Frejborg Internet-Draft Hierarchical IPv4 Framework October 2010 Abstract This document describes a framework how the current IPv4 address space can be divided in two new address categories; a core address space (Area Locators, ALOC) that is globally unique and an edge address space (Endpoint Locators, ELOC) that is regionally unique - in the future the ELOC space is only significant in a private network or in a service provider domain, therefore a 32 x 32 bit addressing scheme and a hierarchical routing architecture is achieved. The hierarchical IPv4 framework is backwards compatible with the current IPv4 Internet; it also discusses a method to decouple the location and identifier functions, future applications can make use of the separation. The framework requires extensions to the existing Domain Name System, the existing IPv4 stack of the endpoints, middleboxes and to routers in the Internet. The framework can be implemented incrementally to endpoints, DNS, middleboxes and routers. Table of Contents 1. Requirements Notation........................................3 2. Introduction.................................................3 3. Definitions of Terms.........................................5 4. Intermediate Routing Architecture of hIPv4...................7 5. Decoupling Location and Identification......................10 6. Mandatory Extensions to Current Architectures (Unicast).....11 7. DNS Extensions to Support hIPv4.............................12 8. hIPv4 Header................................................13 9. ALOC Use Cases..............................................17 10. Life of a hIPv4 Session....................................21 11. Overlapping Source and Destination ELOC Prefixes/Ports.....24 12. ICMP Considerations........................................24 13. Multicast Considerations...................................25 14. Traffic Engineering Considerations.........................26 15. Large Encapsulated Packets.................................28 16. Mobility Considerations....................................28 17. Affected Applications and Implications.....................30 18. Long Term Routing Architecture of hIPv4....................31 19. Transition Considerations..................................34 20. Security Considerations....................................34 21. IANA Considerations........................................34 22. Conclusion.................................................35 23. References.................................................36 23.1. Normative References..................................36 23.2. Informative References................................36 24. Acknowledgements...........................................38 Appendix A. Short Term & Future IPv4 Address Allocation Policy.39 Appendix B. Multi-homing becomes Multi-pathing.................41 Frejborg Expires April 7, 2011 [Page 2] Internet-Draft Hierarchical IPv4 Framework October 2010 Appendix C. Incentives and Transition Arguments................45 Appendix D. Integration with CES Architectures.................47 1. Requirements Notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 2. Introduction The hierarchical IPv4 framework (hIPv4) has been developed to address the issues discussed in the [IAB report] from the Routing and Addressing Workshop that was held by the Internet Architecture Board (IAB) on October 18-19, 2006, in Amsterdam, Netherlands. Note: the hierarchical IPv4 framework is abbreviated as hIPv4 which is close to the abbreviation of Host Identity Protocol [HIP], thus the reader need to pay attention to these two abbreviations - hIPv4 and HIP are two different architectures. This document is a contribution to the IRTF Routing Research Group. The views in this document are considered controversial by the IRTF Routing Research Group (RRG) but the group reached a consensus that the document should still be published. The current IP addressing (IPv4) and the future addressing (IPv6) schemes of Internet are single dimensional by their nature. This limitation, i.e. the single level addressing scheme, has created some roadblocks - e.g. breaking end-to-end connectivity due to NAT, limited deployment of [SCTP] etc. - for further growth of Internet. If we compare Internet's current addressing schemes to other global addressing or location schemes we can notice that the other schemes use several levels in their structures. E.g. the postal system uses street address, city and country to locate a destination. Also to locate a geographical site we are using longitude and latitude in the cartography system. The other global network, the Public Switched Telephone Network (PSTN), have been build upon a three level numbering scheme that have enabled a hierarchical signaling architecture. By expanding the current IPv4 addressing scheme from a single level to a two level addressing structure most of the issues discussed in the IAB report can be solved. Also a hierarchical addressing scheme would better describe the Internet we have in place today - it seems that the architecture of Internet changed quite radically from the designed architecture by the introduction of [RFC1918] which divides the hosts into three categories and the address space into two categories - i.e. globally unique and private address spaces. This invention provided further growth of Internet Frejborg Expires April 7, 2011 [Page 3] Internet-Draft Hierarchical IPv4 Framework October 2010 and extended the life of the IPv4 address space, much longer than expected. RFC1918 didn't solve the multi-homing requirements for category 3 hosts - i.e. multi-homed sites with globally unique addresses at endpoints to be accessed from Internet - multi-homing have imposed some challenges for the routing architecture that [RRG] is addressing in the [RRG Recommendation] report. Almost all proposals in the RRG Recommendation report suggest a core and edge locator separation or elimination to create a scalable routing architecture. The core locator space can be viewed to be similar as the globally unique address space and the edge locator space similar as the private address space. RFC1918 has already demonstrated that Internet scales better with the help of categorized address spaces, the RRG proposals are also indicating that Internet will scale even further by introducing core and edge locators - then why not then change the addressing scheme (both IPv4 and IPv6 addressing schemes though this document is only focusing on IPv4) to better reflect the current and forthcoming Internet routing architecture? Continuing the use of a flat addressing scheme combined with core (global) and edge (private) locator (address) categories, the routing architecture must support additional mechanisms - such as NAT, tunneling or locator rewriting with the help of an identifier - to overcome the mismatch, and information is lost or hided for the endpoints. With a two-level addressing scheme these additional mechanisms can be removed and core/edge locators can be used to create new routing and forwarding directives. A convenient way to understand the two-level addressing scheme of the hIPv4 framework is to compare it to the PSTN numbering scheme (E.164) which uses country codes, national destination codes and subscriber numbers. The Area Locator (ALOC) prefix in the hIPv4 addressing scheme can be considered similar as to the country code in PSTN, i.e. the ALOC prefix locates an area in Internet, the area is called an ALOC realm. The Endpoint Locator (ELOC) prefixes in hIPv4 can be compared to the subscriber numbers in PSTN, i.e. the ELOC is regionally unique (in the future, locally unique) at the attached ALOC realm - the ELOC can also be attached simultaneously to several ALOCs realms. By inserting the ALOC and ELOC elements as a shim header (similar as in [MPLS] and [RBridge] architectures) between the IPv4 header and the transport protocol header, a hIPv4 header is created. From the network point of view, the hIPv4 header "looks and feels like" an IPv4 header - thus fulfilling some of the goals as outlined in [EIP] and in the early definition of [Nimrod] - outcome is that the current forwarding plane do not need to be upgraded though some minor changes are needed in the control plane (e.g. ICMP extensions). Frejborg Expires April 7, 2011 [Page 4] Internet-Draft Hierarchical IPv4 Framework October 2010 3. Definitions of Terms This document makes use of the following terms: Regional Internet Registry (RIR): Is an organization overseeing the allocation and registration of Internet number resources within a particular region of the world. Resources include IP addresses (both IPv4 and IPv6) and autonomous system numbers. Locator: A locator is a name for a point of attachment within the topology at a given layer. Objects that change their point of attachment(s) will need to change their associated locator(s). In the hIPv4 framework two types of locators have been defined, Area Locator (ALOC) and Endpoint Locator (ELOC). Global Locator Block (GLB): An IPv4 address block that is globally unique. Area Locator (ALOC): An IPv4 address (/32) assigned to locate an ALOC realm in Internet. The ALOC is assigned by a RIR to a service provider. Endpoint Locator (ELOC): An IPv4 address assigned to locate an endpoint in a local network. The ELOC block is assigned by a RIR to a service provider or to an enterprise. The ELOC block is only unique in a geographical region. The final policy of uniqueness shall be defined by the RIRs. ALOC realm: An area in the Internet with at least one attached Locator Swap Router (LSR), also an ALOC must be assigned to the ALOC realm. The RIB of an ALOC realm holds both local ELOC prefixes and global ALOC prefixes. An ALOC realm exchanges only ALOC prefixes with other ALOC realms. Locator Swap Router (LSR): Frejborg Expires April 7, 2011 [Page 5] Internet-Draft Hierarchical IPv4 Framework October 2010 A router or node which is capable to process the hIPv4 header; once the header is processed the LSR forwards the packet relying on the destination address in the IP header. The LSR must have the ALOC assigned as its locator. Locator Header: A 4 or 12 byte field, inserted between the IP header and transport protocol header. If a host identifier scheme is used the size of the locator header is further expanded. Identifier: An identifier is the name of an object at a given layer; identifiers have no topological sensitivity, and do not have to change, even if the object changes its point(s) of attachment within the network topology. Session: A session is an interactive information exchange between endpoints that is established at a certain time and torn down at a later time. Provider Independent Address Space (PI addresses/prefixes): An IPv4 address block which is assigned by a Regional Internet Registries directly to an end user organization. Provider Aggregatable Address Space (PA addresses/prefixes): An IPv4 address block assigned by a Regional Internet Registry to an Internet Service Provider which can be aggregated into a single route advertisement. Site mobility: A site wishes to changes its attachment point to the Internet without changing its IP address block. Endpoint mobility: An endpoint moves relatively rapidly between different networks, changing its IP layer network attachment point. Subflow: Frejborg Expires April 7, 2011 [Page 6] Internet-Draft Hierarchical IPv4 Framework October 2010 A flow of packets operating over an individual path, the flow forms part of a larger transport protocol connection. 4. Intermediate Routing Architecture of hIPv4 As mentioned in the introduction section the role of an Area LOCator (ALOC) prefix is similar as to a country code in PSTN (i.e. the ALOC prefix provides a location functionality of an area within an Autonomous System (AS), or an area spanning over a group of AS, in Internet). An AS can have several ALOC prefixes assigned, e.g. for traffic engineering purposes such as load balancing among several ingress/egress points. The ALOC prefix is used for routing and forwarding purposes in Internet, thus the ALOC prefix must be globally unique and is allocated from an IPv4 address block. This globally unique IPv4 address block is called the Global Locator Block (GLB). When an area within an AS (or a group of AS) are assigned an ALOC prefix the area has the potential to become an ALOC realm. In order to establish an ALOC realm more elements, further than the ALOC prefix, are needed. One or multiple Locator Swap Routers (LSR) must be attached to the ALOC realm. A LSR element is a node capable of swapping the prefixes of the IP header and the new shim header, called the locator header. The swap mechanism of the headers is described in detail in section 10, step 4. Today's routers do not support this LSR functionality. Therefore the new functionality will most likely be developed on an external device attached to a router belonging to the ALOC realm. The external LSR might be a server with two interfaces attached to a router, the first interface configured with the prefix of the ALOC and the second interface with any IPv4 prefix. The LSR do not make us of dynamic routing protocols, neither a forwarding information base (FIB) nor a cache is needed - the LSR is producing a service, i.e. swapping headers. The swap mechanism is applied on per packet basis and the information needed to carry out the swap is included in the locator header of the hIPv4 packet. Thus a standalone device with sufficient computing and I/O resources to handle the incoming traffic can take the role as a LSR. Later on, the LSR functionality might be integrated into the forwarding plane of a router. One LSR can not handle all the incoming traffic designated for an ALOC realm; it would also create a potential single point of failure in the network. Therefore, several LSRs might be installed in the ALOC realm and the LSRs shall use the ALOC prefix as their locator and the routers are announcing the ALOC prefix as an anycast address within the local ALOC realm. Also, the ALOC prefix is advertised throughout the DFZ by BGP mechanisms. The placement of the LSRs in the network will influence on the ingress traffic to the ALOC realm. Frejborg Expires April 7, 2011 [Page 7] Internet-Draft Hierarchical IPv4 Framework October 2010 Since the forwarding paradigm of multicast packets is quite different from forwarding unicast packets the multicast functionality will have an impact on the LSR. Also, the multicast LSR (mLSR) functionality is not available on today's routers, an external device is needed - later on the functionality might be integrated to the routers. The mLSR shall take the role of an anycast Rendezvous Point with [MSDP] and [PIM] capabilities, but to forward packets a FIB is not required. As with the LSR, the multicast hIPv4 packets are carrying all needed information in their headers in order to apply the swap, for details see section 13. The ALOC realm is not yet fully constructed, we can now locate the ALOC realm in Internet but to locate the endpoints attached to the ALOC realm a new element is needed, i.e. the Endpoint LOCator (ELOC). As mentioned in the introduction section the ELOC prefixes can be considered similar as to the subscriber numbers in PSTN. Actually, the ELOC is not a new element; the ELOC is a redefinition of the current IPv4 address configured at an endpoint. The redefinition is applied because when the hIPv4 framework is fully implemented the global uniqueness of the IPv4 addresses are no longer valid and a more regional address allocation policy of IPv4 addresses can be deployed as discussed in appendix A. The ELOC prefix will only be used for routing and forwarding purposes inside the local and remote ALOC realms, the ELOC prefix is not used in the intermediate ALOC realms. When an initiator is establishing a session to a responder residing outside the local ALOC realm the destination address field in the IP header of an outgoing packet is no longer the remote destination address (ELOC prefix) - instead the remote ALOC prefix is installed in the destination address field of the IP header. Because the destination address is an ALOC prefix, the intermediate ALOC realms do not need to carry the ELOC prefixes of other ALOC realms in their RIB - it is sufficient for the intermediate ALOC realms to carry only the ALOC prefixes. Outcome is that the RIB and FIB tables at each ALOC realm will be reduced when the hIPV4 framework is fully implemented. The ALOC prefixes are still globally unique and must be installed in the DFZ - thus the service provider can't control the growth of the ALOC prefixes but she/he can control the amount of local ELOC prefixes in her/his local ALOC realm. When the hIPv4 packet arrives at the remote ALOC realm it will be forwarded to the nearest LSR, since the destination address is the remote ALOC prefix. When the LSR has swapped the hIPv4 header, the destination address is the remote ELOC, thus the hIPv4 packet will be forwarded to the final destination at the remote ALOC realm. An endpoint using an ELOC prefix can be attached simultaneously to two different ALOC realms without the requirement to deploy a classical multi-homing solution, for details see section 16. Frejborg Expires April 7, 2011 [Page 8] Internet-Draft Hierarchical IPv4 Framework October 2010 Next, how can we locate the remote ELOC (endpoint) and remote ALOC realm in Internet, also when to assemble the header of the hIPv4 packet? Another matter is that the addressing structure is no longer single dimensional; instead a second level has been added on top of the old one. It is obvious that the Domain Name System needs to support a new record type so that the ALOC information can be distributed to the endpoints. To construct the header of the hIPv4 packet either the endpoint or an intermediate node (e.g. a proxy) should be used. A proxy solution is complicated, the proxy needs to listen to DNS messages and a cache solution does have scalability issues. A better solution is to extend the current IPv4 stack at the endpoints so that the ALOC and ELOC elements are incorporated at the endpoint's stack, but backwards compatibility must be preserved. Most of the application will not be aware of the extensions - other applications, such as Mobile IP, SIP, IPsec AH and so on (see section 17) will suffer and can not be used outside their ALOC realm when the hIPv4 framework is fully implemented - unless these applications are upgraded. The reason is that the IP aware applications are depending upon the underlying network addressing structure to e.g. identify an endpoint. Note: the applications used inside the local ALOC realm (e.g. enterprise's private network) do not need to be upgraded - neither in the intermediate nor in the long term hIPv4 architecture. The classical IPv4 framework is preserved - only IP aware applications used between ALOC realms needs to be upgraded to support the hIPv4 header. Figure 1 shows a conceptual overview of the intermediate hIPv4 routing architecture. When the intermediate architecture is in place the ELOC space is no longer globally unique, instead a regional allocation policy can be implemented, for further details see appendix A. The transition from the current architecture to the intermediate architecture is discussed in appendix D. Legend: UER=Unique ELOC region *=attachment point in the ALOC realm EP=Endpoint Frejborg Expires April 7, 2011 [Page 9] Internet-Draft Hierarchical IPv4 Framework October 2010 |-------------------------------------------------------------| | UER1 | | UER2 | |-------------------------------------------------------------| | Enterprise1 | ISP1 | ISP | ISP2 | Enterprise2 | | ALOC Realm | ALOC Realm | Tier1 | ALOC Realm | ALOC Realm | | | | | | | | *EP | *LSR | | *LSR | *EP | | ELOC1 | ALOC1 | | ALOC2 | ELOC4 | | | | | | | | | *EP | | *EP | | | | ELOC2 | | ELOC3 | | | | | | | | |-------------|xxxxxxxxxxxxxx DFZ xxxxxxxxxxxxxx| ------------| | RIB | RIB | RIB | RIB | RIB | | | | | | | | ALOC1 | ALOC1 | ALOC1 | ALOC2 | ALOC2 | | ELOC1 | ALOC2 | ALOC2 | ALOC1 | ELOC4 | | | ELOC2 | | ELOC3 | | | | ELOC1 | | ELOC4 | | | | | | | | |-------------------------------------------------------------| Figure 1: Intermediate routing architecture of hIPv4 5. Decoupling Location and Identification The design guidelines and rationale behind decoupling the location from identification is stated in the [RRG Design Goals] document. Another important influence source are the report and presentations from the [Dagstuhl] workshop that declared "a future Internet architecture must hence decouple the functions of IP addresses as names, locators, and forwarding directives in order to facilitate the growth and new network-topological dynamisms of the Internet". Therefore, identifier elements need to be added to the hIPv4 framework to provide a path for future applications to be able to remove the current dependency of the underlying network layer addressing scheme (local and remote IP address tuples). Multipath transport protocols, such as [SCTP] and the currently under development Multipath TCP [MPTCP], are the most interesting candidates to enable an identifier functionality for the hIPv4 framework. Especially MPTCP is interesting from hIPv4's point of view - one of the main goals of MPTCP is to provide backwards compatibility with current implementations, hIPv4 share the same goal. MPTCP itself do not provide a host identifier solution as e.g. Host Identity Protocol [HIP] does, instead MPTCP is proposing a token - with local meaning - to manage and bundle subflows under one Frejborg Expires April 7, 2011 [Page 10] Internet-Draft Hierarchical IPv4 Framework October 2010 session between two endpoints. The token can be considered to have the characteristics of a session identifier, providing a generic cookie mechanism for the application layer and creating a session layer between the application and transport layer. Thus the usage of a session identifier will provide a mechanism to improve mobility, both in site and endpoint mobility scenarios. Since the session identifier is improving site and endpoint mobility, routing scalability is improved by introducing a hierarchical address space, why then add a host identifier solution to the hIPv4 framework? Introducing a host identifier scheme, as described in [HIP], Identifier/Locator Network Protocol [ILNP] and Name-Based Sockets [NBS], might ease or remove the locator renumbering dependencies at firewalls that are used to scope security zones but this approach would change fundamentally the current deployed security architecture. However, combining a host identifier scheme with [DNSSEC] is interesting. Today security zones are scoped by using locator prefixes in the security rule sets - instead FQDN could be used in the rule sets and the renumbering of locator prefixes would no longer be depended upon the security rule sets in firewalls. Another interesting aspect is that a FQDN is and needs to be globally unique, the ALOC prefix must be globally unique but ELOC prefixes are only regionally unique, in long term only locally unique. Nevertheless, combining host identifiers with a security architecture and [DNSSEC] needs further studies. In order to provide multi-homing and mobility capabilities for single path transport protocols, e.g. TCP and UDP, a host identifier scheme is needed. The host identifier scheme can also be used to create a bidirectional NAT traversal solution with a locator translation map consisting of private locator prefixes and public host identities at the border router. 6. Mandatory Extensions to Current Architectures (Unicast) To implement the hierarchical IPv4 framework some basic rules are needed: 1. The DNS architecture must support a new extension, i.e. an A type Resource Record should be able to associate ALOC prefix(es). 2. An endpoint upgraded to support hIPv4 shall have information about the local ALOC prefix(es); the local ALOC prefix(es) can be configured manually or provided via provisioning means such as DHCP. Frejborg Expires April 7, 2011 [Page 11] Internet-Draft Hierarchical IPv4 Framework October 2010 3. A globally unique IPv4 address block shall be reserved; this block is called the Global Locator Block (GLB). A service provider can have one or several ALOC prefixes allocated from the GLB. 4. ALOC prefixes are announced via current BGP protocol to adjacent peers, the ALOC prefixes are installed in the RIB of the DFZ. When the hIPV4 framework is fully implemented only ALOC prefixes are announced between the BGP peers in the DFZ. 5. A hIPv4 capable ALOC realm must have one or several LSRs attached to its realm. The ALOC prefix is configured as an anycast IP address on the LSR. The anycast IP address is installed to appropriate routing protocols in order to be distributed to the DFZ. 6. The IPv4 socket API at endpoints must be extended to support local and remote ALOC prefixes. The modified IPv4 socket API must be backwards compatible with the current IPv4 socket API. The outgoing hIPv4 packet must be assembled by the hIPv4 stack with the local IP address from the socket as the source address and the remote ALOC prefix as the destination address in the IP header. The local ALOC prefix is inserted in the ALOC field of the locator header. The remote IP address from the socket API is inserted in the ELOC field of the locator header. 7. DNS Extensions to Support hIPv4 Since the hierarchical IPv4 framework introduces an extended addressing scheme and as DNS serves as the "phone book" for Internet it is obvious that DNS need a new Resource Record (RR) type to serve endpoints that are upgraded to support hIPv4. The new RR type must follow the guidelines as described in [RFC3597] and [RFC5395] with the following characteristics: o associated with the appropriated Fully Qualified Domain Name (FQDN), inserted in the NAME field o assigned a new integer (QTYPE) in the TYPE field, to be assigned by IANA o CLASS field is set to IN o RDATA field is of unknown type as defined in [RFC3597] and shall have the following format: Frejborg Expires April 7, 2011 [Page 12] Internet-Draft Hierarchical IPv4 Framework October 2010 o Preference sub-field: A 16-bit integer which specifies the preference given to this RR among others associated with a FQDN. Lower values are preferred over higher values. o ALOC sub-field: A 32-bit integer which specifics the Area Locator of the associated FQDN. +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Preference | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | | | ALOC | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Figure 2: RDATA format of the ALOC RR Only endpoints that have been upgraded to support hIPv4 shall make use of the new ALOC RR. Also, there is no need to define a new ELOC RR because the A RR is used for that purpose when the ALOC RR is returned. 8. hIPv4 Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | Protocol | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |A|I|S| FI|VLB|L| Protocol | LH Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Area Locator (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Endpoint Locator (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | LH Length (optional) | Padding (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: hIPv4 header Frejborg Expires April 7, 2011 [Page 13] Internet-Draft Hierarchical IPv4 Framework October 2010 Version: 4 bits The Version field is identical to that of RFC 791. IHL: 4 bits Internet Header Length field is identical to that of RFC 791. Type of Service: 8 bits The Type of Service is identical to that of RFC 791. Total Length: 16 bits The Total Length field is identical to that of RFC 791. Identification: 16 bits The Identification field is identical to that of RFC 791. Flags: 3 bits The Flags field is identical to that of RFC 791. Fragment Offset: 13 bits The Fragment Offset field is identical to that of RFC 791. Time to Live: 8 bits The Time to Live field is identical to that of RFC 791. Protocol: 8 bits A new protocol number must be assigned for hIPv4. Header Checksum: 16 bits The Header Checksum field is identical to that of RFC 791. Source Address: 32 bits The Source Address field is identical to that of RFC 791. Destination Address: 32 bits The Destination Address field is identical to that of RFC 791. Frejborg Expires April 7, 2011 [Page 14] Internet-Draft Hierarchical IPv4 Framework October 2010 Options and Padding: Variable length The Options and padding field is identical to that of RFC 791. ALOC Realm Bit, A-bit: 1 bit When the source and destination endpoints reside in different ALOC realms, the A-bit is set to 1 and the Area and Endpoint Locator fields must be used in the locator header. The size of the locator header is 12 bytes. When the A-bit is set to 0 the source and destination endpoints reside within the same ALOC realm, the Area and Endpoint Locator shall not be used in the locator header. The size of the locator header is 4 bytes. Identifier Bit, I-bit: 1 bit The identifier bit is set to 1 if the endpoint is using a host identifier scheme within the locator header. The host identifier scheme must indicate how much the size of the locator header is increased, thus the Locator Header Length field is also added to the locator header. Swap Bit, S-bit: 1 bit The initiator sets the swap bit to 0 of the hIPv4 packet. A LSR will set this bit to 1 when it is swapping the source and destination addresses of the IP header with the ALOC and ELOC prefixes of the locator header. Forwarding Indicator, FI-bits: 2 bits The purpose of the Forwarding Indicator (FI) field is to provide a mechanism for future forwarding plane to identify which forwarding information base should be used for inter-ALOC realm sessions. The new forwarding plane will remove the swap functionality of IP and locator header values for both unicast and multicast sessions. Outcome is that the IP and transport protocol headers will remain intact, only FI and Locator Header Checksum values in the locator header will alternate. The following values are defined: 00: Local ALOC exit forwarding mode. The initiator shall set the FI-bits to 00 and the ALOC prefix in the locator header is used to forward the packets to the border router which is the owner of the local ALOC prefix. The border router shall change the FI-bits to 01. Frejborg Expires April 7, 2011 [Page 15] Internet-Draft Hierarchical IPv4 Framework October 2010 01: DFZ forwarding mode. The local ALOC border router shall forward the packets upon the destination address in the IP header. The DFZ routers shall forward the packets based upon the destination address unless the destination address matches the local ALOC prefix - when this situation occurs the packet enters the remote ALOC realm and the remote border router shall change the FI-bits to 11. 11: Remote ALOC entry forwarding mode. The remote ALOC border router and following routers shall forward the packets upon the ELOC prefix in the locator header. 10: Inter-ALOC RFP check mode. The local ALOC border router change the FI-bits to 10 and following inter-ALOC routers on the shared tree shall apply RFP check upon the ALOC prefix in the locator header. Valiant Load-Balancing, VLB-bits: 2 bits (optional, subject for further research) The purpose of Valiant Load-Balancing field is to provide a mechanism for multipath enabled transport protocols to request explicit paths in the network for subflows, which are component parts of a session between two endpoints. The subflow path request can be set as following: 00: Latency sensitive application, only one single subflow (i.e. multipath not applied), shortest path through the network is requested. 01: First subflow, shortest path or Valiant Load-Balancing might be applied. 11: Next subflow(s), Valiant Load-Balancing should be applied Load-Balanced, L-bit: 1 bit (optional, subject for further research) The initiating endpoint must set the L-bit to zero. A Valiant Load- Balancing capable node can apply VLB switching for the session if the value is set to zero; if the value is set to 1 VLB switching is not allowed. When VLB switching is applied for the session the node applying the VLB algorithm must set the value to 1. Protocol: 8 bits The Protocol field is identical to that of RFC 791. Frejborg Expires April 7, 2011 [Page 16] Internet-Draft Hierarchical IPv4 Framework October 2010 Locator Header Checksum: 16 bits A checksum is calculated on the locator header only. The checksum is computed at the initiator, recomputed at the LSR and verified at the responder. The checksum algorithm is identical to that of RFC 791. Area Locator (optional): 32 bits An IPv4 address, the ALOC is assigned by a RIR to a service provider. The ALOC is globally unique because it is allocated from the GLB. Endpoint Locator (optional): 32 bits An IPv4 address, the ELOC block is assigned by a RIR to a service provider or to an enterprise. The ELOC block is only unique in a geographical region, if forwarding plane is replaced the ELOC block needs only to be locally significant. Locator Header Length (optional): 16 bits Locator Header Length is the length of the locator header. Locator Header Length is applied when the Identifier bit is set to 1. Host identifier parameters are inserted to the locator header after this field. Padding (optional): variable The locator header padding is used to ensure that the locator header ends on a 32 bit boundary. The padding is zero. 9. ALOC Use Cases Several ALOC use cases are explored in this section. As mentioned in section 4 ALOC is describing an area in Internet, the area can span over several Autonomous Systems (AS) or if the area is equal to an AS you can say that the ALOC is describing an AS. When the ALOC is describing an area it is hereafter called an anycast ALOC. The ALOC can also be used to describe a specific node between two ALOC realms, e.g. a node installed between a private and an ISP ALOC realm or between two private ALOC realms. In this use case the ALOC is describing an attachment point, e.g. where a private network is attached to Internet - the ALOC type is hereafter called a unicast ALOC. Frejborg Expires April 7, 2011 [Page 17] Internet-Draft Hierarchical IPv4 Framework October 2010 The main difference between anycast and unicast ALOC types is: o In an anycast ALOC scenario ELOC routing information is shared between the attached ALOC realms. o In a unicast ALOC scenario no ELOC routing information is shared between the attached ALOC realms. Unicast ALOC functionalities shall not yet be deployed between private and ISP ALOC realms - it would require too many locators from the GLB space - instead unicast ALOC functionality shall be used to separate private ALOC realms. ALOC space is divided into two types, a globally unique ALOC space (a.k.a, GLB) that is installed in DFZ and private ALOC space that is used inside private networks. Private ALOC are using the same locator space as defined in [RFC1918], the private ALOC must be unique inside the private network and should not overlap with private ELOC prefixes. Only ISPs should be allowed to apply for global ALOC prefixes, for further discussions see appendix A. The ISP should aggregate global ALOC prefixes as much as possible in order to reduce the size of the routing table in DFZ. When a user logs on the enterprise's network the endpoint will receive via a provisioning means (e.g. DHCP or manually configured) the following locator prefixes: o one ELOC prefix for each network interface o one private ALOC prefix or o several private ALOC prefixes if e.g. the enterprise network spans over high-speed long-distance connections - it is well known that TCP cannot sustain high throughput for extended periods of time, higher throughput might be achieved by using multipath paths concurrently - or the enterprise has recently been merged with another enterprise o one or several global ALOC prefixes, these ALOC describes how the enterprise network is attached to Internet As the user establishes a session to a remote endpoint DNS is usually used to resolve remote locator prefixes, DNS will return ELOC and ALOC prefixes of the remote endpoint. If no ALOC prefixes are returned a classical IPv4 session is initiated to the remote endpoint. When ALOC prefixes are returned the initiating endpoint is Frejborg Expires April 7, 2011 [Page 18] Internet-Draft Hierarchical IPv4 Framework October 2010 comparing the ALOC prefixes with its own local ALOC prefixes (that are provided via DHCP or manually configured): o if the remote ALOC prefix is from the private ALOC space the initiator shall use the given private ALOC prefix for the session. Two use cases exist to design a network to use private ALOC functionality. The remote endpoint is far away, i.e. leveraging high- speed long-distance connections, and in order to improve performance for the session a multipath transport protocol should be used. The other use case is when the remote endpoint resides in a network that recently has been merged and private ELOC [RFC1918] spaces overlaps if no renumbering is applied. One or several node ALOC solutions are needed in the network between the initiator and responder. For long distance sessions with no overlapping ELOC prefixes, anycast or unicast ALOC solutions can be deployed. Third use case follows; again the initiator compares returned ALOC prefixes from DNS with own local ALOC prefixes: o if the remote ALOC prefix is from the global ALOC space and the remote ALOC doesn't match the given global ALOC prefix the initiator shall use the given global ALOC prefix for the session. In this use case the remote endpoint resides outside the enterprise's private network, the global remote ALOC prefixes indicates how the remote network is attached to Internet. When a multipath transport protocol is used the subflows can be routed via separate border routers to the remote endpoint - both at the local and remote sites, if both are multi-homed. The egress hIPv4 packet (the initiator's packets in the local network) can be identified by the protocol value in the IP header, routed to an explicit path (e.g. MPLS LSP, L2TPv3 tunnel etc) upon the ALOC prefix in the locator header - a local ALOC exit routing scheme can be designed, for further details, see section 14 and appendix B. Figure 4 shows a conceptual diagram with two endpoints having a multipath session over a VPN connection and over Internet (in the intermediate hIPv4 routing architecture). The first subflow is established from the initiator (EP1) via uLSR3 and uLSR4 (both use a private unicast ALOC prefix) to the responder (EP2). Normal unicast forwarding is applied, i.e. uLSR4's ALOC prefix is installed in the RIB at the initiator's ALOC realm. Second subflow is established via Internet, i.e. via BR1->BR2 to EP2, 0/0 exit routing is used to enter Internet at the initiator's ALOC realm. Note: ELOC prefixes can Frejborg Expires April 7, 2011 [Page 19] Internet-Draft Hierarchical IPv4 Framework October 2010 overlap since the local and remote ALOC realms reside in different ELOC regions. Legend: UER=Unique ELOC region *=attachment point in the ALOC realm EP=Endpoint aLSR=anycast LSR uLSR=unicast LSR BR=Border Router |-------------------------------------------------------------| | UER1 | | UER2 | |-----------------------------------------------|-------------| | Enterprise1 | | Enterprise2 | | ALOC Realm | | ALOC Realm | | | | | | |---------------------------------| | | | VPN | | | | ALOC Realm | | | *uLSR3 uLSR4* | | |ALOC3 ALOC4| | | |xxxxxxxxxxxX VPN RIB xxxxxxxxxxxx| | | | | | | | ALOC3 & ALOC4 | | | |---------------------------------| | | | | | | *EP1 | | *EP2 | | ELOC1 |---------------------------------| ELOC2 | | | ISP1 | ISP | ISP2 | | | | ALOC Realm | Tier1 | ALOC Realm | | | | | | | | | | | | | | | BR1* *aLSR | | *aLSR *BR2 | | | ALOC1 | | ALOC2 | | | | | | | | | | | | | | |-------------|xxxxxxxxxxxxxx DFZ xxxxxxxxxxxxxx|-------------| | RIB | RIB | RIB | RIB | RIB | | | | | | | | ALOC1 | ALOC1 | ALOC1 | ALOC2 | ALOC2 | | ALOC3 | ALOC2 | ALOC2 | ALOC1 | ALOC4 | | ALOC4 | ELOC1 | | ELOC2 | ALOC3 | | ELOC1 | | | | ELOC2 | | | | | | | |-------------------------------------------------------------| Figure 4: Multi-pathing via VPN and Internet Frejborg Expires April 7, 2011 [Page 20] Internet-Draft Hierarchical IPv4 Framework October 2010 Fourth use case is to leverage the private and global ALOC functionalities to be aligned with the design and implementation of [Split-DNS] solutions. The fifth use case is for residential users, a residential user may use one or several ALOC prefixes; it depends upon the service offer and network design of the ISP. If the ISP prefers to offer advanced support for multipath transport protocols and local ALOC exit routing, the residential user is provided with several ALOC prefixes. The ALOC provided for residential users is taken from the GLB space and anycast ALOC functionality is applied. 10. Life of a hIPv4 Session This section provides an example of a hIPv4 session between two hIPv4 endpoints; an initiator and a responder residing in different ALOC realms. When the hIPv4 stack is assembling the packet for transport the hIPv4 stack shall decide if a classical IPv4 or a hIPv4 header is used upon the ALOC information received by a DNS reply. If the initiator's local ALOC prefix equals the responder's ALOC prefix there is no need to use the hIPv4 header for routing purposes, because both the initiator and responder reside in the local ALOC realm. The packet is routed upon the IP header since the packet will not exit the local ALOC realm. When the local ALOC prefix doesn't match the remote ALOC prefix a hIPv4 header must be assembled because the packet needs to be routed to a remote ALOC realm. A session between two endpoints inside an ALOC realm might use the locator header - not for routing purposes, but to make use of Valiant Load-Balancing [VLB] for multipath enabled transport protocols or make use of a host identifier scheme. The initiator can add the locator header to the packet and by setting the VLB bits to 01 indicating to the responder and intermediate routers that VLB is requested for the subflow. Because this is an intra-ALOC realm session there is no need to add ALOC and ELOC fields to the locator header, thus the size of the locator header will be 4 bytes. If a host identifier scheme is applied for the session (intra-ALOC or inter-ALOC), the scheme must set the I-bit to 1 and make use of the Locator Header Length field. Host identifier information is inserted to the locator header after the Locator Header Length field. How a hIPV4 session is established follows: Frejborg Expires April 7, 2011 [Page 21] Internet-Draft Hierarchical IPv4 Framework October 2010 1. The initiator queries the DNS server; the hIPv4 stack notice that the local and remote ALOC doesn't match and therefore must use the hIPv4 header for the session. The hIPv4 stack of the initiator must assemble the packet in the following way: a. set local IP address from API in the source address field of the IP header b. set remote IP address from API in the ELOC field of the locator header c. set local ALOC prefix in the ALOC field of the locator header d. set remote ALOC prefix in the destination address field of the IP header e. set the transport protocol value in the protocol field of the locator header and set the hIPv4 protocol value in the protocol field of the IP header f. set the desired parameters in the A-, I-, S-, FI-, VLB-, and L-fields of the locator header g. calculate IP-, locator- and transport protocol header checksums, transport protocol header calculation do not include the locator header fields. When completed the packet is transmitted. 2. The hIPv4 packet is routed throughout Internet upon the destination address of the IP header. 3. The hIPv4 packet will reach the closest LSR of the remote ALOC realm. When the LSR notice that the packet matches the local ALOC prefix the LSR must: a. verify the received packet that it uses the hIPv4 protocol value in the protocol field of the IP header b. verify IP-, locator- and transport protocol header checksums, transport protocol header verification do not include the locator header fields c. replace the source address in the IP header with the ALOC prefix of the locator header d. replace the destination address in the IP header with the ELOC prefix of the locator header Frejborg Expires April 7, 2011 [Page 22] Internet-Draft Hierarchical IPv4 Framework October 2010 e. replace the ALOC prefix in the locator header with the destination address of the IP header f. replace the ELOC prefix in the locator header with the source address of the IP header g. set the S-field to 1 h. decrease TTL value with one i. calculate IP-, locator- and transport protocol header checksums, transport header calculation do not include the locator header fields j. forward the packet upon the destination address of the IP header 4. The swapped hIPv4 packet is now routed inside the remote ALOC realm upon the new destination address of the IP header to the final destination. 5. The responder receives the hIPv4 packet a. the hIPv4 stack must verify the received packet that it uses the hIPv4 protocol value in the protocol field of the IP header b. verify IP-, locator- and transport protocol header checksums, transport protocol header verification do not include the locator header fields 6. The hIPv4 stack of the responder must present to the extended IPv4 socket API the following: a. present source address as the remote ALOC prefix b. present destination address as the local IP address c. verify that the received ALOC prefix equals the local ALOC prefix d. present ELOC as the remote IP address Frejborg Expires April 7, 2011 [Page 23] Internet-Draft Hierarchical IPv4 Framework October 2010 7. The responder's application will respond to the initiator and the returning packet will take almost the same steps, which are steps 1 to 6, as when the initiator started the session. In step 1 the responder doesn't need to do a DNS lookup since all information is provided by the packet. 11. Overlapping Source and Destination ELOC Prefixes/Ports Because an ELOC prefix is only significant within the local ALOC realm there is a slight possibility that a session between two endpoints residing in separate ALOC realms might use the same source and destination ELOC prefixes. But the session is still unique because the two processes communicating over the transport protocol form a logical session which is uniquely identifiable by the five tuples involved, i.e. by the combination of . The session might no longer be unique when two initiators with the same source ELOC prefix residing in two separate ALOC realms are accessing a responder located in a third ALOC realm. In this scenario a possibility exists that the initiators will use the same local port value. This situation will cause an "identical session situation" for the application layer. To overcome this scenario the hIPv4 stack must accept only one unique session with the help of the ALOC information. If there is an "identical session situation" - i.e. both initiators uses the same values in the five tuples - the hIPv4 stack shall allow only the first established session to continue, the following sessions must be prohibited and the initiator is informed by ICMP notification about the "identical session situation". MPTCP introduces a token, which is locally significant and currently defined as 32 bit long. The token will provide a sixth tuple for future applications to identify and verify the uniqueness of a session - the probability to have an "identical session situation" is further reduced. By adding a host identifier scheme to the hIPv4 framework the "identical session situation" is completely removed. 12. ICMP Considerations As long as the ICMP request is executed inside the local ALOC realm normal IPv4 ICMP mechanism can be used. As soon as the ICMP request exits the local ALOC realm the locator header shall be used in the notifications. Therefore extension to ICMP protocol shall be implemented; the extensions shall be compatible with [RFC4884] and support ALOC and ELOC elements. Frejborg Expires April 7, 2011 [Page 24] Internet-Draft Hierarchical IPv4 Framework October 2010 13. Multicast Considerations Since source ELOC prefixes are only installed in the routing table of the local ALOC realm there is a constraint with Reverse Path Forwarding (RPF) which is used to ensure loop-free forwarding of multicast packets. The source address of a multicast group (S,G) is used against the RFP check. The source address can no longer be used as a RFP checkpoint outside the local ALOC realm. To enable RPF globally for a (S,G), the multicast enabled LSR (mLSR) must at the source's ALOC realm replace the source address with the local ALOC prefix for inter-ALOC multicast streams. This can be achieved if the local LSR act also as an anycast Rendezvous Point with Multicast Source Discovery Protocol (MSDP) and Protocol Independent Multicast capabilities; with these functionalities the LSR becomes a multicast enabled LSR (mLSR). The source register at the mLSR and a source tree is established between the source and the mLSR. When an inter-ALOC realm receiver subscribes to the multicast group the mLSR have to swap the hIPv4 header in the following way: a. verify the received packet that it uses the hIPv4 protocol value in the protocol field of the IP header b. verify IP-, locator- and transport protocol header checksums c. replace the source address in the IP header with the local ALOC prefix d. set the S-field to 1 e. decrease TTL value with one f. calculate IP-, locator- and transport protocol header checksums, transport protocol header calculation do not include the locator header fields g. forward the packet to the shared multicast tree In order for the mLSR to function as described above the source must assemble the multicast hIPv4 packet in the following way: a. set local IP address (S) from API in the source address and the ELOC field b. set multicast address (G) from API in the destination address field Frejborg Expires April 7, 2011 [Page 25] Internet-Draft Hierarchical IPv4 Framework October 2010 c. set local ALOC prefix in the ALOC field d. set the transport protocol value in the protocol field of the locator header and set the hIPv4 protocol value in the protocol field of the IP header e. set the desired parameters in the A-, I-, S-, FI-, VLB-, and L-fields of the locator header f. calculate IP-, locator- and transport protocol header checksums, transport protocol header calculation do not include the locator header fields. When completed the packet is transmitted. The downstream routers from the mLSR towards the receiver will use the source address (which value is the source's ALOC prefix after the mLSR) in the IP header for RPF verification. In order for the receiver to create RTCP receiver reports all information is provided in the hIPv4 header of the packet. Because Source Specific Multicast (SSM) and IGMPv3 uses IP addresses in the payload both protocols needs to be modified to support the hIPv4 framework. 14. Traffic Engineering Considerations When hIPv4 framework is fully implemented ingress load balancing to an ALOC realm can be influenced by the placement of LSRs at the realm; a LSR provides a shortest path scheme. Also, if RIR policies allows, a service provider can have several ALOC assigned, hence traffic engineering and filtering can be done with the help of ALOC prefixes. E.g. sensitive traffic can be aggregated under one ALOC prefix which is not fully distributed into the DFZ. If needed an ALOC Traffic Engineering solution between ALOC realms might be developed, i.e. create explicit paths that can be engineered via specific ALOC prefixes, e.g. create mechanism similar as described in [Pathlet Routing]. Further studies are needed; first it should be evaluated if there is demand for such a solution. A use case of the ingress load balancing to an ALOC realm can be described as an "remote ALOC entry routing" solution for a multi- homed site, i.e. ingress traffic flow to the site is influenced by how many attachment points to Internet the site uses and where the attachment points are placed at the local network. In order to apply "local ALOC exit routing" from the multi-homed site some new network nodes are needed between the initiator and the border routers. The new network node(s) shall be able to identify hIPv4 packets, based Frejborg Expires April 7, 2011 [Page 26] Internet-Draft Hierarchical IPv4 Framework October 2010 upon the protocol field in the IP header, and switch the packets to explicit paths based upon the ALOC prefix in the locator header. Together with a multipath transport protocol the subflows can be routed via specific attachment points, i.e. border routers sitting between the local network and Internet. Multi-homing becomes multi- pathing, for details, see appendix B. The usage of multipath enabled transport protocols opens up the possibility to develop a new design methodology of backbone networks, i.e. Valiant Load-Balancing [VLB]. If two single-homed endpoints, using multipath enabled transport protocols and attached to the network with only one interface/ELOC-prefix, are communicating over Internet, both subflows will most likely take the shortest path throughout Internet. I.e. both subflows are established over the same links and when there is congestion on a link or a failure of a link both subflows might simultaneously drop packets - the benefit of multipath is lost. The "subflows-over-same-links" scenario can be avoided if the subflows are traffic engineered to traverse Internet on different paths - but this is difficult to achieve by using classical traffic engineering, such as IGP tuning or MPLS based traffic engineering. By adding a mechanism to the locator header the "subflows-over-same-links" scenario might be avoided. If the LSR functionality is deployed on a Valiant Load-Balancing enabled backbone node - hereafter called vLSR - and the backbone nodes are interconnected via logical full meshed connections, Valiant Load- Balancing can be applied for the subflows. When a subflow has the appropriated bits set in the VLB-field of the locator header the first ingress vLSR shall do VLB switching of the subflow. That is, the ingress vLSR is allowed to do VLB switching of the subflow's packets if the VLB bits are set to 01 or 11, the S-bit is set to 0 and the local ALOC prefix of the vLSR matches the ALOC-field's prefix. If there are no ALOC and ELOC fields in the locator header, but the other fields' values are set as described above, the vLSR should apply VLB switching as well for the subflow - because it is an inter-ALOC realm subflow belonging to a multipath enabled session. With this combination of parameters in the locator header the subflow is VLB switched only at the first ALOC realm and the subflows might be routed throughout the Internet on different paths. If VLB switching is applied in every ALOC realm it would most likely add too much latency for the subflows. The VLB switching at the first ALOC realm will not separate the subflows on the first and last mile links (single-homed) - if the subflows on the first and last mile links needs to be routed on separate links the endpoints should be deployed in a multi-homed environment. Studies on how Valiant Load-Balancing is influencing on traffic patterns between interconnected VLB [iVLB] backbone networks has been carried out. Nevertheless, more studies are needed around Valiant Load-Balancing scenarios. Frejborg Expires April 7, 2011 [Page 27] Internet-Draft Hierarchical IPv4 Framework October 2010 15. Large Encapsulated Packets Adding the locator header to an IPv4 packet in order to create a hIPv4 packet will increase the size of it but since the packet is assembled at the endpoint it will not add complications of current Path MTU Discovery (PMTUD) mechanism in the network. The intermediate network between two endpoints will not see any difference in the size of packet; IPv4 and hIPv4 packet sizes are the same from the network point of view. 16. Mobility Considerations This section will consider two types of mobility solutions, site mobility and endpoint mobility. Site mobility: Today, classical multi-homing is the most common solution for enterprises that wishes to achieve site mobility. Multi-homing is one of the key findings behind the growth of the DFZ RIB - see [IAB report], sections 2.1 and 3.1.2. The hIPv4 framework can provide a solution for enterprises to have site mobility without the requirement of implementing a classical multi-homed solution. This new multi-homed solution utilizing PI addresses is depended upon the forthcoming ELOC allocation policy which is discussed in appendix A. If the regional based ELOC allocation policy is enforced the enterprise can be concurrently attached to two different Internet service providers (ISP) without the need to implement AS border routing. The ISPs provide their globally unique ALOC prefixes for the enterprise and the ELOC block of the enterprise is regionally unique, a PI ELOC block. The enterprise can change the local ALOC prefix at endpoints - i.e. from the previous ISP's ALOC prefix to the new ISP's ALOC prefix - without connectivity interruptions inside the local network. Sessions initiated at the enterprise needs to be routed to the correct ISP, i.e. the border router (or intermediate routers between the initiator and border router) of the enterprise needs to apply local ALOC exit routing. For sessions initiated from the Internet the DNS record for an endpoint needs to be updated, also the local ALOC prefix on the endpoint needs be changed to achieve a symmetric entry/exit path at the site. Since the border router (or intermediate routers) is enforcing local ALOC exit routing the responder can apply basic session load balancing over the two ISPs based upon from which ISP the session has been initiated, i.e. if the responder have two valid DNS records with two different ALOC prefixes. Conclusion is that a single-homed enterprise can achieve smooth transition from one ISP to another by only changing the ALOC prefix on the endpoints and at DNS records - the local ELOC scheme Frejborg Expires April 7, 2011 [Page 28] Internet-Draft Hierarchical IPv4 Framework October 2010 remains intact. Also a single-homed enterprise can become multi-homed without implementing AS border routing or to have an own ALOC prefix assigned. If a better session load balancing scheme is required the application should be migrated to a multipath enabled transport protocol such as [SCTP] or [MPTCP]. Multi-homing is discussed in detail in appendix B. Endpoint mobility: Mobile IP [MIP] is used today for endpoints in order to provide mobility. Mobile IP is an overlay protocol; it is also an application that uses IP addresses in its payload. It is obvious that hIPv4 extensions need to be added to the MIP framework. Another approach is to investigate what [MPTCP] can offer to solve endpoint mobility scenarios. MPTCP introduces a token, which is locally significant and currently defined as 32 bit long. The token will provide a sixth tuple to identify and verify the uniqueness of a session. This sixth tuple - the token - is not depended upon the underlying layer, i.e. the IP layer. The session is identified with the help of the token and thus the application is not aware when the IP parameters are changed, e.g. during a roaming situation - but it is required that the application is not making use of IP addresses. Security issues arise; the token can be capture during the session by e.g. a man-in- the-middle attack - these attacks can be mitigated by applying [tcpcrypt]. If the application requires full protection against man- in-the-middle attacks the user should apply Transport Layer Security [TLS] Protocol for the session. To summarize, the most common endpoint mobility use case today is, that the responder resides in the fixed network and the initiator is mobile - thus MPTCP will provide roaming capabilities for the mobile endpoint - if both endpoints are making use of the MPTCP extension. However, in some use cases the fixed endpoint needs to initialize a session to a mobile responder. Thus MIP should incorporate the hIPv4 extension - MIP is providing a rendezvous service for the mobile endpoints. Also many applications are providing rendezvous services for their users, e.g. SIP, peer-to-peer, Instant Messaging services etc. A generic rendezvous service solution can be provided by a host identifier scheme, e.g. [HIP], [ILNP] or [NBS]. If desired the end user (actually the application) can make use of one of these rendezvous service schemes, i.e. MPTCP, extensions to MIP, some application specific rendezvous service or a generic rendezvous service - or some kind of combination of them. The routing architecture is only providing location information for the endpoints, i.e. the identifier mechanisms (session and host identifiers) are decoupled from the routing architecture. Frejborg Expires April 7, 2011 [Page 29] Internet-Draft Hierarchical IPv4 Framework October 2010 17. Affected Applications and Implications There are several applications that are inserting IP address information in the payload of a packet. Some applications use the IP address information to create new sessions or for identification purposes. This section is trying to list the applications that need to be enhanced; however, this is by no means a comprehensive list. The applications can be divided in four main categories: o Applications based on raw sockets, a raw socket is receiving packets containing the complete header in comparison to the other sockets that only receives the payload. o Applications needed to enable the hIPv4 framework, i.e. DNS and DHCP databases which must be extended to support ALOC prefixes. o Applications that insert IP addresses to the payload and uses the IP address for setting up new sessions or for some kind of identification. The application belonging to this category can not set up sessions to other ALOC domains until extensions have been incorporated. Within the local ALOC domain there are no restrictions since the current IPv4 scheme is still valid. The following applications have been identified: o SIP: IP addresses are inserted in the SDP offers/answers, XML body, Contact, Via, maddr, Route, Record-Route SIP headers; o Mobile IP: the mobile node uses several IP addresses during the registration process; o IPsec AH: designed to detect alterations at the IP packet header; o RSVP: RSVP messages are sent hop-by-hop between RSVP-capable routers to construct an explicit path; o ICMP: notifications needs to be able to incorporate ALOC information and assemble the hIPv4 header in order to be routed back to the source; o Source Specific Multicast: the receiver must specify the source address; o IGMPv3: a source-list is included in the IGMP reports; o Applications related to security, such as firewalls, must be enhanced to support ALOC prefixes; Frejborg Expires April 7, 2011 [Page 30] Internet-Draft Hierarchical IPv4 Framework October 2010 o Applications that will function with FQDN but many uses an IP addresses instead, such as ping, traceroute, telnet and so on. The CLI syntax needs to be upgraded to support ALOC and ELOC information via the extended socket API. At a first glance it seems that a lot of applications need to be re- engineered and migrated - but situation is not all that bad. The applications used inside the local ALOC realm (e.g. enterprise's private network) do not need to be upgraded - neither in the intermediate nor in the long term hIPv4 architecture. The classical IPv4 framework is preserved - only IP aware applications used between ALOC realms needs to be upgraded to support the hIPv4 header. IPv6 have the definitions of the applications mentioned above in place - but the migration of applications from IPv4 to IPv6 can impose some expenditure for the enterprises, especially if the applications are customized or homegrown, see [Porting IPv4 applications to dual stack]. As stated earlier, hIPv4 do not require to port applications used inside a private network. Conclusion is that, whatever next generation architecture is deployed, some applications will suffer, either during the transition period or to be re-engineered to be compatible with the new architecture. 18. Long Term Routing Architecture of hIPv4 The LSR was added to the framework in order to provide a smooth transition from the current IPv4 framework to the hIPv4 framework, i.e. a major forklift of the current forwarding plane is avoided by the introduction of the LSR element. In the future, the LSR can be left as such in the network, if preferred, or the LSR functionality can be expanded towards the edge when routers are upgraded due to their natural lifecycle process. Once an upgrade of a router is required because of e.g. increased demand for bandwidth, the modified forwarding plane might support concurrently IPv4 and hIPv4 forwarding - and the LSR functionality can be pushed towards the edge and in the future removed. This is accomplished by adding extension to the current routing protocols, both IGP and BGP. When a LSR receives a hIPv4 packet where the destination address matches the local ALOC prefix the LSR shall - contrary to the tasks defined in section 10, step 4 - lookup the ELOC field in the locator header and compare this prefix against the FIB. If the next-hop entry is LSR capable the packet shall be forwarded upon the ELOC prefix. If the next-hop is a classical IPv4 router the LSR must apply the tasks defined in section 10, step 4 and once completed forward the packet upon the new destination address. Once the routers from the first ingress LSR to the final destination endpoint is upgraded to support hIPv4 forwarding there exist no Frejborg Expires April 7, 2011 [Page 31] Internet-Draft Hierarchical IPv4 Framework October 2010 longer a need to implement LSR functionality in the network, the packet is forwarded as such to the endpoint's extended stack. The hIPv4 stack must check (the swap bit is not set to 1) that the ELOC prefix matches its local IP address. Then the hIPv4 stack of the destination must present to the extended IPv4 socket API the following: a. present source address as the remote IP address b. present the destination address as the local ALOC prefix c. present the ALOC prefix as the remote ALOC prefix d. present the ELOC prefix as the local IP address When all endpoints (that need to establish sessions outside the local ALOC realm) and infrastructure nodes are hIPv4 capable there is no need to apply LSR functionality for either unicast or multicast sessions - forwarding decisions can be made upon information in the IP and locator header. In the local ALOC realm packets are routed to their upstream anycast or unicast ALOC border router upon the ALOC prefix in the locator header - i.e. a local ALOC exit routing scheme is applied upon the local ALOC FIB. Since the anycast or unicast ALOC node is the owner of the local ALOC prefix it will forward the packets upon the destination address in the IP header - local ALOC exit routing halts at the border router and the FI-bits are changed to 01. At the intermediate ALOC realms the forwarding is based upon the packet's destination address in the IP header and the global DFZ FIB is used. Finally the packet arrives to the remote ALOC realm's border router - which is the owner of the remote ALOC prefix and matches the destination address in the IP header, thus the FI-bits are changed to 11. Hereafter forwarding of the packet is applied upon the ELOC prefix in the locator header, i.e. the remote ELOC FIB is used. Note that IP and transport protocol headers will remain intact, only FI and Locator Header Checksum values in the locator header will alternate. Figure 5 shows a conceptual overview of the long term hIPv4 routing architecture. Legend: UER=Unique ELOC region *=attachment point in the ALOC realm EP=Endpoint aLSR=anycast LSR uLSR=unicast LSR Frejborg Expires April 7, 2011 [Page 32] Internet-Draft Hierarchical IPv4 Framework October 2010 |-------------------------------------------------------------| | UER1 | UER2 | | UER3 | UER4 | |-------------------------------------------------------------| | Enterprise1 | ISP1 | ISP | ISP2 | Enterprise2 | | ALOC Realm | ALOC Realm | Tier1 | ALOC Realm | ALOC Realm | | | | | | | | *EP | *aLSR | | *aLSR | *EP | | ELOC1 | ALOC1.1 | | ALOC2.1 | ELOC4 | | | | | | | | *uLSR | | uLSR* | | |ALOC1.2 | | ALOC2.2| | | | | | | | | | *EP | | *EP | | | | ELOC2 | | ELOC3 | | | | | | | | |-------------|xxxxxxxxxxxxxx DFZ xxxxxxxxxxxxxx|-------------| | RIB | RIB | RIB | RIB | RIB | | | | | | | | ALOC1.2 | ALOC1.1 | ALOC1 | ALOC2.1 | ALOC2.2 | | ELOC1 | ALOC1.2 | ALOC2 | ALOC2.2 | ELOC4 | | | ALOC2 | | ALOC1 | | | | ELOC2 | | ELOC3 | | | | | | | | |-------------------------------------------------------------| Figure 5: Long term routing architecture of hIPv4 Also multicast LSR (mLSR) functionality can be removed when the forwarding plane is upgraded. The source's ALOC border router set the FI-bits to 10 and RFP check is hereafter applied on the ALOC prefix in the locator header - also here IP and transport protocol headers will not alternate. A long term evolution will provide a 32 x 32 bit locator space, the ALOC prefixes are allocated only by service providers - ELOC prefixes are only significant at a local ALOC realm, i.e. an enterprise can use a 32 bit locator space for its private network (the ALOC prefix is rented from the attached ISP) and an ISP can use a 32 bit ELOC space (for each ALOC prefix) to provide Internet connectivity services for its directly attached customers (residential and enterprise). An identifier placeholder has been added to the locator header; a host identifier scheme can be added but is not mandatory to enable it - a multipath transport protocol leverages a session identifier mechanism that provide mobility services and in some use cases an identifier mechanism is not needed at all. Frejborg Expires April 7, 2011 [Page 33] Internet-Draft Hierarchical IPv4 Framework October 2010 19. Transition Considerations The hIPv4 framework is not introducing any new protocols that would be mandatory to carry out the transition from IPv4 to hIPv4; instead extensions are added to existing protocols - the hIPv4 framework requires extensions to the current IPv4 stack, infrastructure systems and to some applications that use IP addresses in the payload but the current forwarding plane in Internet remains intact (intermediate architecture) apart from that a new forwarding element (the LSR) is required to create an ALOC realm. Extensions to the IPv4 stack, infrastructure systems and applications that uses IP addresses in the payload and routers can be deployed in parallel with the current IPv4 framework. Even genuine hIPv4 sessions can be established between endpoints though the current single dimensional Internet structure is still present. When will the single dimensional routing architecture then be upgraded to a hierarchical architecture? The author thinks there are two possible tipping points: o When the RIB of DFZ is getting close to the capabilities of current forwarding plane - who will pay for the upgrade? Or will the service provider only accept ALOC prefixes from other service providers and avoid capital expenditures? o When the depletion of IPv4 addresses is causing enough problems for service providers and enterprises. The biggest risk why hIPv4 framework will not succeed is the short timeframe before the expected depletion of the IPv4 address space occurs. Also, will enterprise give up their global allocation of the current IPv4 address block they have gained? Another risk is, will the enterprises and residences carry out an upgrade of their endpoints and middleboxes? Transitions arguments, incentives and methods are discussed in appendix C and appendix D. 20. Security Considerations Hijacking of a single ELOC prefix by longest match from another ALOC realm is no longer possible since the prefixes are separated by a locator, the ALOC. To apply a hijack of a certain ELOC prefix the whole ALOC realm must be routed via a bogus ALOC realm. Studies should be carried out with the Secure Inter-Domain Routing (SIDR) workgroup if the ALOC prefixes can be protected from hijacking. 21. IANA Considerations There are no IANA considerations for this document. Frejborg Expires April 7, 2011 [Page 34] Internet-Draft Hierarchical IPv4 Framework October 2010 22. Conclusion This document provides a high level overview of the hierarchical IPv4 framework which could be build in parallel with the current single dimensional Internet by implementing extensions at several architectures. Implementation of the hIPv4 framework will not require a major service window break in the Internet, neither at the private networks of enterprises. Basically, the hIPv4 framework is an evolution of the current IPv4 framework. The transition to hIPv4 might be attractive for the enterprises since the hIPv4 framework do not create a catch-22 situation, e.g. when should an application used only inside the private network be ported from IPv4 to IPv6 - also what is the business justification to port the application to IPv6? Another matter is, when an IPv4/v6 dual- stack solution is used it could impose operational expenditures, especially with rule sets at firewalls - both in front of servers and at clients. If the enterprise chooses to deploy hIPv4 the legacy applications do not need to be ported - because hIPv4 is backwards compatible with the classical IPv4 framework. Outcome is less costs for the enterprises and an additional bonus is the new stack's capabilities to better serve mobility use cases. But the enterprises must take the decision soon and act promptly, the IPv4 address depletion is a reality in a very near future. If the decision is delayed IPv6 will arrive, then, sooner or later the legacy applications need to be ported. However, though this document has focused only on IPv4, a similar scheme can be deployed for IPv6 in the future, i.e. creating a 64 x 64 bit locator space. But some benefits would have been lost at this stage, such as: o backwards compatibility with the current existing Internet, thus no smooth migration plan is gained. o the locator header, including ALOC and ELOC prefixes, would have been larger, 160 bits versus 96 bits. And the identifier (EUI-64) would always have been present, which can be consider as pros or cons, depending upon ones view of the privacy issue - as discussed in [RFC4941]. If the enterprises prefer hIPv4 (e.g. due to smooth migration capabilities) there is an unintentional side-effect (from the enterprises point of view) for the routing architecture of Internet. Multi-homing becomes multi-pathing, an opportunity opens up for the service providers to create an Internet routing architecture that holds less prefixes and generates less BGP updates in DFZ than compared with the current deployed architecture. Frejborg Expires April 7, 2011 [Page 35] Internet-Draft Hierarchical IPv4 Framework October 2010 23. References 23.1. Normative References [MPLS] Rosen, E., Tappan, D., Fedorkow, G., Rekther, Y., Farinacci, D., Li, T., Conta, A., "MPLS Label Stack Encoding", RFC 3032, January 2001 [RFC4884] Bonica, R., Gan, D., Tappan, D., Pignataro, C. "Extended ICMP to support Multi-Part Messages", RFC 4884, April 2007 [RFC1918] Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G.J., Lear, E. "Address Allocation for Private Internets", RFC 1918, February 1996 [MIP] Perkins, C. "IP Mobility Support for IPv4", RFC 3344, August 2002 [EIP] Wang, Z. "The Extended Internet Protocol", RFC 1385, November 1992 [TLS] Dierks, T., Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.2", RFC 5246, August 2008 [RFC1812] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812, June 1995 [DNSSEC] Arends, R., Austein, R., Larson, M., Massey, D., Rose, S., "DNS Security Introduction and Requirements", RFC 4033, March 2005 [PIM] Fenner, B., Handley, M., Holbrook, H., Kouvelas, I. "Protocol Independent Multicast - Sparse Mode", RFC 4601, August 2006 23.2. Informative References [IAB report] Meyer, D., Zhang, L., Fall, K., "Report from the IAB Workshop on Routing and Addressing", RFC 4984, September 2007 [HIP] Moskowitz, R., Nikander, P. "Host Identity Protocol (HIP) Architecture", RFC 4423, May 2006 Frejborg Expires April 7, 2011 [Page 36] Internet-Draft Hierarchical IPv4 Framework October 2010 [SCTP] Stewart, R. "Stream Control Transmission Protocol", RFC 4960, September 2007 [RBridge] Perlman, R., "RBridges, Transparent Routing", Infocomm, 2004 [Dagstuhl] Arkko, J., Braun, M.B., Brim, S., Eggert, L., Vogt, C., Zhang, L., "Perspectives Workshop: Naming and Addressing in a Future Internet", Dagstuhl, 2009 [Nimrod] Chiappa N., "A New IP Routing and Addressing Architecture", 1991 [MPTCP] Ford, A., Raiciu, C., Handley, M., Barre, S., "TCP Extensions for Multipath Operation with Multiple Addresses", IETF TSV Area, July 2009 [VLB] Zhang-Shen, R., McKeown, N., "Designing a Predictable Internet Backbone with Valiant Load-Balancing", Stanford University [iVLB] Babaioff, M., Chuang, J., "On the Optimality and Interconnection of Valiant Load-Balancing Networks", University of California at Berkeley [RRG] RRG, "IRTF Routing Research Group Home Page", http://tools.ietf.org/group/irtf/trac/wiki/RoutingResearchG roup [BFD] Bidirectional Forwarding Detection Workgroup, http://www.ietf.org/dyn/wg/charter/bfd-charter.html [CES] Jen, D., Meisel, M., Yan, H. Massey, D., Wang, L., Zhang, B., Zhang, L., "Towards A New Internet Routing Architecture: Arguments for Separating Edges from Transit Core", Sigcomm [ILNP] Atkinson, R. "ILNP Concept of Operations", ILNP Intro [NBS] Vogt, C. "Simplifying Internet Applications Development With A Name-Based Sockets Interface", christianvogt [Pathlet Routing] Godfrey, P.G., Shenker, S., Stoica, I., "Pathlet Routing", Sigcomm Frejborg Expires April 7, 2011 [Page 37] Internet-Draft Hierarchical IPv4 Framework October 2010 [tcpcrypt] Bittau, A., Hamburg, M., Handley, M., Mazi`eres, D., Boneh, D., "The case for ubiquitous transport-level encryption", Usenix [LISP] Farinacci, D., Fuller, V., Meyer, D., Lewis, D., "Locator/ID Separation Protocol", LISP WG [RRG Recommendation] Li, T., "Recommendation for a Routing Architecture", IRTF Routing RG, September 2010 [RRG design Goals] Li, T., "Design Goals for Scalable Internet Routing", IRTF Routing RG, September 2010 [MSDP] Fenner, B., Meyer, D., "Multicast Source Discovery Protocol", RFC 3618, October 2003 [Split-DNS] BIND 9 Administrator Reference Manual, BIND9 [Porting IPv4 applications to dual stack] DeLong, O., "Porting IPv4 applications to dual stack, with examples", Apricot, February 2010 [RFC3597] Gustafsson, A., "Handling of Unknown DNS Resource Record (RR) Types", RFC 3597, September 2003 [RFC5395] Eastlake 3 , D., "Domain Name System (DNS) IANA Considerations", RFC 5395, November 2008 [RFC4941] Narten, T., Draves R., Krishnan, S., "Privacy Extensions for Stateless Address Autoconfiguration in IPv6", RFC 4941, September 2007 24. Acknowledgements The active participants at the Routing Research Group [RRG] mailing- list are acknowledged. The participants have provided ideas, proposals and discussions that have influenced on the architecture of the hIPv4 framework. Mohamed Boucadair has provided valuable review input. Frejborg Expires April 7, 2011 [Page 38] Internet-Draft Hierarchical IPv4 Framework October 2010 Appendix A. Short Term & Future IPv4 Address Allocation Policy In this section we will discuss and study how the hIPv4 framework could influence on the IPv4 address allocation policies to ensure that the new framework will enable some re-usage of IPv4 address blocks. It is the Regional Internet Registries (RIRs) that shall define the final policies. When the intermediate hIPv4 routing architecture (see figure 1) is fully implemented every ALOC realm could have a full IPv4 address space - except the GLB - to allocate ELOC blocks from. There are some implications though. In order for an enterprise to achieve site mobility, i.e. to change service provider without changing its ELOC scheme, the enterprise should implement an Autonomous System (AS) solution with ALOC prefix at the attachment point to the service provider. Larger enterprises do have the resources to implement AS border routing; most of the large enterprises have already implemented multi-homing solutions. The small and midsize enterprises (SME) may not have the resources to implement AS border routing, or the implementation introduces unnecessary costs for the SME. Also if every enterprise needs to have an ALOC prefix it will have an impact on the RIB at the DFZ, the RIB will be populated with a huge amount of ALOC prefixes. It is clear that a compromise is needed. A SME is usually single- homed and the SME should be able to reserve a PI ELOC block from the RIR without the need to be forced to create an ALOC realm, i.e. implement a LSR solution and AS border routing. The PI ELOC block is no longer globally unique, the SME can only reserve the PI ELOC block for the region where it is active or has it attachment point to Internet. The attachment point rarely changes to another country; therefore it is sufficient that the PI ELOC block is regionally unique. When the enterprise is replacing its Internet service provider the enterprise do not have to change its ELOC scheme - only the local ALOC prefix at the endpoints are changed. The internal traffic at an enterprise does not use the ALOC prefix, the internal routing is applied by the IP header and thus the internal routing and addressing architectures are preserved. Mergers and acquisitions of enterprise can cause ELOC conflicts, because the PI ELOC block is hereafter only regionally unique. If an enterprise in region A overtakes an enterprise in region B there is a slight chance that both enterprises can have overlapping ELOC spaces. If overlapping of ELOC spaces occurs the private unicast ALOC solution can be implemented to separate them - if all affected endpoints support the hIPv4 framework. Frejborg Expires April 7, 2011 [Page 39] Internet-Draft Hierarchical IPv4 Framework October 2010 Finally, residential users will receive only PA locators. When a residential user changes a service provider the residential user has to replace the locators. A PA ELOC block is no longer globally unique, every Internet service provider can use the PA ELOC blocks at their ALOC realms - the PA locators becomes kind of private locators for the service providers. If the forwarding plane and all hosts, that establish inter-ALOC realm sessions, are upgraded to support the hIPv4 framework - i.e. the long term hIPv4 routing architecture (see figure 3) is achieved - several interesting possibilities occur: o the regional allocation policy for PI ELOC spaces can be removed, the enterprise can make use of the whole IPv4 address space that is globally unique today - the ELOC space is hereafter only significant at a local ALOC realm. o in case of mergers or acquisitions of enterprises the private unicast ALOC solution can be used to separate overlapping ELOC spaces. o the GLB space can be expanded to make use of all 32 bits (since enterprises have no interest or benefits to apply for globally unique locators) for anycast and unicast ALOC allocations, only ISP are allowed to apply for GLB prefixes. o the anycast ALOC solution can be replaced with the global unicast ALOC solution since the ISP and enterprise no longer need to share ELOC routing information between each other. Also there is enough space in the GLB to allocate global unicast ALOC for every enterprise. o residential users will still use global anycast ALOC solutions and if they change service provider their locators needs to be replaced. Outcome is that a 32 x 32 bit locator space is achieved, when an enterprise is replacing an ISP with another ISP only the ALOC information is replaced at endpoints and infrastructure nodes - renumbering of ALOC prefixes can be automated by e.g. DHCP and extensions to IGP. Frejborg Expires April 7, 2011 [Page 40] Internet-Draft Hierarchical IPv4 Framework October 2010 Appendix B. Multi-homing becomes Multi-pathing When the transition to the intermediate hIPv4 routing architecture (see figure 1) is fully completed the RIB of an ISP, that has created an ALOC realm, will have the following entries: o the PA ELOC blocks of directly attached customers (e.g. residential and enterprises) o the PI ELOC blocks of directly attached customers o the globally unique ALOC prefixes, received from other service providers The ISP will not carry in its RIB any PA or PI ELOC blocks from other service providers. In order to do routing and forwarding of packets between ISPs only ALOC information of other ISPs is needed. Then the question is, how to keep the growth of ALOC reasonable - if the enterprise is using PI addresses, having an AS number and implementing BGP, why not apply for an ALOC prefix? The classical multi-homing is causing the biggest impact on growth of the size of the RIB in the DFZ - replacing a /20 IPv4 prefix with a /32 ALOC prefix will not reduce the size of the RIB in the DFZ. Most likely the only way to prevent this from happening is to have a yearly cost for the allocation of an ALOC prefix - except if you are a service provider that are providing access and/or transit traffic for your customers. And it is granted to have cost for allocating an ALOC prefix for the non-service providers, because when an enterprise is using an ALOC prefix the enterprise is reserving a FIB entry throughout the DFZ - and the ALOC FIB entry needs to have power, space, hardware and cooling on all the routers in the DFZ. By implementing this kind of ALOC allocating policy it will reduce the RIB size in the DFZ quite well, multi-homing will no longer increase the RIB size of the DFZ. But this policy will have some impact on the resilience behavior, by compressing routing information we will loose visibility in the network. In today's multi-homing solutions the network always know where the remote endpoint resides, in case of a link or network failure a backup path is calculated and an alternative path is found - all routers in the DFZ are aware of the change in the topology. This functionality has off-loaded the workload of the endpoints; they only need to find the closest ingress router and the network will deliver the packets to the egress router, regardless of what failures (almost) happen in the network. And with Frejborg Expires April 7, 2011 [Page 41] Internet-Draft Hierarchical IPv4 Framework October 2010 the growth of multi-homed prefixes the routers in the DFZ have been forced to carry a greater workload, perhaps close to their limits - the workload between the network and endpoints is not in balance. Conclusion is that the endpoints should take more responsibilities for their sessions and that way off-loads the workload in the network. How, lets walkthrough an example: A remote enterprise has been given an ELOC block 192.168.1.0/24 that is either via static routing or BGP announced to the upstream service providers. The upstream service providers are providing the ALOC information for the enterprise, i.e. 10.1.1.1 and 10.2.2.2. A remote endpoint has been installed, it has been given ELOC 192.168.1.1 - the ELOC is a locator defining where the remote endpoint is attached to the remote network. The remote endpoint has been assigned ALOCs 10.1.1.1 and 10.2.2.2 - an ALOC is a locator defining the attachment point of the remote network to Internet. The initiator (local endpoint), which has ELOC 172.16.1.1 and ALOC prefixes 10.3.3.3 & 10.4.4.4, has established a session by using source ALOC 10.3.3.3 to the responder (remote endpoint) at ELOC 192.168.1.1 and ALOC 10.1.1.1. I.e. both networks 192.168.10/24 and 172.16.1.0/24 are multi-homed. ALOC are not available in current IP stack's API but both ELOC are seen as the local and remote IP addresses in the API, so the application will communicate between IP addresses 172.16.1.1 and 192.168.1.1 - if ALOC prefixes are included the session is established between 10.3.3.3:172.16.1.1 and 10.1.1.1:192.168.1.1. Next a network failure occurs, the link between the responder border router (BR-R1) and service provider that owns ALOC 10.1.1.1 goes down. The border router of the initiator (BR-I3) will not be aware of the situation, because only ALOC information is exchanged between service providers and ELOC information is compressed to stay within ALOC realms. But BR-R1 will notice the link failure; BR-R1 could rewrite the ALOC field in the locator header for this session, i.e. from 10.1.1.1 to 10.2.2.2 and send the packets to the second service provider via BR-R2 - the session between the initiator 10.3.3.3:172.16.1.1 and the responder 10.2.2.2:192.168.1.1 remains intact because the five legacy tuples at the IP stack API do not change - only the ALOC prefix of the responder has changed and this information is not shown to the application. An assumption here is that the hIPv4 stack does accept changes of ALOC prefixes on the fly (more about this later). If the network link between the BR-I3 and ISP providing ALOC 10.3.3.3 fails, BR-I3 could rewrite the ALOC prefix in the locator header and route the packets via BR-I4 - and the session stays up. If there is a Frejborg Expires April 7, 2011 [Page 42] Internet-Draft Hierarchical IPv4 Framework October 2010 failure somewhere in the network the border routers might receive an ICMP destination unreachable message (if not blocked by security functionality) and thus try to switch to session over to the other ISP by replacing the ALOC prefixes in the hIPv4 header. Or the endpoints might try themselves to switch to the other ALOCs after a certain time-out in the session. In all session transition cases the five legacy API tuples remains intact. If border routers or one of the endpoint changes the ALOC prefix without a negotiation with the remote endpoint security issues arises. Can the endpoint(s) trust the remote endpoint when ALOC prefix(es) are changed on the fly - is it still the same remote endpoint or has the session been hijacked by a bogus endpoint? The obvious answer is that an identification mechanism is needed to ensure that after a change in the path or a change of the attachment point of the endpoint the endpoints are still the same. An identifier needs to be exchanged during the transition of the session. Two types of identifiers have been discussed on the [RRG] mailing-list, session and host identifiers. Both types of identifiers can be used to protect the session from being hijacked. A session identifier will provide a low level security mechanism, offering some protection against hijacking of the session and also provide mobility. [SCTP] uses the verification tag to identify the association; [MPTCP] incorporates a token functionality for the same purpose - both can be considered to fulfill the characteristics of a session identifier. [tcpcrypt] can be used to further mitigate session hijacking. If the application requires full protection against man-in-the-middle attacks the user should apply Transport Layer Security [TLS] Protocol for the session. Both transport protocols are also multipath capable. Implementing multipath capable transport protocols in a multi-homed environment will provide new capabilities such as: o concurrent and separate exit/entry paths via different attachment points at multi-homed sites o true dynamic load-balancing, the endpoints do not participate in any routing protocols or do not update rendezvous solutions due to network link or node failures o only a single NIC on the endpoints is required o in case of a border router or ISP failure, the multipath transport protocol will provide resilience By adding more intelligence at the endpoints, i.e. multipath enabled transport protocols, the workload of the network is off-loaded and can take less responsibility for providing visibility of destination Frejborg Expires April 7, 2011 [Page 43] Internet-Draft Hierarchical IPv4 Framework October 2010 prefixes in Internet - i.e. prefix compression in the DFZ can be applied and only the attachment points of a local network needs to be announced in the DFZ. And the IP address space no longer needs to be globally unique; it is sufficient that only a part is globally unique, the rest is only regionally unique (long term, locally unique) as discussed in appendix A. Outcome is that the current multi-homing solution can migrate towards a multi-pathing environment that will have the following characteristics: o AS number is not mandatory for enterprises o BGP protocol is not mandatory at the enterprise's border routers, static routing with Bidirectional Failure Detection [BFD] is an option o allocation of global ALOC prefix(es) for the enterprise should not be allowed, instead upstream ISPs are providing the global ALOC prefixes for the enterprise o MPTCP provides dynamic load-balancing without using routing protocols, several paths can be simultaneously used and thus resilience is achieved o provide low growth of RIB entries at the DFZ o when static routing is used between the enterprise and ISP: o the RIB size at the enterprise's border routers are not depended upon the size of the RIB in DFZ nor adjacent ISPs o the enterprise's border router can not cause BGP churn in the DFZ or in the adjacent ISPs' RIB o when dynamic routing is used between the enterprise and ISP: o the RIB size at the enterprise's border routers are depended upon the size of the RIB in DFZ and adjacent ISPs o the enterprise's border router can cause BGP churn for the adjacent ISPs but not in the DFZ o the cost of border router should be less expensive that in today's multi-homing solution Frejborg Expires April 7, 2011 [Page 44] Internet-Draft Hierarchical IPv4 Framework October 2010 Appendix C. Incentives and Transition Arguments The media has announced several times the meltdown of Internet and the depletion of IPv4 addresses - but the potential chaos has been postponed several times and the general public has lost their interest in these announcements. Perhaps other approaches could be worthwhile to study, instead try to find other valuable arguments that the general public could be interested in, such as: o Not all endpoints needs to be upgraded, only those endpoints that are directly attached to the Internet. These kinds of endpoints are portable laptops, smart mobile phones, proxies, and DMZ/frontend endpoints. But the most critical endpoints, the backend endpoints where enterprises keep their most critical business applications do not need to be upgraded; the backend endpoints should not be reached at all from Internet - only from the private network - and this functionality can be achieved with the hIPv4 framework, since it is backwards compatible with the current IPv4 stack. Outcome is that investments in legacy applications, used inside an ALOC realm, are preserved. o Mobility, it is estimated that the demand for applications that performs well over the wireless access network will increase. Introduction of MPTCP and host identifier schemes opens up a new possibility to create new solutions and applications that are optimized for mobility. The hIPv4 framework requires an upgrade of the endpoints' stack; if possible the hIPv4 stack should also contain MPTCP and host identifier scheme features. Applications designed for mobility could bring competitive benefits for the enterprises. o The intermediate routers in the network do not need immediately to be upgraded; the current forwarding plane can still be used. The benefit is that the current network equipment can be preserved at the service providers, enterprises and residences (except middleboxes). That means that the carbon footprint is a lot lower compared to other solutions. Many enterprises do have green programs and many residential users are concerned with the global warming issue. Frejborg Expires April 7, 2011 [Page 45] Internet-Draft Hierarchical IPv4 Framework October 2010 o The migration from IPv4 to IPv6 (current defined architecture) will increase the RIB and FIB throughout DFZ, will it require a new upgrade of the forwarding plane as discussed in the IAB report is unclear. Most likely an upgrade is needed, the outcome of deploying IPv4 and IPv6 concurrently is that the routers need to have larger memories for the RIB and FIB - every globally unique prefix is installed in the routers that are participating in the DFZ. Since the enterprise is reserving one or several RIB/FIB entries on every router in the DFZ it means that the enterprise is increasing the power consumption of Internet, thus increasing the carbon footprint. And many enterprises are committed to green programs - if hIPv4 gets deployed, the power consumption of Internet will not grow as much as compared in an IPv4 to IPv6 transition scenario. o Another issue, if the migration from IPv4 to IPv6 (current defined architecture) occurs, is that the routers in the DFZ most likely need to be upgraded to more expensive routers - as discussed in the IAB report. In the wealthy part of the world, where a large penetration of Internet users is already present, the service provider can pass along more easily the costs of the upgrade to their subscribers - with a "wealthy/high penetration" ratio the cost will not grow that much that the subscribers would abandon Internet. But in the less wealthy part of the world, where there is usually a lower penetration of subscribers, the cost of the upgrade cannot that easily be covered - a "less wealthy/low penetration" ratio could have a dramatic increase on the cost that needs to be passed along to the subscribers. And thus fewer subscribers could afford to get connected to the Internet. For the global enterprises and the enterprises in the less wealthy part of the world, this scenario could mean less potential customers and there could be situations when the nomads of the enterprises can't get connected to Internet. This is also not fair; every human being should have a fair chance to be able to enjoy the Internet experience - and the wealthy part of the world should take this right into consideration. Many enterprises are committed to Corporate Social Responsibility programs. Not only technical and economical arguments can be found, also other arguments that the general public is interested in and concerned about can be found. Such arguments as that the Internet becomes greener and more affordable for everyone than compared to current forecast of the evolution of Internet. These non-technical values need to be communicated to the general public, you could ask them - do you care about the Planet, People and Internet? If you do, please upgrade the stack on your Internet enabled device. Frejborg Expires April 7, 2011 [Page 46] Internet-Draft Hierarchical IPv4 Framework October 2010 Appendix D. Integration with CES Architectures Because the hIPv4 framework requires changes to the endpoints' stack it will take some time before the migration of the current IPv4 architecture to the intermediate hIPv4 routing architecture is fully completed. If a hIPv4 proxy solution could be used in front of classical IPv4 endpoints, the threshold for early adopters to start to migrate towards the hIPv4 framework would be less questionable and the migration phase most likely would also be much shorter. Thus it should be investigated if the hIPv4 framework can be integrated with Core-Edge Separation [CES] architectures. In a CES architecture the endpoints do not need to be modified - the main design goal of a CES solution is to minimize the PI-address entries in the DFZ and to preserve the current stack at the endpoints. But a CES solution requires a new mapping system and will also introduce a caching mechanism in the map-and-encapsulate network nodes. Much debate about scalability of a mapping system and the caching mechanism has been carried out at the [RRG] list - today it is unclear how well both solutions will scale - research work of both topics are still in progress. Since the CES architectures are dividing the address spaces in two new categories - one that is installed in the RIB of the DFZ and the other that is installed in the local networks - there are to some degree similarities between CES architectures and the hIPv4 framework. Actually, the invention of the LSR functionality was inspired by [LISP]. In order to describe how these two architectures might be integrated some terminology definitions are needed: CES-node: A network node installed in front of a local network that must have the following characteristics o map-and-encapsulate ingress functionality o map-and-encapsulate egress functionality o incorporate the hIPv4 stack o routing functionality, [RFC1812] o be able to apply policy based routing on the ALOC field in the locator header Frejborg Expires April 7, 2011 [Page 47] Internet-Draft Hierarchical IPv4 Framework October 2010 Note that the CES-node do not include the [MPTCP] extension, it would most likely put too much burden on the CES-node to signal and maintain MPTCP subflows for the cached hIPv4 entries. Consumer site: A site that is not publishing any services towards Internet, i.e. there are no entries in DNS for this site. This site is used by local endpoints to establish outbound connectivity, i.e. endpoints are initiating sessions from the site towards content sites. Usually this site is found at small enterprises and residencies. PA-addresses are usually assigned to this type of site. Content site: A site that is publishing services towards Internet, usually these services do have DNS entries. This site is used by local endpoints to establish both inbound and outbound connectivity. The large enterprises are using PI-addresses and the midsize/small enterprises are using either PI- or PA-address space. The CES architectures are aiming to reduce the PI-address entries in the DFZ, thus map-and-encapsulate egress functionality shall be installed in front of the content sites (map-and-encapsulate ingress functionality is required at the Internet Service Providers, ISP, but for the hIPv4-CES integration study the map-and-encapsulate ingress functionality at ISP are not interesting - but LSR functionality and provider map-and-encapsulate ingress functionality might reside in the same node). It is likely that the node containing map-and- encapsulate egress functionality will also contain map-and- encapsulate ingress functionality; it is most likely a router so the node just needs to support the hIPv4 stack and be able to apply policy-based routing upon the ALOC field of the locator header to become a CES-node. It is possible that the Large Content Providers (LCP) are not willing to install map-and-encapsulate functionality in front of their sites - if the caching mechanism is not fully reliable or if the mapping lookup delay do have an impact on their customers' end user experience then most likely the LCP will not adopt the CES architecture. In order to convince a LCP to adopt the CES architecture, it should provide a mechanism to mitigate the caching and mapping lookup delay risks. One method is to push the CES architectures to the edge - the closer to the edge you add new functionality the better it will scale, that is, if the endpoint stack is upgraded the caching mechanism is maintained by the endpoint itself. The mapping mechanism can be removed if the CES architecture's addressing scheme is Frejborg Expires April 7, 2011 [Page 48] Internet-Draft Hierarchical IPv4 Framework October 2010 replaced with the addressing scheme of hIPv4 when the CES solution is integrated at the endpoints. With this approach the LCP might install a CES-node in front of their sites; also some endpoints within the content site might be upgraded with the hIPv4 stack. If the LCP faces issues with the caching or mapping mechanisms the provider can ask its customers to upgrade their endpoints' stack to ensure a proper service level. At the same time the LCP promotes the migration from the current routing architecture to a new routing architecture, not for the sake of the routing architecture but instead to ensure a proper service level - you can say that a business model will promote the migration of a new routing architecture. The hIPv4 framework proposes that the IPv4 addresses (ELOC) should no longer be globally unique; once the transition is completed a more regional allocation can be deployed. But this is only possible once all endpoints (that are establishing sessions to other ALOC realms) have migrated to support the hIPv4 framework. Here the CES architecture can speed up the re-usage of IPv4 addresses, i.e. once an IPv4 address block has become an ELOC block it can be re-used within the other RIR regions - also without the requirement that all endpoints in Internet must first be upgraded. As said earlier the CES architecture is aiming to remove PI-addresses from the DFZ, thus the content sites are more or less the primary target for the roll-out of a CES solution. At large content sites a CES-node most likely will be installed - to upgrade all endpoints (that are providing services towards Internet) at a large content site will take time, it might be that the endpoints at the content site are upgraded only within their normal life-cycle process. But if the size of the content site is small the administrator either installs a CES-node or upgrades the endpoints' stack - a choice to be taken and the decision will be influenced by availability, reliability and economically feasibility. Once the content sites has been upgraded the PI-address entries has been removed from the DFZ. Most likely also some endpoints at the consumer sites have been upgraded to support the hIPv4 stack - especially if there have been issues with the caches or mapping delays that have influenced on the service levels at the LCPs. Then, how to keep track of the upgrade of the content sites - have they been migrated or not? If the content sites or content endpoint has been migrated the DNS records should have either a CES-node entry or ALOC entry for each A-record. When the penetration of CES solutions at content sites (followed up by CES-node/ALOC records in DNS) is high enough the ISP can start to promote the hIPv4 stack upgrade at the consumer sites. Once a PA-address block has been migrated it can be released from global allocation to a regional allocation. Why Frejborg Expires April 7, 2011 [Page 49] Internet-Draft Hierarchical IPv4 Framework October 2010 would then an ISP push its customer to migrate to hIPv4 stacks? It is due to the business model - it will be more expensive to stay in the current architecture. The depletion of IPv4 addresses will either cause more NAT at the service provider's network - operational expenditures will increase because the network will become more complex - or the ISP should force it's customers to migrate to IPv6 - but the ISP could loose customers to other ISPs that are offering IPv4 services. When PA-addresses has been migrated to the hIPv4 framework the ISP will have a more independent routing domain (ALOC realm) with only ALOC prefixes from other ISPs and ELOC prefixes from directly attached customers - BGP churn from other ISPs is no longer received, also the amount of alternative paths is reduced and the ISP can better control the growth of the RIB at their ALOC realm. The operational and capital expenditures should be lower that than in the current routing architecture. To summarize, the content providers might find the CES+hIPv4 solution attractive - it will remove the forthcoming IPv4 address depletion constraints without forcing the consumer to switch to IPv6, thus the content providers can continue to grow (reach more consumers). Also the ISP might find this solution attractive; it should reduce the capital and operational expenditures in long term. Both the content providers and the ISPs are providing the foundation of Internet - if both are adopting this architecture the consumers have to adopt, both providers might find business models to "guide" the consumers towards the new routing architecture. Then how will this affect the consumer and content sites? The residential users will need to upgrade their endpoints - but it doesn't really matter what IP protocol version they use - it is the availability and affordability of Internet that matters the most. Enterprises will be affected a little bit more. The edge devices at the enterprises local network needs to be upgraded - edge nodes such as AS border routers, middleboxes, DNS, DHCP and public nodes needs to be upgraded - but by installing a CES-node in front of them, the upgrade process is postponed - instead the legacy nodes can be upgraded during their normal life-cycle process. The internal infrastructure is preserved as such, internal applications can still use IPv4 and also all investment in IPv4 skills is preserved. Walkthrough of use cases: 1. A legacy endpoint at a content site establishes a session to a content site with a hIPv4 upgraded endpoint When the legacy endpoint resolves the DNS entry for the remote endpoint (a hIPv4 upgraded endpoint) the legacy endpoint will receive Frejborg Expires April 7, 2011 [Page 50] Internet-Draft Hierarchical IPv4 Framework October 2010 an ALOC record in the DNS response. The legacy endpoint will ignore the ALOC record, only the A-record is used to establish the session. Next, the legacy endpoint will initialize the session and a packet is sent towards the map-and-encapsulate ingress node, which need to do a lookup at the CES mapping system (assumption here is that no cache entry exist for the remote endpoint). The mapping system returns either a CES-node prefix or an ALOC prefix for the lookup; since the requested remote endpoint has been upgraded the mapping system returns an ALOC prefix. The CES-node will not use the CES encapsulation scheme for this session, instead the hIPv4 header scheme shall be used and a /32 entry will be created in the cache. A /32 entry must be created; it is possible that not all endpoints at the remote site are upgraded to support the hIPv4 framework. The /32 cache entry can be replaced with a shorter prefix in the cache if all endpoints are upgraded at the remote site. To indicate this situation a sub-field shall be added for the ALOC record in the mapping system. The CES-node must execute the following steps for the egress packets a. verify IP- and transport header checksums b. create the locator header, copy the destination address to the ELOC field of the locator header c. replace the destination address in the IP header with the ALOC prefix given in the cache d. insert the local CES-node prefix in the ALOC field of the locator header e. copy the transport protocol value of the IP header to the protocol field of the locator header and set the hIPv4 protocol value in the protocol field of the IP header f. set the desired parameters in the A-, I-, S-, FI-, VLB-, and L-fields of the locator header g. decrease TTL value with one h. calculate IP-, locator- and transport protocol header checksums, transport protocol header calculation do not include the locator header fields. When completed the packet is transmitted Frejborg Expires April 7, 2011 [Page 51] Internet-Draft Hierarchical IPv4 Framework October 2010 i. because the size of the packet might exceed MTU - due to insertion of the locator header - if MTU is exceeded the CES- node should inform the source endpoint with an ICMP message of the situation and the CES-node should apply fragmentation of the hIPv4 packet 2. A hIPv4 upgraded endpoint at a consumer/content site establishes a session to a content site with a CES-node in front of a legacy endpoint The hIPv4 upgraded endpoint receives in the DNS response either an ALOC record or a CES-node record for the resolved destination. From the requesting hIPv4 endpoint's point of view it really doesn't matter if the new record prefix is used to locate LSR-nodes or CES- nodes in Internet - the CES-node will act as a hIPv4 proxy in front of the remote legacy endpoint. Thus the hIPv4 endpoint assembles a hIPv4 packet to initialize the session, when the packet arrives at the CES-node it must execute the following: a. verify the received packet that it uses the hIPv4 protocol value in the protocol field of the IP header b. verify IP-, locator- and transport protocol header checksums, transport protocol header verification do not include the locator header fields c. replace the protocol field value of the IP header with the protocol field value of the locator header d. replace the source address in the IP header with the ELOC prefix of the locator header e. remove the locator header f. create a cache entry (unless an entry already exists) for returning packets, a /32 entry is required. To optimize the usage of cache entries, the CES-node might ask the CES mapping node if all endpoints at the remote site are upgraded or not - if upgraded a shorter prefix can be used in the cache. g. decrease TTL value with one h. calculate IP- and transport protocol header checksums i. forward the packet upon the destination address of the IP header Frejborg Expires April 7, 2011 [Page 52] Internet-Draft Hierarchical IPv4 Framework October 2010 3. A hIPv4 enabled endpoint with a regional unique ELOC at a consumer site establishes a session to a consumer site with a legacy endpoint. In this use case the sessions will fail unless some mechanisms is invented and implemented at the ISPs' map-and-encapsulate nodes. The sessions will work inside an ALOC realm since the classical IPv4 framework is still valid, sessions between ALOC realms will fail. Some applications are establishing sessions between consumer sites; the most common are gaming and peer-to-peer applications. These communities have historically been in the forefront to adopt new technologies - it is expected that the affected communities either develop workarounds to solve this issue or simply asking their members to upgrade their stacks. 4. A legacy endpoint at a consumer/content site establishes a session to a content site with a CES-node in front of a legacy endpoint Assumed to be described in CES architecture drafts 5. A hIPv4 enabled endpoint at a consumer/content site establishes a session to a content site with a hIPv4 enabled endpoint See section 10 Author's Address Patrick Frejborg Email: pfrejborg@gmail.com Frejborg Expires April 7, 2011 [Page 53]