INTERNET-DRAFT Enke Chen / MCI John W. Stewart, III / ISI January 1997 A Framework for Inter-Domain Route Aggregation Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months. Internet-Drafts may be updated, replaced or obsoleted by other documents at any time. It is not appropriate to use Internet- Drafts as reference material or to cite them other than as a "working draft" or "work in progress." Please check the abstract listing contained in each Internet-Draft directory to learn the current status of this or any other Internet- Draft. Abstract This document presents a framework for inter-domain route aggregation and shows an example router configuration which "implements" this framework. This framework is flexible and scales well as it emphasizes the philosophy of aggregation by the source, both within routing domains as well as towards upstream providers, and it also strongly encourages the use of the "no-export" BGP community to balance the provider-subscriber need for more granular routing information with the Internet's need for scalable inter-domain routing. 1. Introduction The need for route aggregation has long been recognized. Route aggregation is good as it reduces the size, and slows the growth, of the Internet routing table. Thus, the amount of resources (e.g., CPU and memory) required to process routing information is reduced and route calculation is sped up. Another benefit of route aggregation is that route flaps are limited (in number, frequency and scope), which both saves resources and makes the global Internet routing system more stable. Chen & Stewart [Page 1] INTERNET-DRAFT Framework for Route Aggregation January 1997 Since CIDR was introduced, significant progress has been made on route aggregation, particularly in the following two areas: - Formulation and implementation of IP address allocation policies by the top registries that conform to the CIDR principles. [1,2] This policy work is the cornerstone which makes efficient route aggregation technically possible. - Route aggregation by large (especially "Tier 1") providers. To date, the largest reductions in the size of the routing table have resulted from efficient aggregation by large providers. However, the ability of various levels of the global routing system to implement efficient aggregation schemes has varied widely. As a result, the size and growth rate of the Internet routing table remain major issues today. To support the explosive Internet growth, it is important to maximize the efficiency of aggregation at all levels in the routing system. Because of the current size of the routing system and its dynamic nature, the first step towards this goal is to establish a common and clearly defined framework in which scaleable inter-domain route aggregation can be realized. The framework described in this docu- ment emphasizes the philosophy of aggregation by the source, both within routing domains as well as towards upstream providers. The framework also strongly encourages the use of the "no-export" BGP community to balance the provider-subscriber need for more granular routing information with the Internet's need for scalable inter- domain routing. The advantages of this framework include the follow- ing: - Route aggregation is done in a distributed fashion. The Inter- net is too large and too distributed for aggregation to be per- formed successfully by anyone other than the party injecting the aggregatable routing information into the global mesh. - The flexibility of a routing domain to be able to inject more granular routing information to an adjacent domain to control the resulting traffic patterns, without having an impact on the global routing system. In addition to describing the philosophy, we illustrate it by presenting sample configurations. IPv4 prefixes, BGP4 and ASs are used in examples, though the principles are applicable to inter- domain route aggregation in general. Address allocation policies and technologies to readdress entire net- works, while very relevant to the realization of successful and Chen & Stewart [Page 2] INTERNET-DRAFT Framework for Route Aggregation January 1997 sustained inter-domain routing, are not the focus of this document. The references section contains pointers to relevant documents. 2. Route Aggregation Framework The framework of inter-domain route aggregation we are proposing can be summarized as follows: - Aggregation from the originating AS That is, in its outbound route announcement each AS aggregates the BGP routes originated by itself, by dedicated AS and by private-ASs. (Routes originated by an AS refers to routes whose AS number appears first in the AS-path attribute. For example, routes statically configured and injected into BGP fall into this category.) In general no proxy aggregation shall be performed. This is to preserve the capability of multi-homing by other ASs. An AS shall always originate via a stable mechanism (e.g., static route configuration) the BGP routes for the large aggre- gates from which it allocates addresses to customers. This ensures that it is safe for its customers to use BGP "no- export". - Using BGP community "no-export" toward upstream providers That is, in its route announcement toward its upstream provider an AS tags the BGP community "no-export" to routes it originates that do not need to be propagated beyond its upstream provider (e.g., prefixes allocated by the upstream provider). This framework is illustrated in Figure 1. An "Tier 1" provider does not use "no-export" in its announcement as it does not have a upstream provider. However, it shall aggregate the routes it ori- ginates in its outbound announcements towards both peer providers and customers. An AS with an upstream provider shall aggregate the routes it originates and use "no-export" toward its upstream provider for routes that do not need to be propagated beyond its provider's AS. This recursion shall apply to all levels of the routing hierar- chy. Chen & Stewart [Page 3] INTERNET-DRAFT Framework for Route Aggregation January 1997 Tier 1 +-- Provider <--+ | | o aggregate routes | | o announces customer routes it originates | | o aggregate routes it originates | ^ o use "no-export" if appropriate | +---> Tier 2 <--+ Provider | V | | | o aggregate routes | | o announces customer routes it originates | | o aggregate routes it originates | | o use "no-export" if appropriate | | | ^ -> Customer AS Figure 1 This framework scales well as aggregation is done at all levels of the routing system. It is flexible because the originating AS con- trols whether routes of finer granularity are injected to, or pro- pagated by, its upstream provider. It facilitate multi-homing without compromising route aggregation. This framework is detailed in the following sections. 3. Aggregation from the Originating AS It has been well recognized that address allocation and address renumbering are keys to containing the growth of the Internet routing table. They have been well documented in [1,2,11]. Although the strategies discussed in this document do not assume a perfect address allocation, it is strongly urged that an AS receive allocation from its upstream service providers' address block. 3.1 Intra-Domain Aggregation To reduce the number of routes that need to be injected into its own AS, there are a couple of principles that shall be followed: Chen & Stewart [Page 4] INTERNET-DRAFT Framework for Route Aggregation January 1997 - Carry in its BGP table the large route block allocated from its upstream provider or the InterNIC. This can be done by either static configuration of the large block or by aggregating more specific BGP routes. The former is recommented as it does not depend on other routes. - Allocate larger blocks to the access routers where further allo- cation is done. That is, the address allocation shall be done such that only a larger, less specific route (instead of multi- ple more specific routes) needs to be known to the other routers within the AS. For example, a prefix of /17 can be further allocated to dif- ferent access routers as /20s which can then be allocated to customers connected to different interfaces on that router (as shown in Figure 2). Then in general only the /20 needs to be injected into the whole AS. Exceptions need to be made for multi-homed static routes. access router +------------+ | x.x.x.x/20 | +------------+ | | | | | | /24 /22 /25 Figure 2 It is noted that rehoming of customers without renumbering even within the same AS may lead to injection of more specific routes. However, in general the more-specifics do not need to be advertised outside of that AS. Such routes can either be tagged with the BGP community "no-export" or filtered out by a prefix-based filter to prevent them from being advertised out. Chen & Stewart [Page 5] INTERNET-DRAFT Framework for Route Aggregation January 1997 3.2 Inter-Domain Aggregation There are at least two types of routes that need to be advertised by an AS: routes originated by the AS and routes originated by its BGP customers. An AS may need to advertise full routes to certain BGP customers in which case the routing announcement would include routes originated by non-customer ASs. Clearly an AS should and can safely aggregate the routes originated by itself and by its customers multi-homed just to it (using, e.g., the dedicated AS and by the private AS mechanism [draft-stewart-bgp-without-as-00.txt]) in its outbound announcement. But it is far more dangerous to aggregate routes originated by customer ASs due to multi-homing. However, there are several cases in which a route originated by a BGP customer (other than using the dedicated AS or private AS) does not need to be advertised out by its upstream providers. For example, - This route is more specific than the upstream provider's block. However, the customer is either singly homed; or its connection to this particular upstream provider is used for backup only. - The more-specifics of a larger block are announced by the custo- mer in order to balance traffic over the multiple links to the upstream provider. One approach to suppress such routes is to aggregate them (termed "proxy aggregation") although they are originated by other ASs. How- ever, due to the legitimate need for injecting more-specifics for multi-homing, proxy aggregation needs to be done with special coordi- nation and care. The coordination work involved could be non-trivial in a large environment, and it is usually problematic trying to keep the aggregation up-to-date. Instead of performing "proxy aggregation", our approach for dealing with these cases is to give control to the customers that originate the more specifics (as seen by its upstream providers) and let them tag the BGP community "no-export" to these routes when appropriate. The BGP community "no-export" is a well known BGP community [RFC1997]. A route with this attribute would not be propagated beyond an AS boundary. So, if a route is tagged with this community in its announcement to a upstream provider and is accepted by the upstream provider, the route will not be announced further by the upstream provider. This would achieve the goal of suppress these more specifics in the upstream provider's outbound announcement. In this framework, the BGP community "no-export" shall be tagged to routes that are to be advertized to, but not to be propagated by, its Chen & Stewart [Page 6] INTERNET-DRAFT Framework for Route Aggregation January 1997 upstream provider. They may include routes allocated out of its upstream provider's block, or the more specific routes announced to its upstream provider for the purpose of load balancing. This aggre- gation strategy can be implemented via prefix-based filtering as shown in the Example of Section 5. For its own protection, a customer shall announce only its own routes and its customer routes to its upstream providers. Thus, the out- bound routing announcement and aggregation policy can be expressed as follows: For routes originated by itself/dedicated AS/private AS: tag with "no-export" when appropriate, and advertise the large block and suppress the more-specifics. For routes originated by customer ASs: advertise out. For any other routes: do not advertise out. This approach is flexible and scales well as it gives control to the party with the special needs, distributes the workload, and avoids the coordination overhead for proxy aggregation. 4. Aggregation by a Provider A provider shall aggregate all the routes it originates, as docu- mented in Section 3. The only difference is that the provider may be providing full routes to certain BGP customers where no outbound filtering is presently in place. Experience has shown that incon- sistent route announcement (e.g., aggregate at the interconnects but not toward certain customers) can cause serious routing problems (due to longest match) for the Internet as a whole, and every effort shall be made to ensure consistent route aggregation for all BGP peers. This would mean deploying filters for the BGP peers receiving full routes. In certain cases announcing the more-specifics to customers would provide for more accurate IGP metrics and could be useful for better load-balancing by its customers. However, the potential risk seems to outweight this benefit, especially given the increasing complexity of connectivity a customer may have. Chen & Stewart [Page 7] INTERNET-DRAFT Framework for Route Aggregation January 1997 In summary, the aggregation strategy for a provider shall be: - In announcing customer routes: For routes originated by itself/dedicated AS/private AS: tag with "no-export" when appropriate, and advertise the large block and suppress the more specifics. For routes originated by other customer ASs: advertise out. For any other routes: do not advertise out. - In announcing full routes: For routes originated by itself/dedicated AS/private AS: tag with "no-export" when appropriate, and advertise the large block and suppress the more specifics. For any other routes: advertise out. 5. An Example Consider the example shown in Figure 3 where AS 1000 is a "Tier 1" provider with two large aggregates 208.128.0.0/12 and 166.55.0.0/16, and AS 2000 is a customer of AS 1000 with a portable address 160.75.0.0/16 (historical reason) and an address 208.128.0.0/19 allo- cated from AS 1000. Assume that 208.128.0.0/19 does not need to be propagated beyond AS 1000. Chen & Stewart [Page 8] INTERNET-DRAFT Framework for Route Aggregation January 1997 +----------------+ | AS 1000 | | 208.128.0.0/12 | | 166.55.0.0/16 | +----------------+ | | BGP | | +----------------+ | AS 2000 | | 208.128.0.0/19 | | 160.75.0.0/16 | +----------------+ Figure 3 Then, based on the framework presented, AS 1000 would - originate and advertise the BGP routes 208.128.0.0/12 and 166.70.0.0/16, and suppress the more specific originated by itself, and private ASs - advertise the routes received from the customer AS 2000 and AS 2000 would - originate BGP route 208.128.0.0/19 and 160.75.0.0/16 - advertise both 160.75.0.0/16 and 208.128.0.0/19 to its provider AS 1000 and suppress the more specifics originated but itself and private AS. Tag the route 208.128.0.0/19 with "no-export" in the announcement - advertise both 160.75.0.0/16 and 208.128.0.0/19 to its BGP cus- tomers (if any) and suppress the more specifics originated but itself and private AS, plus any other routes the customers may desire to receive. The sample configuration (in Cisco syntax) is given in the Appendix to implement these policies. Chen & Stewart [Page 9] INTERNET-DRAFT Framework for Route Aggregation January 1997 6. Acknowledgments The authors would like to thank Roy Alcala of MCI for a number of interesting hallway discussions related to this work. 7. References [1] Rekhter, Y., Li, T., "An Architecture for IP Address Allocation with CIDR", RFC 1518, September 1993. [2] Fuller, V., Li, T., Yu, J., and K. Varadhan, "Classless Inter- Domain Routing (CIDR): an Address Assignment and Aggregation Stra- tegy", RFC 1519, September 1993. [3] Rekhter, Y., and Li, T., "A Border Gateway Protocol 4 (BGP-4)", RFC1771, March 1995. [4] Rekhter, Y., and Gross, P., "Application of the Border Gateway Protocol in the Internet", RFC1772, March 1995. [5] Rekhter, Y., "Routing in a Multi-provider Internet", RFC1787, April 1995. [6] Chandra, R., Train, P., and Li, T., "BGP Communities Attribute", RFC 1997, August 1996. [7] Chen, E., and Bates, T., "An Application of the BGP Community Attribute in Multi-home Routing", RFC 1998, August 1996. [8] Ferguson, P., Berkowitz, H., "Network Renumbering Overview: Why would I want it and what is it anyway?", RFC 2071, January 1997. [9] Berkowitz, H., "Router Renumbering Guide", RFC 2072, January 1997. [10] Stewart, J., and Chen, E., "Using BGP Without Consuming a Unique ASN", Internet-Draft, January 1997 (expires July 1997), . [11] Carpenter, B., Crowcroft, J., Rekhter, Y., "IPv4 Address Behaviour Today", Internet-Draft, October 1996 (expires April 1997), . [12] Cisco systems, Cisco IOS Software Version 10.3 Router Products Configuration Guide (Addendum), May 1995. Chen & Stewart [Page 10] INTERNET-DRAFT Framework for Route Aggregation January 1997 8. Authors' Addresses Enke Chen MCI 2100 Reston Parkway Reston, VA 20191 phone: +1 703 715 7087 email: enke@mci.net John W. Stewart, III USC/ISI 4350 North Fairfax Drive Suite 620 Arlington, VA 22203 phone: +1 703 807 0132 email: jstewart@isi.edu A. Appendix A: Example Cisco Configuration This appendix lists the Cisco configurations for AS 2000 of the exam- ples presented in Section 5. The configuration here uses the AS- path for outbound filtering although it can also be based on BGP com- munity. Several route-maps are defined that can be used for peering with the upstream provider, and for peering with customers (announc- ing full routes or customer routes). Chen & Stewart [Page 11] INTERNET-DRAFT Framework for Route Aggregation January 1997 !!# inject aggregates ip route 160.75.0.0 255.255.0.0 Null0 254 ip route 208.128.0.0 255.255.224.0 Null0 254 ! router bgp 2000 network 160.75.0.0 mask 255.255.0.0 network 208.128.0.0 mask 255.255.224.0 neighbor x.x.x.x remote-as 1000 neighbor x.x.x.x route-map export-routes-to-provider out neighbor x.x.x.x send-community ! !!# match all ip as-path access-list 1 permit .* ! !!# List of internal AS and private ASs that are safe to aggregate ip as-path access-list 10 permit ^$ ip as-path access-list 10 permit ^64999_ ip as-path access-list 10 deny .* ! !!# list of other customer ASs ip as-path access-list 20 permit ^3000_ !!# List of prefixes to be tagged with "no-export" access-list 101 permit ip 208.128.0.0 0.0.0.0 255.255.224.0 0.0.0.0 !!# list of large aggregates access-list 102 permit ip 160.75.0.0 0.0.0.0 255.255.0.0 0.0.0.0 access-list 102 permit ip 208.128.0.0 0.0.0.0 255.255.224.0 0.0.0.0 ! !!# route-map with the upstream provider route-map export-routes-to-provider permit 10 match ip address 101 set community no-export route-map export-routes-to-provider permit 20 match as-path 10 match ip address 102 route-map export-routes-to-provider permit 20 match as-path 20 ! !!# route-map with BGP customers that desire only customer routes route-map export-customer-routes permit 10 match as-path 10 match ip address 102 route-map export-customer-routes permit 20 match as-path 20 ! !!# route-map with BGP customers that desire full routes route-map export-full-routes permit 10 Chen & Stewart [Page 12] INTERNET-DRAFT Framework for Route Aggregation January 1997 match as-path 10 match ip address 102 route-map export-full-routes permit 20 match as-path 1 ! Chen & Stewart [Page 13]