| Internet-Draft | BGP Route Broker | August 2023 |
| Xu, et al. | Expires 9 February 2024 | [Page] |
This document describes an optimized BGP route reflector mechanism, referred to as a BGP route broker, so as to use BGP-based IP VPN as an overlay routing protocol for hyperscale data center network virtualization environments, also known as Software-Defined Network (SDN) environments.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 9 February 2024.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
BGP/MPLS IP VPN has been successfully deployed in world-wide service provider networks for two decades and therefore it has been proved to be scalable enough in large-scale networks. Here, the BGP/MPLS IP VPN means both BGP/MPLS IPv4 VPN [RFC4364] and BGP/MPLS IPv6 VPN [RFC4659] . In addition, BGP/MPLS IP VPN-based data center network virtualization approaches described in [RFC7814], especially in the virtual PE model described in [I-D.ietf-bess-virtual-pe] have been widely deployed in small to medium-sized data centers for network virtualization purpose, also known as Software Defined Network (SDN). Examples include but not limited to OpenContrail.¶
When it comes to hyperscale cloud data centers typically housing tens of thousands of servers which in turn are virtualized as Virtual Machines (VMs) or containers, it usually means there would be at least tens of thousands of virtual PEs, millions of VPNs and tens of millions of VPN routes from the network virtualization perspective provided the virtual PE model as mentioned above (a.k.a., a host-based network virtualization model) is used. That means a significant challenge on both the BGP session capacity and the VPN routing table capacity of any given BGP router.¶
It’s no doubt that the route reflection mechanism should be considered in order to address the BGP scaling issues as mentioned above. Assume a typical one-level route reflector architecture is used, it's straightforward to distribute all the VPN routes supported by a data center into multiple route reflectors with each route reflector being preconfigured with a block of route targets associated with partial VPNs. In other words, there is no need for a single route reflector to maintain all the VPN routes supported by the data center. For redundancy purpose, more than one route reflector SHOULD be preconfigured with the same block of route targets.¶
Provided each virtual PE had been attached with at least one VPN corresponding to a given route reflector, that particular route reflector would have to establish BGP sessions with all virtual PEs, it would become a huge BGP session pressure on route reflectors.Now assume that another level (bottom-level) of route reflectors is introduced between the existing level (top-level) of router reflectors and the virtual PEs. Each top-level route reflectors would establish BGP sessions with all bottom-level route reflectors rather than all virtual PE routers. In addition, bottom-level just need to establish BGP sessions with a subset of all virtual PEs respectively. As a result, the scaling issue of the BGP session capacity is solved through the above partition mechanism. However, if the collection of VPNs attached to those route reflector clients (i.e., virtual PEs) belonging to a given bottom-level route reflector covers the all VPNs supported by the data center, that particular bottom-level route reflector would have to hold all the VPNs and all the VPN routes. It means a huge challenge on that particular route reflector.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
Assume the number of BGP sessions to be established on each bottom-level route reflectors can not be reduced further due to some reasons (e.g., it becomes unacceptable to manage too many route-reflectors), the number of VPN routes to be maintained on each bottom-level route reflectors should be alleviated by some means.¶
By learning from the widely-adopted high-performance message queuing mechanisms (e.g., RabbitMQ), the bottom-level route reflectors, referred to as route brokers in the following text, work as follows: they just need to maintain the route target membership information of their BGP peers and reflect VPN routes on demands without the requirement of maintaining VPN routes permanently. In a word, route brokers act as the message brokers/exchanges of the message queuing system, while top-level route reflectors, referred to as route servers, and virtual PEs, referred to as route broker clients, act as both message publishers/producers and subscribers/consumers of the message queuing system.¶
Route servers, advertise route target membership information as defined in [RFC4684] according to the preconfigured block of route targets. As such, route brokers know the VPNs associated with each of them. The route target membership information received form route servers SHOULD NOT be reflected by route brokers to any other iBGP peers further. Instead, a default route target membership information as defined in [RFC4684] SHOULD be advertised to its route broker clients so as to collect VPN routes originated from its route broker clients on behalf of route servers.¶
Route broker clients, advertise route target membership information according to the block of route targets which are dynamically configured. The route target membership information received from route broker clients would be deemed by route brokers as an implicit route request for all the VPN routes for the VPNs associated to the corresponding route targets, and only need to be reflected towards the corresponding route servers whose preconfigured block of route target cover the advertised route targets.¶
Upon receiving a route update message from a route server which contains VPN routes for a given VPN, route brokers would reflect the received routes to those of its route broker clients which are associated with that VPN. Upon receiving a route update message from a route broker client which contains VPN routes for a given VPN, route brokers would reflect the received routes to the other iBGP peers (including route servers and route broker clients) which are associated with that VPN.¶
Once the route reflection is finished, the above routes would be deleted.¶
Upon receiving an implicit route request for all the VPN routes for one or more VPNs (via the route target membership information advertisement) from a route broker client, route brokers SHOULD reflect that request to the corresponding route servers which are associated with the VPNs pertaining to the advertised route targets respectively.¶
Upon receiving the implicit route request reflected from the BGP broker, route servers SHOULD respond with the corresponding VPN routes to that broker which in turn reflects the received VPN routes to the route broker client. Once route reflection is finished, the received VPN routes would be deleted.¶
To alleviate the route request processing pressure on route servers, route brokers COULD optionally cache the VPN routes which are returned from route servers as a response to an implicit route request for a pre-configured time period. These cached VPN routes could be directly used when responding to the forthcoming route request for them.¶
When a route broker loses the BGP connection with a given route broker client, it SHOULD send a Notification message towards all route servers to indicate the failure of the BGP connection with that route broker client.¶
Upon receiving the above Notification message, route servers would withdraw all VPN routes with the BGP next-hop address being the failed route broker client.¶
The BGP router ID of the failed route broker client could be carried in a TLV, which in turn is carried in a Notification message with error code of TBD.¶
When all router servers which are configured with the same route target list are down, route brokers SHOULD notify their router broker clients to withdraw all the VPN routes for the VPNs associated with any route target within the above route target list.¶
The authors would like to thank Jie Dong and Robert Rasruk for their valuable comments and suggestions on this document.¶