Flooding Reduction in CLOS Networks

Abstract

For a given OSPF (or ISIS) router within the CLOS topology, it would receive multiple copies of exactly the same LSA (or LSP) from multiple OSPF (or ISIS) neighbors. In addition, two OSPF (or ISIS) neighbors may send each other the same LSA (or LSP) simultaneously. The unnecessary link-state information flooding wastes the precious process resource of OSPF (or ISIS) routers. This document proposes extensions to OSPF (or ISIS) so as to reduce the OSPF (or ISIS) flooding within such CLOS networks. The reduction of the OSPF (or ISIS) flooding is much beneficial to improve the scalability of CLOS networks.¶

1. Introduction

For a given OSPF (or ISIS) router within the CLOS topology, it would receive multiple copies of exactly the same LSA (or LSP) from multiple OSPF (or ISIS) neighbors. In addition, two OSPF (or ISIS) neighbors may send each other the same LSA (or LSP) simultaneously. The unnecessary link-state information flooding wastes the precious process resource of OSPF (or ISIS) routers. This document proposes extensions to OSPF (or ISIS) so as to reduce the OSPF (or ISIS) flooding within such CLOS networks. The reduction of the OSPF (or ISIS) flooding is much beneficial to improve the scalability of CLOS networks.¶

As a result, some CLOS network operators had to choose BGP as the underlay routing protocol in their data centers [RFC7938]. However, with the emergence of high-performance Ethernet networks for AI and high performance computing (HPC), the visibility of the whole network topology, and even the link load information, is crucial for performing the end-to-end path load-balancing, also known as global load-balancing or adaptive routing. As a result, link-state routing protocols, such as OSPF (or ISIS), would have to be reconsidered as the routing protocol for large-scale AI and HPC Ethernet networks. Of course, the prerequisite is the scaling issue associated with link-state routing protocols as mentioned above could be well addressed.¶

This document describes a pragmatic approach to the above scaling issue. The basic idea is to configure partial routers as non-reflectors for a given OSPF area (or a given ISIS level) which are not allowed to reflect the LSAs (or LSPs) received from neighbors in that OSPF area (or that ISIS level) to neighbors in the same OSPF area (or the same ISIS level).¶

For a three-stage CLOS network (i.e., a leaf-spine network) as shown in Figure 1, all nodes are in OSPF area zero (or ISIS Level-2). All leaf nodes SHOULD be configured as non-reflectors. To further reduce the link-state flooding, all spine nodes except two of them COULD be configured as non-refletors as well.¶

For a five-stage CLOS network (i.e., a leaf-spine-superspine network) as shown in Figure 2, each PoD consist of leaf nodes and spine nodes is configured as an OSPF non-zero area (or an ISIS Level-1 area), each PoD-interconnect plane consist of spine nodes and super-spine nodes is configured as an OSPF area zero (or an ISIS Level-2 area) . As such, spine nodes act as OSPF area border routers (or ISIS Level-1-2 routers). All leaf nodes SHOULD be configured as non-reflectors. All spine nodes SHOULD be configured as non-reflectors for OSPF area zero (or ISIS Level-2). To further reduce the link-state flooding, all spine nodes of each pod except at least two of them are recommended to be configured as non-reflectors for the associated non-zero OSPF area (or ISIS Level-1 area), and all super-spine nodes of each PoD-interconnect plane except two of them COULD be configured as non-reflectors.¶

   +----+ +----+ +----+ +----+
   | S1 | | S2 | | S3 | | S4 |  (Spine)
   +----+ +----+ +----+ +----+

   +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+
   | L1 | | L2 | | L3 | | L4 | | L5 | | L6 | | L7 | | L8 |  (Leaf)
   +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+


                              Figure 1

   =========================================
   # +----+ +----+ +----+ +----+           #
   # | L1 | | L2 | | L3 | | L4 | (Leaf)    #
   # +----+ +----+ +----+ +----+           #
   #                                PoD-1  #
   # +----+ +----+ +----+ +----+           #
   # | S1 | | S2 | | S3 | | S4 | (Spine)   #
   # +----+ +----+ +----+ +----+           #
   =========================================

   ===============================     ===============================
   # +----+ +----+ +----+ +----+ #     # +----+ +----+ +----+ +----+ #
   # |SS1 | |SS2 | |SS3 | |SS4 | #     # |SS1 | |SS2 | |SS3 | |SS4 | #
   # +----+ +----+ +----+ +----+ #     # +----+ +----+ +----+ +----+ #
   #   (Super-Spine@Plane-1)     #     #   (Super-Spine@Plane-4)     #
   #============================== ... ===============================

   =========================================
   # +----+ +----+ +----+ +----+           #
   # | S1 | | S2 | | S3 | | S4 | (Spine)   #
   # +----+ +----+ +----+ +----+           #
   #                                PoD-8  #
   # +----+ +----+ +----+ +----+           #
   # | L1 | | L2 | | L3 | | L4 | (Leaf)    #
   # +----+ +----+ +----+ +----+           #
   =========================================

                              Figure 2

The above diagram does not contain the connections between nodes. The reader should assume that leaf nodes in a given PoD is connected to every spine node in that PoD while each spine node (e.g., S1) is connected to all super-spine nodes in the corresponding PoD-interconnect plane (e.g., Plane-1).¶

3. Modifications to Legacy OSPF and ISIS Behaviors

Those OSPF (or ISIS) routers which are configured as non-reflectors for a given OSPF area (or a given ISIS level) SHOULD NOT reflect the LSAs (or LSPs) received from neighbors in that OSPF area (or that ISIS level) to neighbors in the same OSPF area (or the same ISIS level).¶

7. References