Internet-Draft D-PATH for Layer2 EVPN October 2023
Rabadan, et al. Expires 25 April 2024 [Page]
Workgroup:
BESS Workgroup
Internet-Draft:
draft-sr-bess-evpn-dpath-04
Published:
Intended Status:
Standards Track
Expires:
Authors:
J. Rabadan, Ed.
Nokia
S. Sathappan
Nokia
M. Gautam
Nokia
P. Brissette
Cisco Systems
W. Lin
Juniper
B. Wen
Comcast

Domain Path (D-PATH) for Ethernet VPN (EVPN) Interconnect Networks

Abstract

The BGP Domain PATH (D-PATH) attribute is defined for Inter-Subnet Forwarding (ISF) BGP Sub-Address Families that advertise IP prefixes. When used along with EVPN IP Prefix routes or IP-VPN routes, it identifies the domain(s) through which the routes have passed and that information can be used by the receiver BGP speakers to detect routing loops or influence the BGP best path selection. This document extends the use of D-PATH so that it can also be used along with other EVPN route types.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 25 April 2024.

Table of Contents

1. Introduction

The BGP Domain PATH (D-PATH) attribute [I-D.ietf-bess-evpn-ipvpn-interworking] is defined for Inter-Subnet Forwarding (ISF) BGP Sub-Address Families that advertise IP prefixes. When used along with EVPN IP Prefix routes or IP-VPN routes, it identifies the domain(s) through which the routes have passed and that information can be used by the receiver BGP speakers to detect routing loops or influence the BGP best path selection. This document extends the use of D-PATH so that it can also be used along with other EVPN route types.

The D-PATH attribute can be used to prevent control plane loops for EVPN routes, or to provide full path visibility of all the EVPN Interconnect Gateways through which a route has gone and modify the best path selection based on it. Some use cases in which D-PATH can be used along with (non-IP Prefix) EVPN routes follow, but the use cases are not limited to the ones described in this section.

1.1. D-PATH to Prevent Loops for EVPN Routes

Figure 1 illustrates an EVPN Interconnect case where EVPN MAC/IP Advertisement routes can be looped indefinitely. The three Gateways (GW1, GW2 and GW3) and PE1 in the diagram are attached to the same EVPN Broadcast Domain (BD1). However, BD1 is extended throughout three different domains that are interconnected by the Gateways, which follow [RFC9014] procedures. Suppose a host with MAC address M1 is learned on GW1 and GW1 advertises an EVPN MAC/IP Advertisement route for M1 into Domain-1 and Domain-2. When the route gets imported by GW2 and GW3 and later exported into Domain-3, GW2 and GW3 may redistribute each other's route for M1 back into Domain-1 and Domain-2, respectively, creating a loop. D-PATH can be used by the Gateways when redistributing the route between Domains, to identify the Domains through which the route for M1 has gone. When GW1 receives an EVPN MAC/IP Advertisement route for M1 that contains a D-PATH with a domain-id locally assigned, GW1 identifies the route as "looped".

          +----------------+ GW2
          |   EVPN        +-------+
          |   Domain-1    | +---+ |
          |               | |BD1| |---------------+
          |               | +---+ |               |
     GW1  |               +-------+               |    PE1
   +-------+               |     |    EVPN       +-------+
   | +---+ |---------------+     |    Domain-3   | +---+ |
   | |BD1| |                     |               | |BD1| |
   | +---+ |---------------+     |               | +---+ |
   +---|---+               | GW3 |               +---|---+
       |  |   EVPN        +-------+               |  |
   M1--+  |   Domain-2    | +---+ |               |  +--M2
          |               | |BD1| |---------------+
          |               | +---+ |
          |               +-------+
          +----------------+
Figure 1: Loops for EVPN routes

Similar examples are possible with EVPN VPWS services on the Gateways and PEs, where loop prevention for the redistributed A-D per EVI routes is needed. D-PATH provides the end to end path visibility that is required to prevent the loop.

1.2. Add Path Visibility and Influence Best Path Selection for EVPN Routes

Figure 2 illustrates another [RFC9014] EVPN Interconnect case where, in addition to using D-PATH to prevent EVPN MAC/IP Advertisement route loops when redistributing routes between domains, the D-PATH attribute can also influence the best path selection for the routes. For example, if all the Gateways in the diagram are attached to the same BD1, an EVPN MAC/IP Advertisement route for MAC address M1 advertised by GW1 is advertised into Domain-1 and Domain-4. Two routes for M1 will arrive at GW3 with different route distinguishers and BGP Next Hops. If D-PATH is used by all the Gateways, the two routes arriving at GW3 will have a different sequence of domain-ids in the D-PATH attribute. GW3 can use the length of the D-PATH as a way of influencing the selection (i.e., the shortest D-PATH route is selected). D-PATH improves the path visibility of the route since it provides information about all the Domains through which the route has passed.

        +----------+ GW11  +----------+  GW2  +----------+
        | EVPN     +-------+ EVPN     +-------+ EVPN     |
        | Domain-1 | +---+ | Domain-2 | +---+ | Domain-3 |
        |          | |BD1| |          | |BD1| |          |
        |          | +---+ |          | +---+ |          |
   GW1  |          +-------+          +-------+          |  GW3
 +-------+         |       |          |       |         +-------+
 | +---+ |---------+       +----------+       +---------| +---+ |
 | |BD1| |                                              | |BD1| |
 | +---+ |---------+       +----------------------------| +---+ |
 +---|---+         | GW12  |                            +---|---+
     |  | EVPN     +-------+      EVPN                   |  |
 M1--+  | Domain-4 | +---+ |      Domain-5               |  +--M2
        |          | |BD1| |                             |
        |          | +---+ |                             |
        |          +-------+                             |
        +----------+       +-----------------------------+
Figure 2: Influence Best Path Selection for EVPN routes

2. Conventions used in this document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

3. Terminology

This section summarizes the terminology that is used throughout the rest of the document.

4. Use of Domain Path Attribute (D-PATH) with EVPN routes

This document extends the use of the D-PATH attribute specified in [I-D.ietf-bess-evpn-ipvpn-interworking] so that D-PATH can be advertised and processed along with the following EVPN route types:

As discussed, the use of D-PATH with EVPN IP Prefix routes is specified in [I-D.ietf-bess-evpn-ipvpn-interworking]. When used along with EVPN routes other than IP Prefix routes, the D-PATH attribute is characterized as follows:

  1. D-PATH is composed of a sequence of Domain segments following the format specified in [I-D.ietf-bess-evpn-ipvpn-interworking] where each Domain is represented as <DOMAIN-ID:ISF_SAFI_TYPE>. In this specification, DOMAIN-ID is an EVPN Domain identifier configured in an EVPN Domain Gateway and ISF_SAFI_TYPE is set to either 70 (EVPN) or 0 (local route). To simplify the explanation, this document represents the domains for EVPN routes as <Domain-ID:TYPE>.

  2. D-PATH identifies the sequence of EVPN Domains the route has gone through, with the last <Domain-ID:TYPE> entry (rightmost) identifying the first PE or EVPN Domain Gateway that added the D-PATH attribute.

  3. For non-Inter Subnet Forwarding EVPN MAC/IP Advertisement routes or EVPN A-D per EVI routes [I-D.ietf-bess-rfc7432bis], D-PATH SHOULD be added/modified by a EVPN Domain Gateway that redistributes the route between EVPN Domains and MAY be added by a PE or EVPN Domain Gateway that originates the route, as follows:

    1. An EVPN Domain Gateway that connects two EVPN Domains "X" and "Y", and receives a route on a EVPN Domain identified by a Domain-ID "X" SHOULD append a domain <X:EVPN> to the existing (or newly added) D-PATH attribute when redistributing the route to EVPN Domain "Y". The route is redistributed if it is first imported in a MAC-VRF (or VPWS instance), the MAC (or Ethernet Tag) is active, and policy allows the re-export of the route to a BGP neighbor.

    2. An EVPN Domain Gateway MAY also add the D-PATH attribute for locally learned MACs or MAC/IP pairs. In this case, the domain added would be <A:0>, where "A" is the Domain-ID configured on the Gateway's MAC-VRF that is specific to local routes (MAC/IP learned via local AC), and "0" is the TYPE of the EVPN Domain and indicates that the route is locally originated and not redistributed after receiving it from a BGP-EVPN neighbor. The EVPN Domain-ID for local routes MAY be shared by all the EVPN Domain Gateways of the same redundancy group for local routes, or each EVPN Domain Gateway of the redundancy group can use its own Domain-ID.

    3. A PE that is connected to a single EVPN Domain (therefore the PE is not an EVPN Domain Gateway) MAY add the D-PATH with a domain <B:0>, where "B" is the Domain-ID configured on the PE's MAC-VRF (or VPWS) for locally learned MAC/IPs (or Ethernet Tag IDs for VPWS). "0" is the TYPE that indicates the route is not re-advertised, but originated in the PE.

  4. For EVPN Inclusive Multicast Ethernet Tag routes, an EVPN Domain Gateway must not redistribute routes between Domains as specified in [RFC9014]. An EVPN Domain Gateway originates an EVPN Inclusive Multicast Ethernet Tag route per Domain to which the Gateway is attached, so that BUM traffic can be attracted from one Domain to the rest. Therefore, only the above point 3.b. applies to EVPN Domain Gateways. That is, an EVPN Domain Gateway MAY add a <A:0> D-PATH attribute for the Inclusive Multicast Ethernet Tag routes generated for the EVPN Domains, where "A" is the configured local Domain-ID, and "0" is the TYPE that indicates the route is locally originated and not redistributed across EVPN Domains. When two or more EVPN Domain Gateways of the same redundancy group connect two EVPN Domains "X" and "Y" and D-PATH is used for EVPN Inclusive Multicast Ethernet Tag routes, it is RECOMMENDED to add the D-PATH attribute with the same local Domain-ID and only on "X" or "Y" but not on both Domains. In this case, all Gateways under the same redundancy group MUST choose the same Domain in which the D-PATH attribute is used.

  5. On received EVPN routes, D-PATH is processed and used for loop detection (Section 4.4) as well as to influence the best path selection (Section 4.1, Section 4.2 and Section 4.3).

4.1. D-PATH and Best Path Selection for MAC/IP Advertisement routes

When two (or more) MAC/IP Advertisement routes with the same route key (and same or different RDs) are received, a best path selection algorithm is used to select and install only one route. The best path selection for MAC/IP Advertisement routes is specified in [I-D.ietf-bess-rfc7432bis], in section 7.13.1, and this document modifies the algorithm by including the D-PATH comparison across EVPN MAC/IP Advertisement routes after tie-breaking rule 5 in [I-D.ietf-bess-rfc7432bis] section 7.13.1, which removes from consideration routes that are not tied for higher degree of preference.

If none of the tie-breaking rules up to (and including) rule 5 produces a single route, the router compares the D-PATH attribute in the remaining candidate routes:

  1. The routes with the shortest D-PATH are preferred, hence routes not tied for the shortest D-PATH are removed. Routes without D-PATH are considered zero-length D-PATH.

  2. Then routes with the numerically lowest left-most Domain-ID are preferred, hence routes not tied for the numerically lowest left-most Domain-ID are removed from consideration.

If the steps above do not produce a single route, then the rest of the rules in [I-D.ietf-bess-rfc7432bis] follow.

4.2. D-PATH and Best Path Selection for Ethernet A-D per EVI routes

When two (or more) EVPN A-D per EVI routes with the same route key (and same or different RDs) are received for a Virtual Private Wire Service (VPWS), a best path selection algorithm is used. The best path selection for EVPN A-D per EVI routes is specified in [I-D.ietf-bess-rfc7432bis], in section 7.13.2, and this document modifies the algorithm by including the D-PATH comparison across EVPN A-D per EVI routes in the same way Section 4.1 does it for EVPN MAC/IP Advertisement routes. That is, rules 1 and 2 of Section 4.1 are interleaved between rules 5 and 6 of [I-D.ietf-bess-rfc7432bis].

4.3. D-PATH and Best Path Selection for Inclusive Multicast Ethernet Tag routes

When two (or more) EVPN Inclusive Multicast Ethernet Tag routes with the same route key (and same or different RDs) are received for a MAC-VRF, a best path selection algorithm is used and only one of them is programmed. The selection algorithm follows [I-D.ietf-bess-rfc7432bis] the same D-PATH comparison steps as in Section 4.1 interleaved between rules 5 and 6 of [I-D.ietf-bess-rfc7432bis].

4.4. Loop Detection

An EVPN route received by a PE with a D-PATH attribute that contains one or more of its locally associated Domain-IDs for the MAC-VRF or VPWS instance is considered to be a looped route. A looped route MUST NOT be redistributed to a different domain and SHOULD be flagged as "looped".

EVPN Inclusive Multicast Ethernet Tag looped routes MUST NOT be installed, where "install" in this document means "create forwarding state". An EVPN MAC/IP Advertisement looped route or an A-D per EVI looped route (in EVPN VPWS services) MAY be installed if selected as the best route.

For instance, in the example of Figure 3, assuming PE1 advertises M1's MAC/IP and does not add the D-PATH attribute, the EVPN Domain Gateway GW1 receives two MAC/IP Advertisement routes for M1's MAC/IP:

  • A MAC/IP Advertisement route with next hop PE1 and no D-PATH.

  • A MAC/IP Advertisement route with next hop GW2 and D-PATH={length=1,<6500:1:EVPN>}, assuming that the Domain-ID for EVPN Domain 1 is 6500:1.

In this case, EVPN Domain Gateway GW1 flags the MAC/IP Advertisement route with D-PATH as "looped", and does not install the MAC in the BD, and does not redistribute the route back to EVPN Domain 1 (since the route includes one of GW1's Domain-IDs). In case the MAC/IP Advertisement route with next hop PE1 is withdrawn, GW1 may install the route with next hop GW2 and D-PATH <6500:1:EVPN>; this may help speed up convergence in case of failures.

The procedures described in this section, based on D-PATH, can be used along with the Ethernet Segment Identifier of the received routes as a way to detect looped routes on EVPN domain gateways attached to an Interconnect Ethernet Segment as in [RFC9014]. An EVPN domain gateway MUST NOT redistribute a received EVPN MAC/IP route or EVPN A-D per EVI route with an Ethernet Segment Identifier value that matches the value of a local Ethernet Segment, irrespective of the D-PATH Domain-IDs.

4.5. Error Handling

The error handling for the D-PATH attribute is described in [I-D.ietf-bess-evpn-ipvpn-interworking]. This document extends the use of D-PATH to non-Inter Subnet Forwarding (non-ISF) EVPN routes.

5. Use-Case Examples

This section illustrates the use of D-PATH in EVPN routes with examples.

Figure 4 and Figure 5 illustrate an integrated interconnect solution for an EVPN BD, as described in section 4.4 and section 4.6 of [RFC9014]. GW1 and GW2 are EVPN Domain Gateways connecting two EVPN Domains identified by D-PATH domain {1:1:EVPN} and {1:2:EVPN}, respectively. Received Ethernet A-D routes, ES routes, and Inclusive Multicast routes from the routers in one EVPN Domain are consumed and processed by GW1 and GW2, but not redistributed to the other EVPN Domain. However, MAC/IP Advertisement routes received by GW1 and GW2 in one EVPN Domain are processed and, if installed, redistributed into the other EVPN Domain.

         +----EVPN Domain-1---+      +----EVPN Domain-2--+
         |     1:1:EVPN       | GW1  |    1:2:EVPN       |
         |                   +---------+                 |
         |                   | +-----+ |                 |
         |                   | | BD1 | X <-+             |
        PE1                  | +-----+ |   |            PE2
     +---------+             +---------+   |         +---------+
     | +-----+ |              |      |     |         | +-----+ |
M1-----| BD1 | |              |      |     |         | | BD1 |-----M2
     | +-----+ |  ------->    |      |     |         | +-----+ |
     +---------+ (RT2)M1/IP1  |      |     |         +---------+
         |                   +---------+   |             |
         |                   | +-----+ |   |(RT2)M1/IP1  |
         |                   | | BD1 | | --+ <1:1:EVPN>  |
         |                   | +-----+ |                 |
         |                   +---------+                 |
         |                     | GW2 |                   |
         +---------------------+     +-------------------+
Figure 4: EVPN Interconnect

Consider the example of Figure 4, where PE1 advertises a MAC/IP Advertisement route for M1/IP1. The route is processed and installed by GW1 and GW2 in BD1, and both redistribute the routes into the EVPN Domain-2. By using D-PATH in GW1 and GW2, when the route is received on PE2, PE2 has the visibility of the EVPN Domains through which the route has gone, and can also use the D-PATH for best path selection. In addition, GW1 and GW2 can compare the D-PATH of the incoming routes with their local list of EVPN Domain-IDs, and detect looped routes if any of the local EVPN Domain-IDs matches a domain in the received D-PATH. This procedure prevents the redistribution of the route back into EVPN Domain-1. For example, when GW1 receives the MAC/IP Advertisement route for M1/IP1 with D-PATH <1:1:EVPN>, GW1 identifies the route as looped and it does not redistribute it back to Domain-1. The M1/IP1 route with Next Hop PE1 is installed. If M1/IP1 with Next Hop PE1 is withdrawn, GW1 MAY install the route M1/IP1 with Next Hop GW2, as specified in Section 4.4.

The example of Figure 5 illustrates how GW1 and GW2 can also have local ACs in BD1 and learn local MAC (or MAC/IP) addresses from devices connected to the ACs.

         +----EVPN Domain-1---+      +----EVPN Domain-2--+
         |     1:1:EVPN       | GW1  |    1:2:EVPN       |
         |                   +---------+                 |
         |                   | +-----+ |                 |
         |              +-->X| | BD1 | |X<--+            |
        PE1             |    | +-----+ |    |           PE2
     +---------+        |    +---------+    |        +---------+
     | +-----+ |        |     |      |      |        | +-----+ |
M1-----| BD1 | |        |     |      |      +--->    | | BD1 |-----M2
     | +-----+ |        |     |      |      |        | +-----+ |
     +---------+        |     |      |      |        +---------+
         |              |     | GW2  |      |            |
         |          <---+--  +---------+ (RT2)M3/IP3     |
         |       (RT2)M3/IP3 | +-----+ |  {1:3:0}        |
         |        {1:3:0}    | | BD1 | |    |            |
         |                   | +-----+ | ---+            |
         |                   +----|----+                 |
         |                     |  |  |                   |
         +---------------------+  |  +-------------------+
                                  +
                                  M3
Figure 5: EVPN Interconnect local AC

Assuming GW2 learns M3/IP3 via local AC, GW2 advertises a MAC/IP Advertisement route for M3/IP3 into each of the EVPN Domains that GW2 is connected to. As described in Section 4, GW2 can advertise these two MAC/IP Advertisement routes with a configured EVPN Domain-ID for local MAC/IPs routes that can be shared with GW1. Consider this EVPN Domain-ID is 1:3 and it is configured on both, GW1 and GW2. When GW2 advertises the route into each EVPN Domain, it adds the D-PATH attribute with a domain {1:3:0}. These routes are flagged by GW1 as "looped" since 1:3 is configured as a local EVPN Domain-ID in GW1. In addition, PE1 and PE2 receive the routes with the D-PATH and they have the visibility of the origin of the routes, in this case local EVPN Domain Gateway routes. This information can be used to influence the best path selection in case of multiple routes for M3/IP3 are received on PE1 or PE2 for BD1.

As an alternative solution to configuring the same EVPN Domain-ID for local routes on both EVPN Domain Gateways, GW2 can be configured with EVPN Domain-ID 1:3 for local routes, and GW1 can use a different EVPN Domain-ID, e.g., 1:4. In this case, GW2 advertises the route for M3/IP3 into each EVPN Domain as before, but now GW1 does not flag the route as "looped" since 1:3 is not on the list of GW1's local EVPN Domain-IDs. GW1 receives the routes from both EVPN Domains, and GW1 selects the route from e.g., EVPN Domain-1. GW1 then installs the route in its BD and redistributes the route into EVPN Domain-2 with D-PATH {1:1:EVPN, 1:3:0}. When PE2 receives two routes for M3/IP3, one from GW2 with D-PATH {1:3:0} and another from GW1 with D-PATH {1:1:EVPN, 1:3:0}, PE2 uses best path selection and choose to send its traffic to GW2. Also GW2 receives the route for M3/IP3 from GW1 and mark it as "looped" since that route conveys its own EVPN Domain-IDs 1:1 and 1:3.

In a nutshell, the use of D-PATH in MAC/IP Advertisement routes helps prevent loops and influences the best path selection so that PEs choose the shortest paths to the destination PEs.

6. Security Considerations

Most of the considerations included in [I-D.ietf-bess-evpn-ipvpn-interworking] apply to this document.

7. IANA Considerations

None.

8. Acknowledgments

9. Contributors

In addition to the authors included on the front page, the following people contributed significantly:

Vinod Prabhu, Nokia

10. References

10.1. Normative References

[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
[I-D.ietf-bess-evpn-ipvpn-interworking]
Rabadan, J., Sajassi, A., Rosen, E. C., Drake, J., Lin, W., Uttaro, J., and A. Simpson, "EVPN Interworking with IPVPN", Work in Progress, Internet-Draft, draft-ietf-bess-evpn-ipvpn-interworking-09, , <https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-ipvpn-interworking-09>.
[I-D.ietf-bess-rfc7432bis]
Sajassi, A., Burdet, L. A., Drake, J., and J. Rabadan, "BGP MPLS-Based Ethernet VPN", Work in Progress, Internet-Draft, draft-ietf-bess-rfc7432bis-07, , <https://datatracker.ietf.org/doc/html/draft-ietf-bess-rfc7432bis-07>.
[RFC9014]
Rabadan, J., Ed., Sathappan, S., Henderickx, W., Sajassi, A., and J. Drake, "Interconnect Solution for Ethernet VPN (EVPN) Overlay Networks", RFC 9014, DOI 10.17487/RFC9014, , <https://www.rfc-editor.org/info/rfc9014>.
[RFC8214]
Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. Rabadan, "Virtual Private Wire Service Support in Ethernet VPN", RFC 8214, DOI 10.17487/RFC8214, , <https://www.rfc-editor.org/info/rfc8214>.

10.2. Informative References

[RFC9135]
Sajassi, A., Salam, S., Thoria, S., Drake, J., and J. Rabadan, "Integrated Routing and Bridging in Ethernet VPN (EVPN)", RFC 9135, DOI 10.17487/RFC9135, , <https://www.rfc-editor.org/info/rfc9135>.

Authors' Addresses

J. Rabadan (editor)
Nokia
520 Almanor Avenue
Sunnyvale, CA 94085
United States of America
S. Sathappan
Nokia
520 Almanor Avenue
Sunnyvale, CA 94085
United States of America
M. Gautam
Nokia
520 Almanor Avenue
Sunnyvale, CA 94085
United States of America
P. Brissette
Cisco Systems
Canada
W. Lin
Juniper
United States of America
B. Wen
Comcast
United States of America