Internet Engineering Task Force S. Jacob, Ed. Internet-Draft K. Tiruveedhula Intended status: Standards Track Juniper Networks Expires: December 2, 2016 May 31, 2016 Benchmarking Methodology for EVPN draft-kishjac-bmwg-evpntest-01 Abstract This document defines the methodologies for benchmarking performance of EVPN. EVPN is defined in RFC 7432. It is being deployed in provider network. This document provides the benchmarking methodologies for EVPN convergance, data plane and control plane learning of mac. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on December 2, 2016. Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Jacob & Tiruveedhula Expires December 2, 2016 [Page 1] Internet-Draft EVPN Benchmarking Methodology May 2016 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4. Test Procedure . . . . . . . . . . . . . . . . . . . . . . . 5 4.1. MAC Learning in Control plane and Data Plane . . . . . . 5 4.2. MAC flush for locally learned and remote learned MAC . . 5 4.3. High Availablity . . . . . . . . . . . . . . . . . . . . 5 4.4. Reliablity . . . . . . . . . . . . . . . . . . . . . . . 5 4.5. Convergence Time . . . . . . . . . . . . . . . . . . . . 5 4.6. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.7. SOAK . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.8. Measurement Statistics . . . . . . . . . . . . . . . . . 6 5. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.1. MAC Learning in Control plane and Data Plane . . . . . . 6 5.1.1. To check the time taken to learn the mac address in DUT . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.1.2. To check the time taken to learn 100,000 routes from remote peer by DUT. . . . . . . . . . . . . . . . . . 7 5.1.3. To check the time taken to flush the local entry due to CE link failure . . . . . . . . . . . . . . . . . 7 5.1.4. To check the time taken by DUT to flush 100,000 routes learned from remote PE after R1 traffic generator link failure . . . . . . . . . . . 8 5.1.5. To measure the mac ageing time. . . . . . . . . . . . 8 5.1.6. To check the time taken by DUT to age 100,000 routes learned from remote PE after stopping the traffic at remote PE. . . . . . . . . . . . . 9 5.2. Convergence . . . . . . . . . . . . . . . . . . . . . . . 9 5.2.1. To check the time taken by DUT to learn 100,000 routes from local and 100,000 from remote and measure the time of flood from DUT. . . . . . . . . . 9 5.2.2. To measure the time taken to elect a new DF by adding a MHPE. . . . . . . . . . . . . . . . . . . 10 5.3. Reliability . . . . . . . . . . . . . . . . . . . . . . . 10 5.3.1. To check the whether there is traffic loss due to routing engine failover for redundancy test. . . . . 10 5.3.2. To check the whether DF election is taking place properly after the one of the MH PE reboot. . . . . 11 5.4. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.4.1. To Scale the DUT to 8k EVI and clear bgp in DUT with out traffic. . . . . . . . . . . . . . . . . . . . . 12 5.4.2. To Scale the DUT to 8k EVI and clear bgp in DUT with traffic. Measure the convergence time . . . . . . . . 12 5.5. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 13 5.5.1. To Scale the DUT to 8k EVI in DUT with traffic and Jacob & Tiruveedhula Expires December 2, 2016 [Page 2] Internet-Draft EVPN Benchmarking Methodology May 2016 run the set up for 24hrs . . . . . . . . . . . . . . 13 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 8. Security Considerations . . . . . . . . . . . . . . . . . . . 13 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 9.1. Normative References . . . . . . . . . . . . . . . . . . 14 9.2. Informative References . . . . . . . . . . . . . . . . . 14 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 1. Introduction EVPN which is defined in RFC7432 which describes procedures for BGP MPLS-based Ethernet VPNs(EVPN). This document defines the methodologies for benchmarking performance of EVPN. EVPN has been implemented with many varying designs in order to achieve their intended network functionality. The scope of this document is to provide methodologies for benchmarking evpn data, control plane mac learning, mac flush, mac ageing, convergence, high availablity, reliablity and scale. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Test Topology Jacob & Tiruveedhula Expires December 2, 2016 [Page 3] Internet-Draft EVPN Benchmarking Methodology May 2016 Topology Diagram | Traffic Generator +----------+ | | | R1 | | | +----------+ | | +----------+ | | | RR | | | +----------+-------------| | | | | +----------+ +-----------+ | | | MHPE2 | | DUT | | | | MHPE1 | | | +----------+ +-----------+ | | +----------+------------ | | | CE | | | +----------+------------ Traffic Generator Topology Diagram Figure 1 3. Network The network consists of 5 routers and 2 traffic generator ports. DUT is acting as one of MH PE to CE. The RR is acting as route reflector and core router. R1 is a Single home router running evpn. All four routers except CE are running mpls, bgp. CE is a dual home connected to DUT and MH PE1. The testing will be done on DUT to bench mark the service. DUT and MHPE2 is running EVPN with SA/AA with CE. The DUT and and other PE's will be running 100 EVI's(EVPN instances) on 100 sub interfaces. Jacob & Tiruveedhula Expires December 2, 2016 [Page 4] Internet-Draft EVPN Benchmarking Methodology May 2016 4. Test Procedure The test defined to bench mark the performance of EVPN mac learning in control pland and data plane, Mac flush, High Availablity, convergence time in link failures, Reliablity and scale scenarios. 4.1. MAC Learning in Control plane and Data Plane The MAC will be learned in data plane and control plane, test is to measure the time taken to learn the "X" number of mac's in time "T". The data plane learning will be from the locally connected interface. The control plane learning is through BGP advertisements from the remote PE. Let the local learning time be "T" and control plane learning time be "T'". 4.2. MAC flush for locally learned and remote learned MAC The time taken to flush the "X" locally learned mac, let it be "T1" once the traffic is stopped. The time taken flush the remote mac which is learned by control plane ("X") macs when the traffic is stopped at remote side.Let the measured time is "T2". 4.3. High Availablity The traffic is flowing bi direction. The bgp is converged, let the "X" numbers of macs learned locally and remotely. Then traffic is flowing at "P" packets per sec. The traffic generator is measuring the Tx and Rx packets, while the routing engine failover there should not any packet loss the router tester must show both "P" packet per seconds. 4.4. Reliablity This is to measure during any events like adding a new PE or rebooting on of the MHPE in the same ethernet segment,there should not any issues in DF election. There should not be any loop. The DF elction must take place in t time where 4>t>3. The DF election must work on V mod N. There should not be any loop in traffic. 4.5. Convergence Time During any events like link failure, hard reset measure the time taken to learn "X" mac's locally and remotely or time taken to learn "X" mac locally and remote. Jacob & Tiruveedhula Expires December 2, 2016 [Page 5] Internet-Draft EVPN Benchmarking Methodology May 2016 4.6. Scale This is to measure the performance of DUT in scaling to "X" EVPN instances. The measured parameters are CPU usage, memory leak,crashes. 4.7. SOAK This is measuring the performance of DUT running with scaled configuration with traffic over a peroid of time "T'". In each interval "t1" the parameters measured are CPU usage, memory usage, crashes. 4.8. Measurement Statistics The test is repeated for "N" times and the value is taken by averaging the values. 5. Test Cases 5.1. MAC Learning in Control plane and Data Plane The following tests are conducted to measure mac learning local and remote. 5.1.1. To check the time taken to learn the mac address in DUT Objective: To check the time taken to learn the mac address locally and time taken to send the locally learned routes to peers. a. Send 100,000 unicast frames from CE to MHPE1(DUT) working in SA mode where DUT is the DF so that it can forward the traffic with different source and destination address,Measure the time taken to learn these mac in forwarding table and control plane. b. Measure the time taken to send these 100,000 type 2 routes from DUT to its peers. Procedure: Configure EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT. Once the bgp comes up check the DUT evpn table. For MH PE ESI must be configured per IFD/Interface. Using RT (traffic generator) to send the traffic to the routers. Measurement The DUT mac table must learn the 100,000 mac address and measure the time taken to send Jacob & Tiruveedhula Expires December 2, 2016 [Page 6] Internet-Draft EVPN Benchmarking Methodology May 2016 100,000 routes to its peers by DUT. Measurement The DUT mac table must learn the 100,000 measure the time taken to learn the 100,000 mac address in forwarding table and time taken to advertise to remote Peer. 5.1.2. To check the time taken to learn 100,000 routes from remote peer by DUT. Objective: Send 100,000 frames with different SA and DA to DUT from R1 using traffic generator. Measure the time taken to learn these 100,000 routes from remote peer and program the mac address table. Procedure: Configure EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table. For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: The DUT mac table must learn the 100,000 measure the time taken to learn the 100,000 mac address from remote peers. 5.1.3. To check the time taken to flush the local entry due to CE link failure Objective: Send 100,000 frames with different SA and DA to DUT from CE using traffic generator. Then wait to learn all 100,000 mac address. Then fail the DUT CE link and measure the time taken to flush these 100,000 routes from the mac table and from control plane. Procedure: Configure EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT. Once the bgp comes up check the DUT evpn table. For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: Jacob & Tiruveedhula Expires December 2, 2016 [Page 7] Internet-Draft EVPN Benchmarking Methodology May 2016 The DUT mac table must learn the 100,000 measure the time taken for flushing these 100,000 mac address. 5.1.4. To check the time taken by DUT to flush 100,000 routes learned from remote PE after R1 traffic generator link failure Objective: Send 100,000 frames with different SA and DA to DUT from R1 using traffic generator. Bring down the link between R1 and traffic generator. Then measure the time taken to flush the DUT mac address table. Procedure: Configure EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are running mpls, bgp, RR is acting as route reflector to R1, MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT (traffic generator) to send the traffic to the routers. Measurement: Measure the time taken to flush 100,000 remote routes from mac table of DUT. 5.1.5. To measure the mac ageing time. Objective: Send 100,000 frames with different SA and DA to DUT from CE using traffic generator. Then wait to learn all 100,000 mac address. Then stop the traffic.Wait to see how long it takes to flush these mac entries due to ageing. Procedure: Configure EVPN EVI in R1, MHPE2, DUT. All 4 routers except CE are running mpls, bgp, RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table. For MH PE ESI must be configured per IFD/Interface. Using RT (traffic generator) to send the traffic to the routers. Measurement: The DUT mac table must learn the 100,000 measure the time taken for flushing these 100,000 mac address due to ageing. Jacob & Tiruveedhula Expires December 2, 2016 [Page 8] Internet-Draft EVPN Benchmarking Methodology May 2016 5.1.6. To check the time taken by DUT to age 100,000 routes learned from remote PE after stopping the traffic at remote PE. Objective: Send 100,000 frames with different SA and DA to DUT from R1 using traffic generator. After stopping the traffic at remote PE R1 traffic generator due to mac ageing it will withdraw its routes from remote PE's. Measure the time taken to remove these macs from DUT mac table. Procedure: Configure EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are running mpls, bgp, RR is acting as route reflector to R1, MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: Measure the time taken to flush 100,000 remote macs from mac table of DUT due to ageing. 5.2. Convergence The following tests are executed to measure the convergence time in case of an event or by lerning the mac without any external trigger. 5.2.1. To check the time taken by DUT to learn 100,000 routes from local and 100,000 from remote and measure the time of flood from DUT. Objective: Send 100,000 frames with different SA and DA to DUT from R1 using traffic generator. Send 100,000 frames with different SA and DA from traffic generator connected to CE. The SA and DA of flows must be complimentary to have unicast flows. Measure the time taken by the DUT to learn 200,000 in mac table and in control plane.Measure the flood time period of DUT. Procedure: Configure EVPN EVI in R1, MHPE2, DUT. All 4 routers except CE are running mpls, bgp, RR is acting as route reflector to R1, MHPE2 and DUT.Once the bgp comes up check the DUT evpn table. For MH PE ESI Jacob & Tiruveedhula Expires December 2, 2016 [Page 9] Internet-Draft EVPN Benchmarking Methodology May 2016 must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: Measure the time taken to learn 200,000 routes in control and mac address table in DUT and measure the flood time of DUT be measuring the time taken to stop the flood to unicast flows. 5.2.2. To measure the time taken to elect a new DF by adding a MHPE. Objective: Send 100,000 frames from CE to DUT from traffic generator with different SA and DA. Wait to learn all 100,000 mac address. Then add R1 to the same Ethernet segment by configuring the same ESI value configured in DUT, MHPE2 in IFD. Then the new DF election take place during that time there should not be any loop and measure the time taken to come up the new DF. Measure the time taken to elect the new DF. Procedure: Configure EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are running mpls, bgp, RR is acting as route reflector to R1, MHPE2 and DUT.Once the bgp comes up check the DUT evpn table. For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: Measure the time taken for new DF election in DUT and there should not be any loop or forwarding the BUM traffic back to the same segment. 5.3. Reliability These tests are conducted to check after an event there wont be any change in functionality. 5.3.1. To check the whether there is traffic loss due to routing engine failover for redundancy test. Objective: Send 100,000 frames from CE to DUT from traffic generator with different SA and DA. Send 100,000 frames from traffic generator to R1 with different SA and DA so that 200,000 mac address will be Jacob & Tiruveedhula Expires December 2, 2016 [Page 10] Internet-Draft EVPN Benchmarking Methodology May 2016 learned in DUT. There is a bi directional traffic flow with 100,000 pps in each direction. Then do a routing engine failover. Procedure: Configure EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: There should not be any traffic loss,No change in the DF role. No withdraw of any routes. 5.3.2. To check the whether DF election is taking place properly after the one of the MH PE reboot. Objective: Send 100,000 frames from CE to DUT from traffic generator with different SA and DA. Send 100,000 frames from traffic generator to R1 with different SA and DA so that 200,000 mac address will be learned in DUT. There is a bi directional traffic flow with 100,000 pps in each direction. Then reboot DUT since there are 2 MH PE's the other PE will become the DF then after DUT comes back, then DF election has to re initiate. Procedure: Configure EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement The DF election has to take place again once the DUT comes back online.There should not be any DF stuck casefor 100 EVI's. 5.4. Scale Jacob & Tiruveedhula Expires December 2, 2016 [Page 11] Internet-Draft EVPN Benchmarking Methodology May 2016 5.4.1. To Scale the DUT to 8k EVI and clear bgp in DUT with out traffic. Objective: The main purpose of the test the DUT performance to scale 8k EVI's. Then clear bgp neighbor. There should not be any loss of routes or any crashes. Procedure: Configure EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement There should not be any loss of route types 1,3 and 4 in DUT. 5.4.2. To Scale the DUT to 8k EVI and clear bgp in DUT with traffic. Measure the convergence time Objective: The main purpose of the test the DUT performance to scale 8k EVI's with traffic. Then clear bgp neighbor. There should not be any loss of routes or any crashes. Send 100,000 frames from CE to DUT from traffic generator with different SA and DA for 5 EVI's. Send 100,000 frames from traffic generator to R1 with different SA and DA for 5 EVI's.so that 1,000,000 mac address will be learned in DUT. There is a bi directional traffic flow with 500,000 pps in each direction. Then clear the bgp nei in DUT after the bgp comes up and started learning the routes, measure the time taken to learn all 1,000,000 mac routes. Procedure: Configure EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT (traffic generator) to send the traffic to the routers. Measurement: Jacob & Tiruveedhula Expires December 2, 2016 [Page 12] Internet-Draft EVPN Benchmarking Methodology May 2016 The DUT must learn all 1,000,000 mac address. Measure the time taken to learn 1,000,000 mac routes. 5.5. Soak Test 5.5.1. To Scale the DUT to 8k EVI in DUT with traffic and run the set up for 24hrs Objective: The main purpose of the test the DUT performance to scale 8k EVI's with traffic. Then clear bgp neighbor. There should not be any loss of routes or any crashes. Send 100,000 frames from CE to DUT from traffic generator with different SA and DA for 5 EVI's. Send 100,000 frames from traffic generator to R1 with different SA and DA for 5 EVI's.so that 1,000,000 mac address will be learned in DUT. There is a bi directional traffic flow with 500,000 pps in each direction.Keep the setup up and running for 24 hrs,take hourly CPU utilization,memory usage. Procedure: Configure EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must be configured per IFD/Interface. Using RT(traffic generator) to send the traffic to the routers. Measurement: Take the hourly reading of CPU,process memory.There should not be any leak,crashes,CPU spikes. 6. Acknowledgements We would like to thank Bhuvaneswaran Vengainathan of Veryx Technologies and Al Morton of (ATT) for their support and encouragements. 7. IANA Considerations This memo includes no request to IANA. 8. Security Considerations There is no additional consideration from RFC 6192. Jacob & Tiruveedhula Expires December 2, 2016 [Page 13] Internet-Draft EVPN Benchmarking Methodology May 2016 9. References 9.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999, . 9.2. Informative References [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . Appendix A. Appendix Authors' Addresses Sudhin Jacob (editor) Juniper Networks Bangalore India Phone: +91 8061212543 Email: sjacob@juniper.net Kishore Tiruveedhula Juniper Networks 10 Technology Park Dr Westford, MA 01886 USA Phone: +1 9785898861 Email: kishoret@juniper.net Jacob & Tiruveedhula Expires December 2, 2016 [Page 14]