Routing Area Working Group G. Mirsky
Internet-Draft Ericsson
Intended status: Informational E. Nordmark
Expires: September 22, 2016 Arista Networks
C. Pignataro
N. Kumar
D. Kumar
Cisco Systems, Inc.
M. Chen
Huawei Technologies
D. Mozes
Mellanox Technologies Ltd.
S. Pallagatti
March 21, 2016

Operations, Administration and Maintenance (OAM) for Overlay Networks: Gap Analysis


This document provides an overview of the Operations, Administration, and Maintenance (OAM) for overlay networks. The OAM toolset includes set of fault management and performance monitoring capabilities (operating in the data plane) that comply with the Overlay OAM Requirements. Insufficient functional coverage of existing OAM protocols also noted in this document. The protocol definitions for each of the Overlay OAM tools to be defined in separate documents.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on September 22, 2016.

Copyright Notice

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

Operations, Administration, and Maintenance (OAM) toolset provides methods for fault management and performance monitoring in each layer of the network, in order to improve their ability to support services with guaranteed and strict Service Level Agreements (SLAs) while reducing operational costs.

[RFC7276] provided detailed analysis of OAM protocols. Since its completion several new protocols that define data plane encapsulation were introduced. That presented both need to re-evaluate existing set of OAM tools and opportunity to build it into set of tools that can be used and re-used for different data plane protocols.

Overlay OAM Requirements define the set of requirements for OAM in Overlay networks. The OAM solution for Overlay networks, developed by the design team, has two objectives:

The Overlay OAM toolset may use some or all of the following OAM protocols designed at IETF:

1.1. Conventions used in this document

1.1.1. Terminology

Term "Overlay OAM" used in this document interchangeably with longer version "set of OAM protocols, methods and tools for Overlay networks".

BFD Bidirectional Forwarding Detection

FM Fault Management

OAM Operations, Administration, and Maintenance

PM Performance Measurement

SLA Service Level Agreement

TWAMP Two-Way Active Measurement Protocol

1.1.2. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

2. Overlay OAM Toolset

2.1. Overlay OAM Fault Management

Protocols that enable Fault Management functions of OAM toolset are comprised of protocols that perform proactive and on-demand defect detection and failure localization.

2.1.1. Proactive Continuity Check and Connectivity Verification

Bidirectional Forwarding Detection (BFD) [RFC5880] is the protocol of choice for proactive Continuity Check and Connectivity Verification [RFC6428]. Proactive CC/CV in BIER

. Bit-Indexed Explicit Replication (BIER) provides the multicast service. For that BFD over multipoint network [I-D.ietf-bfd-multipoint] and [I-D.ietf-bfd-multipoint-active-tail] are the most suitable of BFD family Figure 1 presents IP/UDP format of BFD over BIER in MPLS network.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|                   Label Stack Element                         |
|                   Label Stack Element                         |
|              BIER-MPLS label          |     |1|               |
|0 1 0 1|  Ver  |  Len  |              Entropy                  |
|                BitString  (first 32 bits)                     ~
~                                                               ~
~                BitString  (last 32 bits)                      |
|OAM|     Reserved      | Proto |            BFIR-id            |
~                       IP Header                               ~
|        Source Port            |   Destination Port (3784)     |
|             Length            |           Checksum            |
~                  BFD control packet                           ~

Figure 1: BFD over BIER with IP/UDP format

Proto field MUST be set to IPv4 or IPv6 vlalue. Note that IP Destination address in Figure 1 must follow Section 7 [RFC5884], i.e. ?the destination IP address MUST be randomly chosen from the 127/8 range for IPv4 and from the 0:0:0:0:0:FFFF:7F00/104 range for IPv6.? BFD packets in the reverse direction of the BFD session will be transmitted on IP network to the IP address mapped to the BFIR-id and the destination UDP port number set as source UDP port number of the received BFD packet.

IP/UDP format presents overhead, particularly in case of IPv6 address family. Thus option to avoid use of extra headers for OAM seems attractive. Figure 2 presents G-ACh format of BFD over BIER in MPLS network. Proto field of the BIER header MUST be set to OAM value. BFD control packet follows the BIER OAM header as defined in [I-D.kumarzheng-bier-ping]. According to the Section 3.1 of [I-D.kumarzheng-bier-ping], Ver is set to 1; BFD control packet over multi-point without or with active tail accordingly identified in Message Type Field. The Proto field ?is used to define if there is any data packet immediately following the OAM payload?.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|                   Label Stack Element                         |
|                   Label Stack Element                         |
|              BIER-MPLS label          |     |1|               |
|0 1 0 1|  Ver  |  Len  |              Entropy                  |
|                BitString  (first 32 bits)                     ~
~                                                               ~
~                BitString  (last 32 bits)                      |
|OAM|     Reserved    | Proto |             BFIR-id             |
| Ver | Message Type  | Proto |          Reserved               |
~                  BFD control packet                           ~

Figure 2: BFD over BIER with G-ACh format Proactive CC/CV in NVO3 Proactive CC/CV over SFP

2.1.2. On-demand Continuity Check and Connectivity Verification On-demand CC/CV in BIER

[I-D.kumarzheng-bier-ping] defines format of Echo Request/Reply control packet and set of TLVs that can be used to perform failure detection and isolation in BIER domain over MPLS network. On-demand CC/CV in NVO3 On-demand CC/CV over SFP

2.1.3. Alarm Indication Signal AIS in BIER AIS in NVO3 AIS over SFP

2.2. Overlay OAM Performance Measurement

2.2.1. Overlay OAM PM Active Active PM in BIER Active PM in NVO3 Active PM over SFP

2.2.2. Overlay OAM PM Passive Passive PM in BIER

[I-D.mirsky-bier-pmmm-oam] describes how the Marking Method can be used in BIER domain over MPLS networks. Passive PM in NVO3 Passive PM over SFP

2.3. Telemetry in Overlay OAM

Excessive use of the in-band OAM channel may affect user flow and thus change network behavior. For example, if operator uses passive measurement exporting massive amount of data over the OAM channel may affect network. I think that a management channel should be used in such case. Obviously it may traverse the same nodes and links but may not require the same QoS. We can refer to LMAP Reference Model [RFC7594] with Controller, Measurement Agent and Data Collector.

[I-D.lapukhov-dataplane-probe] proposes transport independent generic telemetry probe structure.

2.4. Conclusions

3. IANA Considerations

This document does not propose any IANA consideration. This section may be removed.

4. Security Considerations

This document list the OAM requirement for BIER-enabled domain and does not raise any security concerns or issues in addition to ones common to networking.

5. Acknowledgement


6. References

6.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.

6.2. Informative References

[I-D.ietf-bfd-multipoint] Katz, D., Ward, D. and J. Networks, "BFD for Multipoint Networks", Internet-Draft draft-ietf-bfd-multipoint-07, August 2015.
[I-D.ietf-bfd-multipoint-active-tail] Katz, D., Ward, D. and J. Networks, "BFD Multipoint Active Tails.", Internet-Draft draft-ietf-bfd-multipoint-active-tail-01, November 2015.
[I-D.ietf-bfd-seamless-base] Akiya, N., Pignataro, C., Ward, D., Bhatia, M. and J. Networks, "Seamless Bidirectional Forwarding Detection (S-BFD)", Internet-Draft draft-ietf-bfd-seamless-base-08, February 2016.
[I-D.ietf-bfd-seamless-ip] Akiya, N., Pignataro, C. and D. Ward, "Seamless Bidirectional Forwarding Detection (S-BFD) for IPv4, IPv6 and MPLS", Internet-Draft draft-ietf-bfd-seamless-ip-03, February 2016.
[I-D.ietf-mpls-rfc6374-udp-return-path] Bryant, S., Sivabalan, S. and S. Soni, "RFC6374 UDP Return Path", Internet-Draft draft-ietf-mpls-rfc6374-udp-return-path-04, August 2015.
[I-D.kumarzheng-bier-ping] Kumar, N., Pignataro, C., Akiya, N., Zheng, L., Chen, M. and G. Mirsky, "BIER Ping and Trace", Internet-Draft draft-kumarzheng-bier-ping-02, December 2015.
[I-D.lapukhov-dataplane-probe] Lapukhov, P., "Data-plane probe for in-band telemetry collection", Internet-Draft draft-lapukhov-dataplane-probe-00, March 2016.
[I-D.mirsky-bier-pmmm-oam] Mirsky, G., Zheng, L., Chen, M. and G. Fioccola, "Performance Measurement (PM) with Marking Method in Bit Index Explicit Replication (BIER) Layer", Internet-Draft draft-mirsky-bier-pmmm-oam-01, March 2016.
[I-D.tempia-ippm-p3m] Capello, A., Cociglio, M., Fioccola, G., Castaldelli, L. and A. Bonda, "A packet based method for passive performance monitoring", Internet-Draft draft-tempia-ippm-p3m-02, October 2015.
[RFC4379] Kompella, K. and G. Swallow, "Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures", RFC 4379, DOI 10.17487/RFC4379, February 2006.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K. and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, DOI 10.17487/RFC5357, October 2008.
[RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010.
[RFC5882] Katz, D. and D. Ward, "Generic Application of Bidirectional Forwarding Detection (BFD)", RFC 5882, DOI 10.17487/RFC5882, June 2010.
[RFC5883] Katz, D. and D. Ward, "Bidirectional Forwarding Detection (BFD) for Multihop Paths", RFC 5883, DOI 10.17487/RFC5883, June 2010.
[RFC5884] Aggarwal, R., Kompella, K., Nadeau, T. and G. Swallow, "Bidirectional Forwarding Detection (BFD) for MPLS Label Switched Paths (LSPs)", RFC 5884, DOI 10.17487/RFC5884, June 2010.
[RFC5885] Nadeau, T. and C. Pignataro, "Bidirectional Forwarding Detection (BFD) for the Pseudowire Virtual Circuit Connectivity Verification (VCCV)", RFC 5885, DOI 10.17487/RFC5885, June 2010.
[RFC6038] Morton, A. and L. Ciavattone, "Two-Way Active Measurement Protocol (TWAMP) Reflect Octets and Symmetrical Size Features", RFC 6038, DOI 10.17487/RFC6038, October 2010.
[RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay Measurement for MPLS Networks", RFC 6374, DOI 10.17487/RFC6374, September 2011.
[RFC6428] Allan, D., Swallow, G. and J. Drake, "Proactive Connectivity Verification, Continuity Check, and Remote Defect Indication for the MPLS Transport Profile", RFC 6428, DOI 10.17487/RFC6428, November 2011.
[RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E. and Y. Weingarten, "An Overview of Operations, Administration, and Maintenance (OAM) Tools", RFC 7276, DOI 10.17487/RFC7276, June 2014.
[RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., Aitken, P. and A. Akhter, "A Framework for Large-Scale Measurement of Broadband Performance (LMAP)", RFC 7594, DOI 10.17487/RFC7594, September 2015.
[RFC7726] Govindan, V., Rajaraman, K., Mirsky, G., Akiya, N. and S. Aldrin, "Clarifying Procedures for Establishing BFD Sessions for MPLS Label Switched Paths (LSPs)", RFC 7726, DOI 10.17487/RFC7726, January 2016.
[RFC7746] Bonica, R., Minei, I., Conn, M., Pacella, D. and L. Tomotaki, "Label Switched Path (LSP) Self-Ping", RFC 7746, DOI 10.17487/RFC7746, January 2016.
[RFC7750] Hedin, J., Mirsky, G. and S. Baillargeon, "Differentiated Service Code Point and Explicit Congestion Notification Monitoring in the Two-Way Active Measurement Protocol (TWAMP)", RFC 7750, DOI 10.17487/RFC7750, February 2016.

Authors' Addresses

Greg Mirsky Ericsson EMail:
Erik Nordmark Arista Networks EMail:
Carlos Pignataro Cisco Systems, Inc. EMail:
Nagendra Kumar Cisco Systems, Inc. EMail:
Deepak Kumar Cisco Systems, Inc. EMail:
Mach Chen Huawei Technologies EMail:
David Mozes Mellanox Technologies Ltd. EMail:
Santosh Pallagatti EMail: