Internet Engineering Task Force Q. Wang, Ed.
Internet-Draft Y. Zhang
Intended status: Informational ZTE
Expires: August 11, 2017 R. Valiveti
I. Hussain, Ed.
R. Rao
Infinera Corp
H. Helvoort
Hai Gaoming B.V
February 7, 2017

GMPLS Routing and Signaling Framework for B100G


The latest revision of G.709 introduces support for OTU links with rates larger than 100G. This document provides a framework to address the GMPLS routing and signalling extensions that enable GMPLS to setup paths through network that contain these newly introduced OTUCn links.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on August 11, 2017.

Copyright Notice

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

The current GMPLS routing [RFC7138] and signaling extensions [RFC7139] includes coverage for all the OTN capabilities that were defined in the 2012 version of G.709 [ITU-T_G709_2012]. The 2012 version of the G.709 included support for the following capabilities:

  1. Introduction of ODU0
  2. Mapping of 100BASE-X client (1GE) and other sub-1.25G client signals into ODU0
  3. Mapping of FC-1200 into ODU2e
  4. Mappings for 100GBASE-R and 40GBASE-R Ethernet client signals.
  5. OTU4 layer with a rate of 100G.
  6. Support for 1.25G tributary slots in OPU2, OPU3, OPU4 -- to fully support the newly introduced ODU0 signal.
  7. Support for multi-lane interfaces for OTU3, and OTU4 signals

The 2016 version of G.709 [ITU-T_G709_2012] introduces support for higher rate OTU signals, termed OTUCn (which have a nominal rate of 100n Gbps). The newly introduced OTUCn represent a very powerful extension to the OTN capabilities, and one which naturally scales to transport any newer clients with bit rates in excess of 100G, as they are introduced.

This document presents an overview of the changes introduced in [ITU-T_G709_2016] and analyzes them to identify the extensions that would be required in GMPLS routing and signaling to enable the new OTN capabilities. In an OTN network as defined by [ITU-T_G709_2016] and [ITU-T_G798], two layers can be switched at the intermediate nodes: (a) ODU (digital switching), (b) OCh/Optical Tributary Signal (OTSi) (optical switching). This document focuses on the GMPLS extensions that are necessary to support ODU switching in networks that include the beyond 100G OTU links.

1.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

2. Terminology

  1. OPUCn: Optical Payload Unit -Cn. This signal can be seen as the interleaving of n OPUC "slices".
  2. ODUCn: Optical Data Unit - Cn. Like the OPUCn, this signal can be seen as the interleaving of n slices of ODUC signals.
  3. OTUCn: Fully standardized Optical Transport Unit - Cn. This signal can be viewed as being formed as a result of multiplexing n OTUC "slices". An OTUCn has a bandwidth of (approximately) nx100G. An OTUCn signal has n OTUC/ODUC/OPUC overhead instances.
  4. OTUCn-M: This signal is an extension of the OTUCn signal introduced above. This signal contains the same amount of overhead as the OTUCn signal, but contains a reduced amount of payload area. Specifically the payload area consists of M 5G tributary slots (where M is strictly less than 20*n).
  5. GMP: Generic Mapping Procedure. This procedure allows a uniform asynchronous mapping procedure for a adapting a client signal to a server layer. This generic mapping procedure computes the population of stuff bytes for all client/server signal rates. Specifically this procedure is used to map client signals into ODU(s), and Low-Order ODUs into High-Order ODUs.
  6. PSI: OPU Payload structure Indicator. This is a multi-frame message and describes the composition of the OPU signal. This structure includes a field called Payload Type (PT) whose values indicates whether the OPU payload area has been formed by (a) mapping a single non-OTN client (b) multiplexing LO-ODUs. The MSI field is a substructure of the PSI structure, and contains information about the ODU mix contained in a HO-ODU. For mappings of type (b), the following PT codepoints are defined:
    1. 0x20: indicates 2.5G time slots (with AMP)
    2. 0x21: indicates 1.25G tributary slots (with GMP).
    3. 0x22: (introduced in [ITU-T_G709_2016] is used for ODUCn entities, and implies a tributary slot granularity of 5G (with GMP).

  7. MSI: Multiplex Structure Indicator. This structure indicates the grouping of the tributary slots in an OPU payload area to realize a client signal that is multiplexed into an OPU. The individual clients multiplexed into the OPU payload area are distinguished by the Tributary Port number (TPN).
  8. FlexO lane: Refers to an electrical/optical lane of a FlexO interface, used to carry OTUC transport signals.
  9. FlexO group: Refers to the group of m * FlexO interfaces.
  10. AMP: Asynchronous Mapping Procedure
  11. BMP: Bitsynchronous Mapping Procedure
  12. GMP: Generic Mapping Procedure

3. Overview of B100G links in G.709

3.1. The OTUCn signal

In G.709 [ITU-T_G709_2012], the standard mechanism for transporting a client signal is to first map it into an ODU signal (of the appropriate rate), and then switch the resulting ODU signal through the OTN network. In the course of its traversal through the OTN network, the ODU signal generated by the mapper is either (a) multiplexed into higher-order ODU, and then encapsulated to form an OTU or (b) directly encapsulated into an OTU signal that defines the section layer. The option (b), i.e. direct encapsulation into an OTU was possible only for ODU1/ODU2/ODU3/ODU4; ODU signals with other rates (e.g. ODUflex) would first have to be processed per option (a) above. The term "client signal" is generic in the sense that it encompasses both Constant Bit rate (CBR) clients (e.g. 10GBASE-R, SONET OC-768), or packet traffic -- where the goal is to transfer the payload from end-to-end (without regard for bit transparency at the PCS layer). Given that OTU4 was the highest rate section layer signal supported in [ITU-T_G709_2012], the client signal rates were limited to be less than 100G (if ODU-VCAT was not used).

With the emergence of client signals with rates greater than 100Gbps (e.g. 200GBASE-R, 400GBASE-R Ethernet), aggregate signals such as FlexE ([OIF_FLEXE_1.0], and the availability of NPUs which can source packet traffic of n*100G, it becomes necessary for the OTN to transport these signals. This means that the OTN must be capable of creating, and switching ODU entities with rates in excess of n*100G. Traditionally, the ITU-T has introduced OTUx/ODUx signals in G.709 as and when new client signals with higher rates are defined by other standards bodies (e.g. IEEE). Rather than follow the traditional trajectory, [ITU-T_G709_2016] takes a general and scalable approach to decoupling the rates of OTU signals from the client rate evolution. The new OTU signal is called OTUCn; this signal is defined to have a rate of (approximately) n*100G. The following are the key characteristics of the OTUCn signal:

  1. Unlike the signals OTU1..OTU4, the OTUCn signals are not realized by keeping the frame format intact and increasing the frame rate.
  2. The OTUCn signal contains one ODUCn, which in turn contains one OPUCn signal. The OTUCn and ODUCn signals serve as section layer entities.
  3. The OTUCn signals can be viewed as formed by interleaving n OTUC signals (where are labeled 1, 2, ..., n), each of which has the format of a standard OTUk signal without the FEC columns (per [ITU-T_G709_2016]:Figure 7-1). The ODUCn, and OPUCn have a similar structure, i.e. they can be seen as being formed by interleaving n instances of ODUC, OPUC signals (respectively) The OTUC signal contains the ODUC, and OPUC signals, just as in the case of fixed rate OTUs defined in G.709 [ITU-T_G709_2016].
  4. Each of the OTUC "slices" have the same overhead (OH) as the standard OTUk signal in G.709 [ITU-T_G709_2016]. The combined signal OTUCn has n instances of OTUC OH, ODUC OH, and OPUC OH.
  5. The OTUC signal has a slightly highly rate compared to the OTU4 signal (without FEC); this is to ensure that the OPUC payload area can carry an ODU4 signal.

3.1.1. Carrying OTUCn signal between 3R points

As explained above, within G.709 [ITU-T_G709_2016], the OTUCn, ODUCn and OPUCn signal structures are presented in an interface independent manner, by means of n OTUC, ODUC and OPUC instances that are marked #1 to #n. Specifically, the definition of the OTUCn signal does not cover aspects such as FEC, modulation formats, etc. These details are defined as part of the adaptation of the OTUCn layer to the optical layer(s). The specific interleaving of OTUC/ODUC/OPUC signals onto the optical signals is interface specific and specified for OTN interfaces with standardized application codes in the interface specific recommendations (G.709.x).

The original working assumption was that the first B100G inter-domain interface for an OTUC4 would use the optical modules developed for the 400GbE signal. This assumption has been revised as a result of new insights into how the notions developed for FlexE can be applied to the OTN domain. The new developments make it possible to support OTUC4 signals without having to wait for the 400GbE optical modules. The main motivation for developing the interoperable FlexO interfaces is to (a) reuse already existing optical modules developed for carrying Ethernet signals and (b) realize higher rate OTUCn interfaces by bonding the required number of available PHY(s) -- thereby decoupling the rates of OTUCn interfaces from the rates of the available Ethernet PHY(s) . The FlexO layer can be viewed as an encapsulation layer for the OTUCn signal.

Recommendation [ITU-T_G709.1] specifies a flexible interoperable short-reach OTN interface over which an OTUCn (n >=1) is transferred, using bonded FlexO interfaces which belong to a FlexO group. Conceptually, the FlexO can be seen as the adaptation of the idea of FlexE [OIF_FLEXE_1.0] to OTN signals. Like the FlexE group, the FlexO group supports physical interface bonding, the management of the group members, overhead for communication between FlexO peers etc. (these overheads are separate from the GCC0 channel defined over OTUCn). In its current form, Recommendation [ITU-T_G709.1] is limited to the case of transporting OTUCn signals using n 100G Ethernet PHY(s). When the PHY(s) for the emerging set of Ethernet signals, e.g. 200GbE and 400GbE, become available, new recommendations can define the required adaptations.

The (high-level) sequence of steps performed at the FlexO/OTUCn adaptation source are the following:

  1. one OTUCn is split into n instances of OTUC at the FlexO source node.
  2. One or more OTUC instances are associated with one FlexO interface (which could have a rate of p*100G. As of this document's writing, Ethernet PHY(s) exist for transporting 100GBASE-R signals (i.e. p=1). This is the basis for the FlexO interface specified in [ITU-T_G709.1]. For this specific instance, the mapping between OTUC, and the FlexO interface is 1:1, and the mapping is illustrated in Figure 1. Figure 2 illustrates the scenario in which OTUCn transport makes use of 200GbE PHY(s).
  3. The contents of the selected subset of OTUC signals are mapped to FlexO frames directed at one of the FlexO interfaces in the FlexO group.
  4. Alignment markers are added to these FlexO frames so that the resulting stream can be transported across multiple physical/electrical lanes. The standard IEEE FEC used in conjuction with the appropriate Ethernet signal(e.g. 100GbE, or 200GbE) is also added to the frames.

The sink performs the reverse sequence of operations and reconstructs the OTUCn signal. As a result of the direct encapsulation of the OTUCn signal into the FlexO layer, full transparency for the OTUCn layer is guaranteed. Once the OTUCn signal is transported between 3R regeneration points, all B100G capabilities -- such as the support for ODUs with rates higher than 100G, and client signals larger than 100G are enabled.


   |          OTUCn                   |    OTUCn signal
       |           |              |
       |           |     ....     |
   +-------+   +-------+      +-------+    n X OTUC instances
   | OTUC  |   | OTUC  |      | OTUC  |
   +-------+   +-------+      +-------+
       |           |              |
       |           |              |
   +-------+   +-------+      +-------+    n FlexO interfaces
   | FlexO |   | FlexO |      | FlexO |
   | Frame |   | Frame |      | Frame |
   +-------+   +-------+      +-------+
       |           |              |
+---------------------------------------->  Electrical lanes
       |           |              |
       |           |              |
   +-------+   +-------+      +-------+    n 100GbE Eth PHY(s)
   | 100GE |   | 100GE |      | 100GE |
   | PHY   |   | PHY   |      | PHY   |
   +-------+   +-------+      +-------+


Figure 1: OTUCn transport using 100GbE PHY(s)


   |          OTUCn                   |    OTUCn signal
       |           |              |
       |           |     ....     |
   +---+---+   +---+---+      +---+---+    n X OTUC instances
   | OTUCx2|   | OTUCx2|      | OTUCx2|    n/2 groups, 2 OTUC/group
   +---+---+   +---+---+      +---+---+
       |           |              |
       |           |              |
   +---+---+   +---+---+      +---+---+    n/2  FlexO interfaces
   | FlexO |   | FlexO |      | FlexO |
   | Frame |   | Frame |      | Frame |
   +---+---+   +---+---+      +---+---+
       |           |              |
+---------------------------------------->  Electrical lanes
       |           |              |
       |           |              |
   +---+---+   +---+---+      +---+---+    n/2 200GbE Eth PHY(s)
   | 200GE |   | 200GE |      | 200GE |
   | PHY   |   | PHY   |      | PHY   |
   +-------+   +-------+      +-------+


Figure 2: OTUCn transport with 200G PHY(s)

3.2. The OTUCn-M signal

The standard OTUCn signal has the same rate as that of the ODUCn signal as captured in Table 1. This implies that the OTUCn signal can only be transported over wavelengths which have a capacity of multiples of (approximately) 100G. Modern DSPs support a variety of bit rates per wavelength, depending on the reach requirements for the optical link. With this in mind, ITU-T supports the notion of a reduced rate OTUCn signal, termed the OTUCn-M. The OTUCn-M signal is derived from the OTUCn signal by retaining all the n slices of overhead (one per OTUC slice) and trimming the OPUC tributary slots marked as "unavailable". This operation is equivalent to that referred to as "crunching" in the context of FlexE PHY(s).

3.3. The ODUCn signal

The ODUCn signal [ITU-T_G709_2016] can be viewed as being formed by the appropriate interleaving of content from n ODUC frames. The ODUC frames have the same structure as a standard ODU -- in the sense that it has the same Overhead (OH), and the payload area -- but has a higher rate since its payload area can embed an ODU4 signal. The ODUCn is meant to be used as a high-order signal only -- implying that only other lower-rate (i.e. low-order) ODUs can be multiplexed into an ODUCn signal; in other words, no client signals can be directly mapped to an ODUCn signal. The ODUCn signals have a rate that is captured in Table 1.

ODUCn rates
ODU Type ODU Bit Rate
ODUCn n x 239/226 x 99,532,800 kbit/s

Not all values of 'n' may be standardized by ITU-T-T.

The ODUCn is a higher-order ODU signal, and is encapsulated into an OTUCn signal which occupies the section layer. In most common scenarios, the ODUCn, and OTUCn signals will be co-terminous, i.e. they will have identical source/sink locations. [ITU-T_G709_2016] and [ITU-T_G872] allow for the ODUCn signal to pass through a digital regenerator node which will terminate the OTUCn layer, but will pass the regenerated (but otherwise untouched) ODUCn towards a different OTUCn interface where a fresh OTUCn layer will be initiated. Specifically, the OPUCn signal flows through these regenerators unchanged. That is, the set of client signals, their TPNs, trib-slot allocation remains unchanged. Note however that the ODUCn Overhead (OH) might be modified if TCM sub-layers are instantiated in order to monitor the performance of the repeater hops. In this sense, the ODUCn should not be seen as a general, switchable ODU.

3.4. OPUCn Time Slot Granularity

[ITU-T_G709_2012] introduced the support for 1.25G granular tributary slots in OPU2, OPU3, and OPU4 signals. With the introduction of higher rate signals such as the OPUCn (which are formed by interleaving n OPUC signals), it is no longer practical for the optical networks (and the datapath hardware) to support a very large number of flows at such a fine granularity. ITU-T has defined the OPUC with a tributary slot granularity of 5G. This choice is motivated by the following reasons:

  1. Low bitrate flows will usually be aggregated into higher-order ODUs before they are transported over the core of the transport network, and as a consequence, the network is not expected to see a large number of small bitrate flows such as ODU0.
  2. The IEEE is planning to define new MAC rates such as 25Gbps. The choice of 5G TS for OPUC nicely accomodates 25GE clients, without wasting a large amount of bandwidth
  3. The OIF FlexE Implementation agreement also defines the FlexE PHY calendar slots to have a bandwidth of 5G; the OPUC granularity perfectly matches the capacity of the finest FlexE client.

3.5. Structure of OPUCn MSI

An OPUCn signal can be viewed as being formed from an interleaving of n OPUC signals. Each of the OPUC "slices" has a format that is very similar to that OPU4 -- albeit with a slightly higher rate (since an ODU4 can be fully embedded in the payload area of the OPUC signal). As mentioned above, the OPUC signal has 20 tributary slots, each with a bandwidth of 5G. The PSI structure for an OPUCn signal can be viewed as the concatenation of n PSI structures (one per OPUC). The PSI structure of an OPUC includes the following fields:

  1. the Payload Type (PT) - 1byte - with a value of 0x22. This indicates that ODUC has been formed by multiplexing zero or more low-order ODUs into OPUC.
  2. Reserved Field - 1 byte. In ODUs other than ODUC (e.g. ODU0/1/2/3/4/flex), this byte carries the "Client Signal Failure Indication". This field is unused in the case of ODUC entities since no non-OTN client signal is directly mapped to these server layers.
  3. The MSI field (of size 40 bytes) which contains the information about 20 tributary slots; each such information structure occupies 2 bytes and has the following format (G.709:Section 20.4.1 [ITU-T_G709_2016]):

OPUC MSI information (for each tributary slot
Bit Position # (Bit 1= MSB; Bit 16 = LSB) Description
1 The TS availability bit 1 indicates if the tributary slot is available or unavailable
9 The TS occupation bit 9 indicates if the tributary slot is allocated or unallocated
2-8, 10-16 The tributary port number (TPN) of the client signal that is being transported in this TS. A given client uses the same TPN value in all the TS(s) that are being used to transport the client signal. ODTUCn.ts tributary ports are numbered 1 to 10n. The current 14-bit field for the TPN will allow the index 'n' to grow as large as 1638 -- which is sufficient for all conceivable OTUCn links.

3.6. Client Signal Mappings

Note that [ITU-T_G709_2016] introduces support for OTUCn signals with rates of n*100G and also introduces support for client signals with rates larger than 100G (e.g. the future 400GBASE-R client being standardized by IEEE, higher packet streams from NPUs). The approach taken by the ITU-T to map non-OTN client signals to the appropriate ODU containers is as follows:

  1. All client signals with rates less than 100G are mapped as specified in [ITU-T_G709_2016]:Clause 17. These mappings are identical to those specified in the earlier revision of G.709 [ITU-T_G709_2012]. Thus, for example, the 1000BASE-X/10GBASE-R signals are mapped to ODU0/ODU2e respectively (see Table 3 -- based on Table 7-2 in [ITU-T_G709_2016])
  2. Always map the new and emerging client signals to ODUflex signals of the appropriate rates (see Table 3 -- based on Table 7-2 in [ITU-T_G709_2016])
  3. Drop support for ODU Virtual Concatenation. This simplifies the network, and the supporting hardware since multiple different mappings for the same client are no longer necessary.
  4. ODUflex signals are low-order signals only. If the ODUflex entities have rates of 100G or less, they can be transported using either an ODUk (k=1..4) or an ODUCn server layer. On the other hand, ODUflex connections with rates greater than 100G will require the server layer to be ODUCn. The ODUCn signals must be adapted to an OTUCn signal. Figure 3 illstrates the hierarchy of the digital signals defined in G.709; this figure does not illustrate the handed off to the optical layers.

Types and rates of ODUs usable for client mappings
ODU Type ODU Bit Rate
ODU0 1,244,160 Kbps
ODU1 239/238 x 2,488,320 Kbps
ODU2 239/237 x 9,953,280 Kbps
ODU2e 239/237 x 10,312,500 Kbps
ODU3 239/236 x 39,813,120 Kbps
ODU4 239/227 x 99,532,800 Kbps
ODUflex for CBR client signals 239/238 x Client signal Bit rate
ODUflex for GFP-F mapped packet traffic Configured bit rate
ODUflex for IMP mapped packet traffic s x 239/238 x 5 156 250 kbit/s: s=2,8,5*n, n ≥ 1
ODUflex for FlexE aware transport 103 125 000 x 240/238 x n/20 kbit/s, where n is total number of available tributary slots among all PHYs which have been crunched and combined.

Note that this table doesn't include ODUCn -- since it cannot be generated by mapping a non-OTN signal. An ODUCn is always formed by multiplexing multiple LO-ODUs.


              Clients (e.g. SONET/SDH, Ethernet)
                   +       +      +
                   |       |      |
|                     OPUk                                 |
|                     ODUk                                 |
| OTUk, OTUk.V, OTUkV   |          OPUk             |      |
+----------+----------------------------------------+      |
| OTLk.n   |            |          ODUk             |      |
+----------+            +---------------------+-----+      |
                        | OTUk, OTUk.V, OTUkV |   OPUCn    |
                        | OTLk.n   |          |   ODUCn    |
                        +----------+          +------------+
                                              |   OTUCn    |


Figure 3: Digital Structure of OTN interfaces (from G.709:Figure 6-1)

4. Usecases

This section introduces various usecases, in increasing order of complexity. This material serves as background information that provides the rationale for the requirements that any solution must satisfy. At a later point in time, it is possible to consolidate these usecases so that all the multiplexing (and demultiplexing) variants are encountered along the path of an end-to-end ODU circuit.

Note: These usecases present scenarios in which OTUCn links are depicted. These illustrations do not highlight how the OTUCn is transported between the 3R points. That is, these usecases cover cases in which a standard FlexO interface (e.g. as defined in [ITU-T_G709.1]) is used, or whether a vendor specific mapping of OTUCn to OTSiG (as defined in [ITU-T_G872]) is used. In other words, multiple variants of these usecases based on FlexO usage (or not) are not included in this document.

4.1. 100GE Client Service with a homogeneous chain of OTUC1 links

In the scenario illustrated in Figure 4 a 100GBASE-R client is mapped into an ODU4 at NE1. The resulting ODU4 signal is multiplexed into the ODUC1 server layer (using GMP) and further encapsulated to form the OTUC1 signal. The links NE1-NE2, and NE2-NE3 are both OTUC1 links -- and they can carry one 100GE client mapped into an ODU4 server layer. Actions performed at NE2 are: (a) terminate OTUC1, and ODUC1 towards NE1 (b) demultiplex the ODU4 signal from ODUC1 (c) map the ODU4 signal onto a different ODUC1/OTUC1 towards NE3. NE3 performs the inverse sequence of steps performed at NE1, and recovers the 100GBASE-R client from the ODU4 signal. Note that the ODU4 and ODUC1 signals are not "interoperable" and that the ODUC1 is a server layer to the ODU4 signal.

This illustration is also applicable to the usecase in which members of a FlexE group are transported in a flexe-unaware mode in the transport network. Although this illustration included only OTUC1 signals, any higher rate OTUCn signal can be substituted for these signals. In this particular scenario, there are two adjacent ODUC1 hops, and the NE2 demultiplexs (and multiplexes) the ODU4 onto the ODUC1. It is possible to construct an alternative scenario in the case when NE2 acts as a regenerator, and doesn't terminate the ODUC1 signals in the two hops, and instead repeats the ODUC1 signal; this scenario is specifically discussed in Section 4.6.


    +----------+                                  +----------+
    |  100GE   |                                  |  100GE   |
    +----------+        +---------------+         +----------+
    |  ODU4    |        |      ODU4     |         |  ODU4    |
    +----------+        +---------------+         +----------+
    |  ODUC1   |        | ODUC1 | ODUC1 |         |  ODUC1   |
    +----------+        +---------------+         +----------+
    |  OTUC1   +--------+ OTUC1 | OTUC1 +---------+  OTUC1   |
    +----------+        +---------------+         +----------+
       NE1                  NE2                     NE3

             +------------->            +------------->
                Scope of                    Scope of
                OTUC1, ODUC1                OTUC1, ODUC1


Figure 4: 100GE Client service

4.2. 100GE Client Service with a mix of OTU4, and OTUC1 links

In the scenario illustrated in Figure 5 a 100GBASE-R client is mapped into an ODU4 at NE1. The resulting ODU4 signal is encapsulated with an OTU layer to form the OTU4 signal. Actions performed at NE2 are: (a) terminate OTU4 layer, and extract the ODU4 signal (b) map the ODU4 signal onto a different ODUC1/OTUC1 towards NE3. NE3 performs the same set of actions that were performed by NE3 in Figure 4.


+----------+                                  +----------+
|  100GE   |                                  |  100GE   |
+----------+        +---------------+         +----------+
|  ODU4    |        |      ODU4     |         |  ODU4    |
+----------+        +-------+-------+         +----------+
|  OTU4    +--------+ OTU4  | ODUC1 |         |  ODUC1   |
+----------+        +---------------+         +----------+
                            | OTUC1 +---------+  OTUC1   |
                            --------+         +----------+
     NE1                 NE2                     NE3

        +-------------->         +---------------->
            Scope of                    Scope of
            OTU4 layer                  OTUC1, ODUC1


Figure 5: 100GE Client Service with a mix of OTU4, and OTUC1 links

4.3. 400GE Client Service with a mix of OTUCn links

In the scenario illustrated in Figure 6 a 400GBASE-R client is mapped into an ODUflex at NE1. The resulting ODUflex signal is multiplexed into an ODUC4 (using GMP), and then transformed into an OTUC4 signal. The links between NE1-NE2, and NE2-NE3 are OTUC4 and OTUC6 (respectively). Actions performed at NE2 are: (a) terminate OTUC4, and ODUC4 towards NE1 (b) demultiplex the ODUflex signal from ODUC4 (c) map the ODU4 signal onto ODUC6/OTUC6 towards NE3. NE3 performs the inverse sequence of steps performed at NE1, and recovers the 400GBASE-R client from the ODUflex signal.

Although not specifically illustrated in this figure, the 200G of spare capacity in the NE2-NE3 links can be used to carry other client signals.. Although the scenario illustrated in Figure 6 is specific to 400GE, the treatment for packet clients at other rates (e.g. 25G, 50G, 200G) follows a very similar processing sequence. In the case of 25GBASE-R clients , the 25GE client signal will be mapped to an ODUflex, and can be multiplexed into an ODU4 signal, or an ODUCn signal as illustrated here.


+----------+                                  +----------+
|  400GE   |                                  |  400GE   |
+----------+        +---------------+         +----------+
|  ODUflex |        |    ODUflex    |         |  ODUflex |
+----------+        +-------+-------+         +----------+
|  ODUC4   |        | ODUC4 | ODUC6 |         |  ODUC6   |
+----------+        +---------------+         +----------+
|  OTUC4   +--------+ OTUC4 | OTUC6 +---------+  OTUC6   |
+----------+        +-------+-------+         +----------+
   NE1                  NE2                     NE3

         <------------->            <------------->
            Scope of                    Scope of
            OTUC4, ODUC4                OTUC6, ODUC6



Figure 6: 400GE transport over OTUCn links

4.4. FlexE aware transport over OTUCn links

In the scenario illustrated in Figure 7 NE1 interfaces to a client equipment which includes the FlexE SHIM functions which originate/terminate a FlexE group. The transport network edge node NE2 is FlexE aware -- but doesn't terminate the FlexE group. NE1 may (as defined in the FlexE draft [I-D.izh-ccamp-flexe-fwk]), crunch the unavailable tributary slots in the FlexE PHY signals, and map the resultant stream to one or more ODUflex signals. The links between NE1-NE2, and NE2-NE3 are OTUC4 and OTUC6 (respectively). Actions performed at NE2 are: (a) terminate OTUC4, and ODUC4 towards NE1 (b) demultiplex the ODUflex signal from ODUC4 (c) map the ODUflex signal onto ODUC6/OTUC6 towards NE3. NE3 recovers the Crunched and combined PHY(s) from the ODUflex signal, re-adds the unavailable calendar slots, and outputs the resulting stream towards the FlexE PHY(s).

In the scenario illustrated in Figure 7 the lowest rate OTUCn link is the OTUC4 link between NE1-NE2. This means that the size of the FlexE group is at most 4. FlexE groups with greater sizes can be handled by utilizing appropriate OTUCn links. Note that at most 400G of the capacity of OTUC6 (or 600G) NE2-NE3 link is occupied by the ODUflex signal; the remaining bandwidth can be allocated to other client signals.


FlexE    +----------+                       +----------+       FlexE
group    | Crunched |                       | Crunche  |       Group
  +------> and      |                       | and      +-------->
         | Combined |                       | Combined | Add
         | PHY(s)   |                       | PHY(s)   | unavailable
         +----------+   +---------------+   +----------+ Calendar
         |  ODUflex |   |    ODUflex    |   |  ODUflex | slots
         +----------+   +-------+-------+   +----------+
         |  ODUC4   |   | ODUC4 | ODUC6 |   |  ODUC6   |
         +----------+   +---------------+   +----------+
         |  OTUC4   +---+ OTUC4 | OTUC6 +---+  OTUC6   |
         +----------+   +-------+-------+   +----------+
            NE1             NE2               NE3

                  <--------->        <----------->
                  Scope of            Scope of
                  OTUC4, ODUC4        OTUC6, ODUC6



Figure 7: FlexE aware transport over OTUCn links

4.5. FlexE Client transport over OTUCn links

This use case (see Figure 8) concerns the scenario in which a FlexE group is terminated at the transport network edge node (via the FlexE SHIM function), and the FlexE clients are demultiplexed, and independently transported through the OTN network. In the scenario illustrated in Figure 8 the lowest rate OTUCn link is the OTUC4 link between NE1-NE2. This means that the maximum bit rate of the FlexE client is at most 400G. FlexE clients with greater sizes can be handled by utilizing appropriate OTUCn links. This figure illustrates the case in which one FlexE client is transported between NE1 and NE3. Other FlexE clients recovered at NE1 can routed independently to NE3, or to other network elements.


      +-----------------+                       +---------------+
      |     FlexE       |                       |    FlexE      |
      |     Client      |                       |    Client     |
      +-----------------+                       +---------------+
      | FlexE |         |   +---------------+   |       | Flex  |
      | SHIM  | ODUflex |   |    ODUflex    |   |ODUflex| SHIM  |
      +-----------------+   +---------------+   +---------------+
      | FlexE | ODUC4   |   | ODUC4 | ODUC6 |   | ODUC6 | FlexE |
 +----+ PHY(s +---------+   +---------------+   +-------+ PHY(s)+---->
FlexE |       | OTUC4   +---+ OTUC4 | OTUC6 +---+ OTUC6 |       | FlexE
Group +-----------------+   +---------------+   +---------------+ Group
                 NE1            NE2                 NE3

                   <------------>       <----------->
                      Scope of            Scope of
                      OTUC4, ODUC4        OTUC6, ODUC6


Figure 8: FlexE client transport over OTUCn links

4.6. Multihop ODUCn link

As mentioned in the introductory section, the ODUCn is not a switchable entity. The ODUCn layer is a server layer, which more-or-less occupies the position of a section layer in OTN networks. As such, the ODUCn signal must be terminated and the contained low-order ODU flows can be switched independently to other OTN interfaces. G.709 and G.872 however allow for digital regenerators to terminate the OTUCn layer, and reinject the ODUCn layer towards another interface (where a new OTUCn section layer is started). This scenario is illustrated in Figure 9. In this figure, NE3 is the regenerator. The ODUC2 signal is terminated at NE2, and NE4. At the regeneration points, all the clients embedded inside the ODUCn signal are not touched (i.e. no TS changes can occur). More specifically, the OPUC2 signal is not modified in any way. However, the ODUC2 OH may be modified if intrusive TCM monitoring points are applied to the ODUC2 signal at NE3. It is for this reason that the ODUC2 entity must be visible at NE3.

In scenarios involving multi-hop ODUCn links, GMPLS signalling will be required to first establish the ODUCn LSP, and then use it as an FA-LSP to switch any contained Low-order ODUs.


+----------+                                               +----------+
|  100GE   |                                               |  100GE   |
+----------+     +---------------+                         +----------+
|  ODU4    |     |      ODU4     |                         |  ODU4    |
+----------+     +-------+-------+       +---------+       +----------+
|  ODUC1   |     | ODUC1 | ODUC2 |       |  ODUC2  |       |  ODUC2   |
+----------+     +---------------+       +---------+       +----------+
|  OTUC1   +-----+ OTUC1 | OTUC2 +-------+  OTUC2  +-------+  OTUC2   |
+----------+     +-------+-------+       +---------+       +----------+
   NE1               NE2                     NE3             NE4

         <------------->      <------------->     <------------->
            Scope of               OTUC2                OTUC2
            OTUC1, ODUC1




Figure 9: Multihop ODUCn link

4.7. Use of OTUCn-M links

The scenario illustrated in Figure 10 is a variant of the basic usecase presented in Figure 4. The only difference is that the second hop of the ODU4 connection makes use of a OTUC2-30 link which has a capacity of 150G.


    +----------+                                  +-----------+
    |  100GE   |                                  |  100GE    |
    +----------+        +------------------+      +-----------+
    |  ODU4    |        |      ODU4        |      |  ODU4     |
    +----------+        +-------+----------+      +-----------+
    |  ODUC1   |        | ODUC1 | ODUC2    |      |  ODUC2    |
    +----------+        +------------------+      +-----------+
    |  OTUC1   +--------+ OTUC1 | OTUC2-30 +------+  OTUC2-30 |
    +----------+        +-------+----------+      +-----------+
       NE1                  NE2                     NE3

             +------------->            +------------->
                Scope of                    Scope of
                OTUC1, ODUC1                OTUC2-30



Figure 10: 100GE Client service over OTUCn-M links

4.8. Intermediate State of ODU mux

The ODUCn links have a tributary slot granularity of 5G -- and this makes it a bit inefficient if a small number of ODU0 flows have to be switched across an ODUCn links. In these cases, it is conceivable that the intermediate nodes may offer the convenience of a intermediate-stage multiplexing, whereby multiple ODU0 flows are first multiplexed into a higher rate container (e.g. ODU2), and then multiplexed into an ODUCn signal. This however assumes that all these ODU0 flows are co-routed in the network. If this assumption cannot be made, the only solution is to multiplex these ODU0 flows into higher rate flows, from the source of the traffic. This usecase isn't elaborated in this document. We can add details if required.

5. GMPLS Implications

5.1. OTN ODUCn layer network

As described in the overview section, ODUCn can not be used to support non-OTN client signal, so the mapping hierarchy would be the OTN client signals (e.g. ODU0, ODU1, ODU2, ODU2e, ODU3, ODU4, ODUflex) are first multiplexed into an ODUCn container, then the ODUCn container is then mapped into OTUCn (see Figure 3). The signal hierarchy supported by the ODUCn and OTUCn layers needs to be taken into consideration in control plane Routing and Signaling.

5.2. Implications for GMPLS Signaling

As described in Section 3 [ITU-T_G709_2016] introduced some new containers, such as OPUCn, ODUCn, and OTUCn. The GMPLS signaling mechanisms defined in [RFC4328] and [RFC7139] do not support these new OTN features. Therefore, GMPLS signaling protocol extensions will be necessary to support this new functionality. The following summarizes key aspects that should be considered for GMPLS signaling extensions:

  1. The GMPLS signalling protocol SHALL be able to specify the new ODUCn/OTUCn signal types and related traffic information. The traffic parameters should be extended in a signalling message to support the new ODUCn/OTUCn signal types
  2. The GMPLS signalling protocol SHALL be able to set up ODUCn/OTUCn LSP with related mapping and multiplexing capabilities, and allocate slot resources for ODU clients signal. [Note: Under Discussion]
  3. The GMPLS signalling protocol SHALL be able to set up LSP using 5G TS granularity
  4. The GMPLS signalling protocol SHALL support the TPN allocation and negotiation
  5. The GMPLS signalling protocol SHALL support the setup of OTUCn sub rates (OTUCn-M) LSP, which includes the negotiation of unavaliable slots number, slots postion and allocation of slot resources. [Note: Under Discussion]
  6. The GMPLS signalling protocol SHALL be able to set up ODUCn/OTUCn/OTUCn-M LSP over FlexO group. [Note: Under Discussion]
  7. The GMPLS signalling protocol SHALL be able to set up ODUCn/OTUCn/OTUCn-M LSP over different kinds of FlexO interfaces (e.g., 100G/200G FlexO interfaces) [Note: Under Discussion]

5.3. Implications for GMPLS Routing

The path computation process needs to select a suitable route and capabilities for an ODUCn/OTUCn/OTUCn-M connection request. In order to perform the path computation, it needs to evaluate the available bandwidth/slots available on one or more candidate links. The routing protocol should be extended to convey sufficient information to represent ODU Traffic Engineering (TE) topology. Following GMPLS Routing Implications should be considered:

  1. The GMPLS Routing protocol should be able to indicate the ODUCn/OTUCn/OTUCn subrates (OTUCn-M) support information
  2. The GMPLS Routing protocol SHALL support the advertisement of 5G Tributary Slot Granularity
  3. The GMPLS Routing protocol SHALL support the advertisement of unused ODUCn tributary slot resource information.
  4. The GMPLS Routing protocol SHALL support the advertisement of available/unavailable tributary slot information in OTUCn-M scenario
  5. The GMPLS Routing protocol SHALL support the advertisement of OTUCn implementation (FlexO) specific information, including the specific Flexible OTN interface support information [Note: Under Discussion]

6. Acknowledgements

7. IANA Considerations

This memo includes no request to IANA.

8. Security Considerations


9. References

9.1. Normative References

[ITU-T_G709.1] ITU-T, "ITU-T G.709.1: Flexible OTN short-reach interface; 2016", , 2016.
[ITU-T_G709_2012] ITU-T, "ITU-T G.709: Optical Transport Network Interfaces; 02/2012",, February 2012.
[ITU-T_G709_2016] ITU-T, "ITU-T G.709: Optical Transport Network Interfaces; 07/2016",, July 2016.
[ITU-T_G798] ITU-T, "ITU-T G.798: Characteristics of optical transport network hierarchy equipment functional blocks; 02/2014",, February 2014.
[ITU-T_G872] ITU-T, "ITU-T G.872: The Architecture of Optical Transport Networks; 2017",, January 2017.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.
[RFC4328] Papadimitriou, D., "Generalized Multi-Protocol Label Switching (GMPLS) Signaling Extensions for G.709 Optical Transport Networks Control", RFC 4328, DOI 10.17487/RFC4328, January 2006.
[RFC7138] Ceccarelli, D., Zhang, F., Belotti, S., Rao, R. and J. Drake, "Traffic Engineering Extensions to OSPF for GMPLS Control of Evolving G.709 Optical Transport Networks", RFC 7138, DOI 10.17487/RFC7138, March 2014.
[RFC7139] Zhang, F., Zhang, G., Belotti, S., Ceccarelli, D. and K. Pithewan, "GMPLS Signaling Extensions for Control of Evolving G.709 Optical Transport Networks", RFC 7139, DOI 10.17487/RFC7139, March 2014.

9.2. Informative References

[I-D.izh-ccamp-flexe-fwk] Hussain, I., Valiveti, R., Pithewan, K., Wang, Q., Andersson, L., Zhang, F., Chen, M., Dong, J., Du, Z.,, z., Zhang, X., Huang, J. and Q. Zhong, "GMPLS Routing and Signaling Framework for Flexible Ethernet (FlexE)", Internet-Draft draft-izh-ccamp-flexe-fwk-00, October 2016.
[OIF_FLEXE_1.0] OIF, "FLex Ethernet Implementation Agreement Version 1.0 (OIF-FLEXE-01.0)", March 2016.

Authors' Addresses

Qilei Wang (editor) ZTE Nanjing, CN EMail:
Yuanbin Zhang ZTE Beijing, CN EMail:
Radha Valiveti Infinera Corp Sunnyvale, USA EMail:
Iftekhar Hussain (editor) Infinera Corp Sunnyvale, USA EMail:
Rajan Rao Infinera Corp Sunnyvale, USA EMail:
Huub van Helvoort Hai Gaoming B.V EMail: