Internet-Draft MPLS Sub-Stack for DLA April 2023
Song, et al. Expires 28 October 2023 [Page]
DetNet Working Group
Intended Status:
Standards Track
X. Song
ZTE Corp.
Q. Xiong
ZTE Corp.
R. Gandhi
Cisco Systems, Inc.

MPLS Sub-Stack Encapsulation for Deterministic Latency Action


This document specifies formats and principals for the MPLS header which contains the Deterministic Latency Action (DLA) option, designed for use over a DetNet network with MPLS data plane. It enables guaranteed latency support and makes scheduling decisions for time-sensitive service running on DetNet nodes that operate within a constrained network domain.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 28 October 2023.

Table of Contents

1. Introduction

As specified in [RFC8655] and [RFC8938], Deterministic Networking (DetNet) operates at the IP layer and delivers service with low data loss rates and bounded latency guarantee within a network domain.

As defined in [RFC8964], the DetNet MPLS data plane provides a foundation of building blocks to enable PREOF (Packet Replication, Elimination and Ordering Functions (PREOF)) functions to DetNet service and forwarding sub-layer. The DetNet service sub-layer includes a DetNet Control Word (d-CW), service label (S-Label), an aggregation label (A-Label) in special case of S-Label used for aggregation. The DetNet forwarding sub-layer supports one or more forwarding labels (F-Labels) used to forward a DetNet flow over MPLS domains. The DetNet forwarding sub-layer provides corresponding forwarding assurance with IETF existing functions using resource allocations and explicit routes. But these functions may not be enough to provide the deterministic latency (including bounded latency, low packet loss and in-order delivery) assurance. Because latency variation in one DetNet system results in the need for extra buffer space in the next-hop DetNet system(s), which in turn increases the worst-case per-hop latency. So standard queuing and scheduling algorithms are required to compute and reserve the sufficient buffer space for DetNet nodes along the path of DetNet flows.

To support time-sensitive service with ultra-low loss rates and deterministic latency, it is required to apply feasible scheduling mechanisms to specific applications for deterministic networking. As described in [RFC9320], the end-to-end bounded latency is considered as the sum of non-queuing and queuing delay bounds along with the queuing mechanisms. The value for non-queuing delay bounds (which consist of packet output delay, link delay, frame preemption delay and processing delay) is relative with the physical capability of on-used networks and can be considered to be stable. The unstable latency delay bounds are mainly from queuing delay and regulation delay. The regulation delay is mainly from regulation policy. To simplify the question this draft assumes there is no regulation policy. So the question is left to address the selection for queuing mechanisms and queuing delay information encapsulation in data plane.

The queuing mechanisms, as mentioned in [RFC9320] and [RFC8655], which include Time Aware Shaping IEEE802.1Qbv, Asynchronous Traffic Shaping IEEE802.1Qcr, cyclic-scheduling queuing mechanism proposed in IEEE802.1Qch. There are also discussions on new queuing or scheduling mechanisms such as [I-D.peng-6man-deadline-option] and [I-D.dang-queuing-with-multiple-cyclic-buffers]. In terms of delay guarantee for different applications, to select the right scheduling/queuing mechanism applied to a specific application is required.

Based on the existing DetNet MPLS encapsulations and mechanisms [RFC8964], the draft defines the encoding format for Deterministic Latency Action (DLA) option carried in MPLS sub-stack.

2. Conventions

2.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

2.2. Terminology

Refer to [RFC8655], [RFC8964], [I-D.ietf-mpls-mna-hdr] and [RFC9320] for the key terms used in this document.

Deterministic Latency (DL):the bound of network latency and delay variation between two DetNet endpoints. It may includes parameters such as bounded latency, bounded delay variation, etc.

Deterministic Latency Action (DLA): used to indicate deterministic latency actions for MPLS Sub-Stack.

3. DetNet DLA Option

3.1. Queuing delay

[RFC8655] provides the architecture for deterministic networking (DetNet) which enables the service delivery of DetNet flows with extremely low packet loss rates and deterministic latency. The forwarding sub-layer provides corresponding forwarding assurance but can not provide the deterministic latency (including bounded latency, low packet loss and in-order delivery). As described at [RFC9320], the end-to-end bounded latency for one DetNet flow is the sum of delay bound of non-queuing and queuing processing latency. The delay bound for non-queuing processing may include output delay, link delay, frame preemption delay, and processing delay, the delay bound for queuing processing may include regulator delay, queuing delay. It is assumed that the delay of non-queuing processing is fixed or be ignorable, the delay of queuing processing is variable. To realize the guarantee of bounded latency service it is important to select right queuing methodology applied to specific applications and carry necessary queuing delay information for computation of end-to-end latency.

3.2. DLA Option

The DetNet data plane encapsulation in transport network with MPLS data plane is specified in [RFC8964]. [I-D.xiong-detnet-data-fields-edp] has proposed a commom DetNet data fields for enhanced DetNet data plane and defined a DLA option to carry queuing-based metadata.This document provides additional encapsulation for the DLA in MPLS data plane.

The DetNet routers in data plane perform MPLS forwarding functions to choose a feasible way with sufficient network resources for the incoming packets, and makes right selection on the queuing or scheduling mechanisms applied for specific DetNet flows to satisfy strict QoS criteria in the forwarding output port. The information for the queuing or scheduling mechanisms are carried in DetNet DLA header. Refer to [I-D.stein-srtsn], considering the time latency information are processed per hop so the time latency informations (such as deadline time, cycle identify, etc.) of each DetNet node for DetNet flows are expected to be carried as a set of lists of LSEs in MPLS data plane.

4. MPLS Extension for DLA

4.1. DetNet MPLS Header for DLA

The DetNet MPLS header follows [RFC8964], as showed the below figure 1, the SP-Lable (SPL) is added to indicate Deterministic Latency Action (DLA).

|       DetNet App-Flow     |
|       Payload Packet      |
|     DetNet Control Word   |   |
+---------------------------+   |
|          S-Label          |   | DetNet Data Plane
+---------------------------+   | MPLS Encapsulation
|          SP-Lable         |   |
+---------------------------+   |
|          F-Label(s)       |   |
|         Data-Link         |
|          Physical         |
Figure 1: DetNet MPLS Header

The SP-L (Special Purpose-Label) may be B-SPL [RFC9017], new-SPL, extended SPL [RFC9017]. This draft follows the MNA (MPLS Network Action) solution specified in [I-D.ietf-mpls-mna-hdr] and [I-D.ietf-mpls-mna-fwk], and uses b-SPL to indicate the presence of the MPLS Network Action Sub-Stack (NASS). The value for the bSPL value is to be assigned by IANA and follows the assignment in [I-D.ietf-mpls-mna-hdr]. The SP-Label field is formatted as below figure.

4.2. MPLS ISD for DLA

 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
|     NASI Label = bSPL (TBA3)          | TC  |S|      TTL      |
Figure 2: SP-Label Format

NASI Label:

A new bSPL value is to be assigned by IANA. It is used to indicate the presence of the MPLS Network Action Sub-Stack (NASS). The assignment for this field value refers to [I-D.ietf-mpls-mna-hdr].

The MPLS sub-stack encoding format for DLA option is showed as figure 3. The format provides DetNet Latency Network Action Indicator (NAI) indicates the specific DLA. Its detailed information is carried in Ancillary Data.

 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
| IS-DLA=TBA1 |                Flag           |S|  Data |  NAL  |
|1|             AD LSE1                       |S|    AD LSE1    |
|1|             AD LSE2                       |S|    AD LSE2    |
|                 ...  ...  ...                                 |
|1|             AD LSEn                       |S|    AD LSEn    |

Figure 3: MPLS LSE Format for DLA

IS-DLA (7 bits): This is the first 7-bit value in the Label Field. The value is used to indicate In-Stack DLA and to be assigned by IANA as value TBA1.

Flag (16 bits): identifies the type of queuing mechanisms used in the network. The queuing type format is defined in section 4.2 of [I-D.xiong-detnet-data-fields-edp].

S (1 bit) : The Bottom of Stack [RFC3032].

Data (4 bits) : Reserved bits for future use.

NAL (4 bits): The DLA action length. It indicates the number of AD LSEs in the sub-stack.

The first bit in the Label field of the following AD LSEs MUST be set to "1". As specified in [I-D.ietf-mpls-mna-hdr] this is to prevent aliasing the label field with other bSPLs on the legacy routers.

Ancillary Data:

The 19-bit Label field and 4-bit TC field and 8-bit TTL field (except S bit) in the additional LSEs are used to carry the Ancillary Data for specific DLA latency information.

The Ancillary Data LSE1 is expected to carry the latency informationDLA option data defined in section 4.2.2 of [I-D.xiong-detnet-data-fields-edp] of the edge DetNet node. Depend on specific queuing mechanisms used in the network the DLA field length for the latency information for one DetNet node is variable. The specific queuing (including cycle, deadline, etc.) data encapsulation are described in [I-D.xiong-detnet-data-fields-edp].

The first AD LSE is expected to carry the latency information for edge DetNet node. When the length of latency information is more than 32bits less than 64bits, both AD LSE1 and LSE2 are expected to carry the latency information for edge DetNet node. In this case, the DLA option data of next hop is carried in AD LSE3 and LSE4. Along the path of DetNet flows the AD LSE(n-1) and LSEn carry the DLA option data for the peer edge DetNet node. With the DetNet flows are being forwarded in its output ports the corresponding node latency information carried in DLA options are processed hop by hop.

4.3. MPLS PSD for DLA

There are some limitaions to use MPLS ISD for DLA. For example, the total length for ISD LSEs seems small (no more than 16LSEs), but on the other hand the time-latency information of each nodes (nodes number may be more than 16 in DetNet) along the path of DetNet flows needs be carried in the MPLS sub-stack. Besides HBH scenario brings high packet overload and low encapsulation efficiency. This document does not exclude MPLS PSD option for DLA. Considering the MPLS PSD is under discussion and has not been reached consensus in MPLS WG whether to use MPLS ISD or PSD for DLA is left for future decision.

The MPLS PSD indicator format refers to [I-D.ietf-mpls-mna-hdr] and [I-D.jags-mpls-ps-mna-hdr] showed as figure 4.

 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
|           Label=MNA bSPL              | TC  |S|    TTL        |
| IS-DLA=TBA1 |          Flag           |1|IHS|S| Res |U| NASL=0|
~                                                               ~
|                                             |1|               |
~                   Post-Stack Network Actions                  ~
|N N N N|Version| PS-MNA-LEN    | TYPE = POST-STACK-MNA         | TH
| PS-DLA=TBA2 |       DLA Ancillary Data      |R|R| DLA-HDR-LEN |  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  |
|                                                               |  D
|                                                               |  L
~              DLA Option and Ancillary Data                    ~  A
|                                                               |  |
|                                                               |  |

Figure 4: MPLS PSD for DLA

As specified at [I-D.ietf-mpls-mna-hdr], The header for In-Stack Network Action encodes:

P (1 Bit) : This document sets the value to 1 to identify the presence of Post-Stack Network Action.

IHS (2 Bit) : Indicates the combined scope of the In-Stack and the Post-Stack Network Actions.

U (1 Bit) : Indicates the combined Unknown Action Handling of the In-Stack and the Post-Stack Network Actions.

As specified at [I-D.jags-mpls-ps-mna-hdr], The header for Post-Stack Network Action encodes:

NNNN (4 bits): This first nibble identifies the start of the Post-Stack Network Actions. A new value can be assigned by IANA (value TBA1). Generic Associated Channel (0001b) can be used instead.

Version (4 bits): This is Post-Stack MNA version.

PS-MNA-LEN (8 bits): Post-Stack MNA Total Length in words. This excludes the Post-Stack Top header.

TYPE (16 bits): Type is set to POST-STACK-MNA. In case of DetNet MPLS, this is DLA Type.

PS-DLA (7 bits): Post-Stack Network Action Opcode. The value is used to indicate Post-Stack DLA and to be assigned by IANA as value TBA2.

DLA Ancillary Data (16 bits): Post-Stack Ancillary Data associated with the DLA.

R (2 bits): Reserved bits.

DLA-HDR-LEN (7 bits): Post-Stack Network Action Length for the DLA Action.

The detailed MNA PSD encapsulation of the DLA will refer to the MNA PSD WG document once the MNA PSD option is adopted in MPLS WG.

5. IANA Considerations

This document requests two new IANA-managed code-points for DetNet application processing. IANA maintains the "In-Stack MNA Opcode" registry when created from IANA request in [I-D.ietf-mpls-mna-hdr]. IANA is requested to allocate a value for In-Stack Network Action Opcode for DLA application from this registry:

Table 1: In-Stack DLA Indicator
Value Description Reference
TBA1 IS-DLA this document

IANA maintains the "Post-Stack MNA Opcode" registry when created from IANA request in [I-D.jags-mpls-ps-mna-hdr]. IANA is requested to allocate a value for Post-Stack Network Action Opcode for DLA application from this registry:

Table 2: Post-Stack DLA Indicator
Value Description Reference
TBA2 PS-DLA this document

6. Security Considerations

Security considerations for DetNet are covered in the DetNet Architecture RFC8655 and DetNet Security Considerations [RFC9055]. MPLS security considerations are covered in [RFC8964], [RFC3031], [RFC3032]. These security considerations also apply to this document.

7. Acknowledgements

The authors would like to acknowledge Shaofu Peng for his thorough review and very helpful comments.

8. References

8.1. Normative References

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <>.
Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, DOI 10.17487/RFC3031, , <>.
Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Encoding", RFC 3032, DOI 10.17487/RFC3032, , <>.
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <>.
Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", RFC 8655, DOI 10.17487/RFC8655, , <>.
Varga, B., Ed., Farkas, J., Berger, L., Malis, A., Bryant, S., and J. Korhonen, "Deterministic Networking (DetNet) Data Plane: MPLS", RFC 8964, DOI 10.17487/RFC8964, , <>.
Andersson, L., Kompella, K., and A. Farrel, "Special-Purpose Label Terminology", RFC 9017, DOI 10.17487/RFC9017, , <>.

8.2. Informative References

Liu, B. and J. Dang, "A Queuing Mechanism with Multiple Cyclic Buffers", Work in Progress, Internet-Draft, draft-dang-queuing-with-multiple-cyclic-buffers-00, , <>.
Andersson, L., Bryant, S., Bocci, M., and T. Li, "MPLS Network Actions Framework", Work in Progress, Internet-Draft, draft-ietf-mpls-mna-fwk-03, , <>.
Rajamanickam, J., Gandhi, R., Zigler, R., Song, H., and K. Kompella, "MPLS Network Action (MNA) Sub-Stack Solution", Work in Progress, Internet-Draft, draft-ietf-mpls-mna-hdr-02, , <>.
Rajamanickam, J., Gandhi, R., Zigler, R., Li, T., and J. Dong, "Post-Stack MPLS Network Action (MNA) Solution", Work in Progress, Internet-Draft, draft-jags-mpls-ps-mna-hdr-00, , <>.
Peng, S., Tan, B., and P. Liu, "Deadline Option", Work in Progress, Internet-Draft, draft-peng-6man-deadline-option-01, , <>.
Stein, Y. J., "Segment Routed Time Sensitive Networking", Work in Progress, Internet-Draft, draft-stein-srtsn-01, , <>.
Xiong, Q. and D. Yang, "Data Fields for DetNet Enhanced Data Plane", Work in Progress, Internet-Draft, draft-xiong-detnet-data-fields-edp-00, , <>.
Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S. Bryant, "Deterministic Networking (DetNet) Data Plane Framework", RFC 8938, DOI 10.17487/RFC8938, , <>.
Grossman, E., Ed., Mizrahi, T., and A. Hacker, "Deterministic Networking (DetNet) Security Considerations", RFC 9055, DOI 10.17487/RFC9055, , <>.
Finn, N., Le Boudec, J.-Y., Mohammadpour, E., Zhang, J., and B. Varga, "Deterministic Networking (DetNet) Bounded Latency", RFC 9320, DOI 10.17487/RFC9320, , <>.

Authors' Addresses

Xueyan Song
ZTE Corp.
Quan Xiong
ZTE Corp.
Rakesh Gandhi
Cisco Systems, Inc.