DetNet Working Group D. Trossen INTERNET-DRAFT Huawei Intended Status: Standards Track F.-J. Goetz Expires: April 25, 2021 J. Schmitt Siemens October 23, 2020 DetNet Control Plane Signaling draft-trossen-detnet-control-signaling-00.txt Abstract This document provides solutions for control plane signaling, in accordance with the control plane framework developed in the DetNet WG. The solutions cover distributed, centralized, and hybrid signaling scenarios in the TSN and SDN domain. We propose changes to RSVP IntServ for a better integration with Layer 2 technologies for resource reservation, outlining example API specifications for the realization of the revised RSVP (called RSVP-detnet in the document). Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Trossen, et al. Expires April 25, 2021 [Page 1] INTERNET DRAFT DetNet Control Plane Signaling Copyright and License Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Distributed Control in Bridged TSN-based Ethernet Deployment . 3 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 RAP Reservation in TSN vs RSVP IntServ Model . . . . . . . . 4 2.3 Interactions between L2 and L3 . . . . . . . . . . . . . . . 5 2.4 Similarities and Differences between RSVP and RAP . . . . . 6 2.5. RSVP-DetNet . . . . . . . . . . . . . . . . . . . . . . . . 8 2.6. API Specifications . . . . . . . . . . . . . . . . . . . . 9 3. Centralized Control Signaling in SDN Domain . . . . . . . . . . 16 4. Hybrid Control Signaling in SDN Domain . . . . . . . . . . . . 16 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 16 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 16 7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 16 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16 8.1 Normative References . . . . . . . . . . . . . . . . . . . 17 8.2 Informative References . . . . . . . . . . . . . . . . . . 17 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 Trossen, et al. Expires April 25, 2021 [Page 2] INTERNET DRAFT DetNet Control Plane Signaling 1 Introduction The authors in [ID.malis-detnet-controller-plane-framework-03] provide an overview of the DetNet control plane architecture along three possible classes, namely (i) fully distributed control plane utilizing dynamic signaling protocols, (ii) a centralized, SDN-like, control plane, and (iii) a hybrid control plane. We structure the following sections of this draft along those three classes in order to present example solutions for each class of the DetNet control plane architecture. Specifically, Section 2 will present a solution for a Bridged Ethernet deployment scenario. We will introduce changes to RSVP with the proposal for an RSVP-DetNet model that splits resource style and sender selection between sender and receiver, unlike in RSVP for IntServer, for an optimized realization of L2 integrations. Section 3 will present a solution that realizes a centralized SDN- based approach for switched Ethernet deployment scenarios. Section 4 will finally outline a hybrid solution in an SDN domain with path allocation through signaling and switching configuration as a centralized solution. At this stage, Section 3 and 4 will be detailed in future updates to the draft. 1.1 Terminology This document uses the terminology established in the DetNet Architecture [RFC8655], and the reader is assumed to be familiar with that document and its terminology. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2 Distributed Control in Bridged TSN-based Ethernet Deployment The first solution addresses deployments of bridged TSN-based customer Ethernet network, possibly interconnected through DetNet IP flows for multi-site deployment. 2.1 Overview Figure 1 provides an example deployment of TSN-based Ethernet edge (customer) networks, using the RSVP/RAP combined signaling presented in the following sections. The customer sites are interconnected via edge routers that aggregate DetNet IP flow requirements from hosts Trossen, et al. Expires April 25, 2021 [Page 3] INTERNET DRAFT DetNet Control Plane Signaling for reservation of aggregated flows within the core network. Starting point from an DetNet perspective is the RSVP IntServ model with guaranteed QoS. Resource Allocation Protocol (RAP), as defined in [RAP_IEEE], is an example of a target lower layer reservation mechanism. In this section, we focus on the necessary integration of the RSVP and RAP concepts to enable such (and similar) deployment. +-------+ +-------+ +-----+ +-------+ +-------+ +-------+ +-----+ |RSVP/| |DetNet | +------+ | DetNet| +------+ |DetNet | |RSVP/| |RAP |-|IP Flow| |Edge | |IP Flow| |Edge | |IP Flow|-|RAP | |Node | | Edge |--|Router|-| Core |-|Router|--| Edge | |Node | +-----+ |Network| +------+ |Network| +------+ |Network| +-----+ +-------+ +-------+ +-------+ Figure 1 : IP + Bridged Ethernet Within Customer Networks 2.2 RAP Reservation in TSN vs RSVP IntServ Model Layer2 reservation in TSN-based networks is supported through RAP, providing a maximum of 8 classes of traffic where the frame priority code point (PCP) is used to select the Resource Allocation (RA) class at the ingress bridge. Streams within a single RA class are queued in a single traffic class where the latency of the stream is guaranteed per hop and per RA class. This model contrasts with the RSVP IntServ [RFC2212] model, which provides a flow bandwidth driven latency model with a separate transmission queue per flow, not a class-based model like in the aforementioned RAP model. This difference in models poses a number of challenges: 1. Is RSVP IntServ (as defined in [RFC2212]) the right starting point? 2. How to efficiently map the different reservation styles of RSVP onto RAP? 3. What is the nature of the RSVP-RAP interface? 4. How is the binding between L3 signaling (RSVP IntServer) and L2 signaling (RAP ) realized, e.g., mapping of Stream-ID? The following sub-sections elaborate on the various aspects in addressing those challenges. Trossen, et al. Expires April 25, 2021 [Page 4] INTERNET DRAFT DetNet Control Plane Signaling 2.3 Interactions between L2 and L3 Figure 2 provides an overview of the interactions between L2 and L3 elements in a network deployment as an elaboration of the elements in Figure 1. The application utilizes an application-specific QoS API (qAPI) that controls and signals the establishment of deterministic end-to-end flows via the DetNet API (dAPI) between the L3 control plane entities in the L3 end systems and routers, utilizing the dUNI, based on RSVP, at Layer 3. The DetNet lower API (dlAPI) maps the RSVP model onto the RAP model and signals the establishment of appropriate L2 segments via the TSN API (tAPI) between the L2 control plane entities of the end systems, routers, and L2 bridges, utilizing the tUNI, based on RAP, at Layer 2. The L2 CP entities in turn control the establishment of appropriate resources on the HW bridge (with no specification for the handling of resource reservations on the end systems). Within the DetNet router (on the right side of Figure 2), the second L2 control plane entity bridges to the second Ethernet sub-network. +-----------+ |Application| +-----------+ | qAPI | C| |S | duAPI | User Network DetNet Router +-----------+ +-------------+ | L3 CP |<**************************************>| L3 CP | +-----------+ +-------------+ |dlAPI | | | | | M&A| |S M| S| M| S| | tAPI | Bridge Bridge | | | | +-----------+ +-----------+ +-----------+ +-----+ +-----+ | L2 CP |<===>| L2 CP | | L2 CP |<===>|L2 CP| |L2 CP| +-----------+ +-----------+ +-----------+ +-----+ +-----+ | | | | | C| C| C| C| C| | | | | | +-----------+ +-----------+ +-----------+ +-------------+ |HW Endpoint|-----| HW Bridge |----| HW Bridge |-----| HW Router + +-----------+ +-----------+ +-----------+ +-------------+ Trossen, et al. Expires April 25, 2021 [Page 5] INTERNET DRAFT DetNet Control Plane Signaling ***** L3 Signaling Control Protocol DetNet UNI (dUNI RSVP) ===== L2 Reservation Control Protocol TSN UNI (tUNI IEEE P802.1Qdd) HW Hardware CP Control Plane qAPI QoS API duAPI DetNet upper API dlAPI DetNet lower API tAPI TSN API C Controls S Signals M&A Maps and Aggregation Figure 2 : Interaction between L2 and L3 Based on the above interactions, the main body of work is the proposed change to RSVP, dubbed RSVP-DetNet, for an efficient mapping of L3 to L2, and motivated in Section 2.5, while we will present the various API specifications in Figure 2 throughout Section 2.6, including an example mapping between dlAPI and RAP in Section 2.6.3. Before doing so, we will outline similarities and differences in the RSVP and RAP models at Layer 3 and 2, respectively. 2.4 Similarities and Differences between RSVP and RAP The following sub-sections will outline various aspects to be considered when designing the RSVP-RAP interface, namely the assumptions on network nodes (Section 2.4.1), the mapping of the latency model used in both models (Section 2.4.2), the dealing with latency margins (Section 2.4.3), the dealing with Jitter and non- shaping nodes (Section 2.4.4), and the mapping of resource reservation styles (Section 2.4.5). 2.4.1 Assumptions on Network Nodes RSVP assumes three different nodes over which a reservation can be done, namely - Shaping node, which implements the RSVP signaling and shaping on the data plane, - None shaping node, which implements the RSVP signaling and is capable of estimating the latency caused by this node - Legacy node, which does neither implement RSVP nor any shaping. RAP assumes properties common to all nodes within a reservation domain: - All nodes take part in the signaling process Trossen, et al. Expires April 25, 2021 [Page 6] INTERNET DRAFT DetNet Control Plane Signaling - Different data plane architectures are supported albeit limited to those defined in IEEE 802.1Q. - Bridging between different (heterogeneous) data planes is achieved through a peer-to-peer model where every upstream node is a talker for the next downstream node. 2.4.2 Mapping of Latency Model RSVP assumes a weighted fair queuing (WFQ) at the data plane, where a listener is able to influence therefore the latency through the reserved bandwidth per flow. RAP assumes one traffic class with given interference per common RA class, resulting in a per hop latency for all stream within a single RA class. The E2E latency is just signaled by accumulating hop latency while the allowed interference determines the amount of allowed flow per RA class. Here, the listener is unable to influence the latency but the stream requirement is signaled upstream. 2.4.3 Dealing with Latency Margins RSVP provides the notion of slack [RFC2212] per flow, which can be consumed by the processing node in the network to enable additional reservations. In RAP, every listener of a stream propagates its required latency upstream to the talker. Latency margins are not handled directly by RAP, while the per hop latency of an RA class is preconfigured by management. In each node, the per RA class upstream required latency of all streams can be used to locally calculate the latency margins per hop. The management system can then use this information to adjust the per hop maximum latency at runtime. 2.4.4 Dealing with Jitter and Non-Shaping Nodes RSVP has two different parameters to propagate the maximum non- conformance to the leaky bucket model introduced through jitter and non-shaping nodes. These can be accumulated by non-shaping nodes, i.e., those which implement the RSVP protocol but are not performing shaping at the data plane. Within RAP, there is no distinction between shaping and non-shaping nodes since all nodes adhere to the data plane architecture defined in IEEE 802.1Q. Heterogeneous data planes are possible as long as assurances to the next hop can be upheld, while RA class attributes are used to propagate data plane behavior (e.g., shaper) to the next Trossen, et al. Expires April 25, 2021 [Page 7] INTERNET DRAFT DetNet Control Plane Signaling neighbor. 2.4.5 Mapping Resource Reservation Styles RSVP uses the notion of 'sessions', which are able to maintain different kinds of end-to-end connectivity and resource styles, namely fixed (i) filter style, (ii) shared explicit style, and (iii) wildcard filter style - see [RFC2205]. It is important to note that in RSVP, both sender selection and resource styles are controlled by the receiver; we return to this issue in our next section. RAP supports only distinct explicit mode of reservation, while in principle supporting reservation between one talker and multiple listeners or one listener and multiple scheduled talkers. Bridged Ethernet technology is also able to support the shared resource modes. || Resource Style | Sender || | Selection || Distinct | Shared | Coordinated Shared | -----------------||-------------|-------------|--------------------| || | | | Explicit || supported | supported | supported | -----------------||-------------|-------------|--------------------| || | | | Wildcard || | supported | | -----------------||------------------------------------------------| Figure 3 : Resource Style and Sender Selection [RFC2205] 2.5. RSVP-DetNet In this section, we motivate the introduction of a new signaling model for RSVP in combination with sub-nets like TSN. We outline first the rationale for its introduction before outlining the proposed changes. 2.5.1. Rationale Continuing from Section 2.4.5. , in RSVP (for IntServ), the receiver initiates resource style and sender selection through the Resv message being sent upstream, while path state being setup through the Path message from the sender to the receiver upon receiving the Resv message. When looking into an integration with lower layer APIs, such as the TSN API, we identify key differences in WHEN these lower layer APIs decide if a reservation is possible: Trossen, et al. Expires April 25, 2021 [Page 8] INTERNET DRAFT DetNet Control Plane Signaling 1. For a new Announce downstream, each L2 node decides that if this stream was reserved at this port, would there be enough resources available to do so? 2. For a new Attachment upstream, each L2 node will lock the required resources and bandwidth exclusively for this stream. For every L2 node local non-locked Announce, the L2 node will decide the same question as in item 1 and refresh and propagate the necessary states accordingly. It is important to note that steps 1 and 2 only work if the 'resource style' is already known by the Announce propagation. 2.5.2. Splitting Control over Resource Style and Sender Selection In order to allow for an efficient resource reservation at the lower network level by implementing the steps 1 and 2 in Section 2.5.1. , we propose to split the control over 'resource style' and 'sender selection' in that in RSVP-DetNet the sender controls the 'resource style' and the listener controls the 'sender selection'. 2.5.3. Coordinated Shared Resource Style Independent from the efficient realization of lower layer resource reservation, we have also identified a requirement in industrial use cases to support a large amount of deterministic connections with small data usage. In those cases, separate reservation for each connection could be inefficient. To address this, we propose to introduce another 'resource style' called 'Coordinated Shared', which would indicate the use of scheduling (of those many deterministic connections) at L2-Listener and L3-Receiver level. A first proposal for a solution in the TSN RAP protocol was presented to the IEEE in [CHEN-IEEE] 2.6. API Specifications The following sub-sections describe the upper and lower Interfaces, following the proposed RSVP-DetNet changes, and provides an example mapping of the RSVP lower API to RAP in a TSN setup. 2.6.1. RSVP-DetNet upper API (duAPI) The RSVP-DetNet interface is oriented on the interface specified by RSVP-IntServ (RFC 2205). Most of the changes are due to mapping resource reservation styles (see chapter 2.4.5). Sender Trossen, et al. Expires April 25, 2021 [Page 9] INTERNET DRAFT DetNet Control Plane Signaling Call: Open L3 Session (oriented to the RSVP-IntServ interface) Request parameter: - Flow destination IP address, Protocol ID, Destination Port Response parameter: - L3 Session ID (local handle) Call: Add IP Flow Request parameter - L3 Session ID, Sender source IP address, Source Port, DSCP - Traffic Specification: Maximum IP packet size (per flow, <= MTU), Minimum IP packet size, Burstiness, IP packet information rate - Select one of the Resource Style: Distinct, Shared, Coordinated Shared - Data TTL, PATH MTU size, Loss Rate Notes for new parameter: The DSCP is required to map IP flows according their service class to offered service classes of the lower layer. The traffic specification is enhanced by Minimum IP packet size for optimization interference calculation. The resource style for an IP flow is announced by the sender within the path message. The Loss Rate is accumulated per IP Flow. Upcall: IP Flow - L3 Session ID - One of the Info_type: RESV_EVENT; PATH_ERROR Receiver Call: Open L3 Session Request parameter Trossen, et al. Expires April 25, 2021 [Page 10] INTERNET DRAFT DetNet Control Plane Signaling - Flow destination IP address, Protocol ID, Destination Port Response parameter - L3 Session ID Call: Attach IP Flow Request parameter - L3 Session ID - Select one of the IP flow Source Selection: Wildcard, List of explicit sources with Source Port - Maximum packet size - Extended Traffic Specification: Maximum Expected Latency Notes for new parameter: The Source Selection is split form the RSVP-IntServ Reservation Style but still follows the rules defined by RSVP-IntServ. The extended traffic specification Maximum Expected Latency is propagated and merged to a minimum upstream form receiver to sender. Upcall: IP Flow - L3 Session ID - One of the Info_type: RESV_EVENT; PATH_ERROR General Call: Close L3 Session Request parameter - L3 Session ID 2.6.2. RSVP-DetNet lower API (dlAPI) The RSVP-DetNet lower API shall be lower layer network technology neutral. Sender Call: Add Flow Trossen, et al. Expires April 25, 2021 [Page 11] INTERNET DRAFT DetNet Control Plane Signaling Request parameter - L3 Session ID, Interface, L3 Flow handle, Flow destination IP address, DSCP - Traffic Specification: Maximum IP packet size, Minimum IP packet size, Burstiness, IP packet information rate - One of the Resource Styles: Distinct, Shared, Coordinated Shared Response parameter - Transport Flow Identification Notes for new parameter: The L3 Flow handle is a local parameter to correlate IP Flows to transport flows. The Transport Flow Identifier correlates the IP flow to the lower layer transport flow e.g. TSN Stream ID. Upcall: Flow Response parameter - L3 Session ID, Transport Flow Identification - One of the Info_type: RESV_EVENT, RES_MODIFY_EVENT Receiver Call: Attach Flow Request parameter - L3 Session ID, Interface, L3 Flow handle, Transport Flow Identification, Maximum packet size - Extended Traffic Specification: Maximum expected latency - One of the Info_type: ANNOUNCE_EVENT, ANNOUNCE_MODIFY_EVENT Notes for new parameter: (see notes above) Trossen, et al. Expires April 25, 2021 [Page 12] INTERNET DRAFT DetNet Control Plane Signaling Upcall: Flow Response parameter - L3 Session ID, Transport Flow Identification - One of the Info_type: ANNOUNCE_EVENT, ANNOUNCE_MODIFY_EVENT 2.6.3. Example tAPI on RAP defined by IEEE 802.1Q The following section defines a preliminary interface for RAP (IEEE P802.1Qdd). Currently RAP draft version 0.3 is available and the service interface of RAP is not yet stable. Call: Open L2 Reservation Group Request parameter - Stream destination MAC address (unicast or multicast - VLAN - PCP Response parameter Low-Layer Group ID (local handle) Notes: RAP identifies its session by destination MAC address, VLAN and PCP. Call: Close L2 Reservation Group Request parameter - Low-Layer Group ID Talker Call: Add Stream Request parameter - Low-Layer Group ID Trossen, et al. Expires April 25, 2021 [Page 13] INTERNET DRAFT DetNet Control Plane Signaling - Stream Identification - Traffic Specification Template - Resource Style (Distinct, Shared, Coordinated Shared) - End-System/End-Station source MAC addres - End-System/End-Station destination MAC address (extension for Coordinated Shared) - Sync Domain ID (extension for Coordinated Shared) - PATH Max frame size - Talker Loss Rate Notes: Traffic Specification Template is queue drain algorithm dependent For efficient mapping it has advantages when RAP supports all Resource Styles The Resource Style "Coordinated Shared" allows only one destination and all nodes must belong to the same sync domain PATH MTU frame size is determined downstream from Talker to Listener The Loss Rate is accumulated per Stream. Call: Modify Stream Request parameter - Low-Layer Group ID - Stream Identification, - Traffic Specification Template Call: Release Stream Request parameter - Low-Layer Group ID Trossen, et al. Expires April 25, 2021 [Page 14] INTERNET DRAFT DetNet Control Plane Signaling - Stream Identification Upcall: Stream Response parameter - Low-Layer Group ID, Stream Identification - One of the Info_type: RESV_EVENT, RESV_MODIFY_EVENT Listener Call: Attach Stream Request parameter - Low-Layer Group ID, - Stream Identification - Maximum frame size - Schedule Specification (extension for Coordinated Shared) - Extended Traffic Specification: Maximum expected latency Notes: Schedule Specification includes the schedule for the group of Steams which are coordinated by synchronization Maximum frame size will be delivered upstream from Listener to Talker Call: Attach Modify Stream Request parameter - Low-Layer Group ID - Stream Identification - Schedule Specification (extension only for Coordinated Shared) Trossen, et al. Expires April 25, 2021 [Page 15] INTERNET DRAFT DetNet Control Plane Signaling Call: Release Stream Request parameter - Low-Layer Group ID - Stream Identification Upcall: Stream Response parameter - Low-Layer Group ID - Stream Identification - One of the Info_type: ANNOUNCE_EVENT, ANNOUNCE_MODIFY_EVENT 3. Centralized Control Signaling in SDN Domain For future work. 4. Hybrid Control Signaling in SDN Domain For future work. 5. Security Considerations Editor's note: This section needs more details. 6. IANA Considerations N/A 7. Conclusion This draft outlines the possible control plane signaling in deterministic networking environments for distributed, centralized and hybrid deployments. For the first, we have proposed the introduction of RSVP-detnet for a better alignment of the Layer 3 signaling with that of emerging Layer 2 solutions, together with suggested API specifications for the realization of the L3 to L2 interfaces in endpoints. 8. References Trossen, et al. Expires April 25, 2021 [Page 16] INTERNET DRAFT DetNet Control Plane Signaling 8.1 Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", RFC 8655, DOI 10.17487/RFC8655, October 2019, . [RFC2212] Shenker, S., Partridge, C., and Guerin, R., "Specification of Guaranteed Quality of Service", RFC 2212, September 1997. [RFC2205] R. Braden, L. Zhang, S. Berson, S. Herzog, S. Jasmin, " Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, September 1997. 8.2 Informative References [ID.malis-detnet-controller-plane-framework-04] A. Malis, X. Geng, M. Chen, F. Qin, B. Varga, "Deterministic Networking (DetNet) Controller Plane Framework", draft-malis-detnet- controller-plane-framework-04 (work in progress), 2020. [CHEN-IEEE] F. Chen, F.J. Goetz, M. Kiessling, J. Schmitt, " Support for uStream Aggregation in RAP (ver 0.3)" (work in progess), Jan 2019, [RAP_IEEE] IEEE, "P802.1Qdd - Resource Allocation Protocol", (work in progress), Authors' Addresses Dirk Trossen Huawei Technologies Duesseldorf GmbH Riesstr. 25C 80992 Munich Germany Email: Dirk.Trossen@Huawei.com Trossen, et al. Expires April 25, 2021 [Page 17] INTERNET DRAFT DetNet Control Plane Signaling Franz-Josef Goetz Siemens AG Gleiwitzer-Str. 555 90475 Nuremberg Germany Email: franz-josef.goetz@siemens.com Juergen Schmitt Siemens AG Gleiwitzer Str. 555 90475 Nuremberg Germany Email: juergen.jues.schmitt@siemens.com Trossen, et al. Expires April 25, 2021 [Page 18]