NETCONF Working Group K. Watsen Internet-Draft Watsen Network Intended status: Standards Track Q. Wu Expires: May 6, 2021 Huawei O. Hagsand Netgate H. Li HPE November 2, 2020 RESTCONF Extensions to Support List Pagination draft-wwlh-netconf-list-pagination-rc-01 Abstract In some circumstance, a server may contain many instances of a particular YANG list or leaf-list. Retrieval of the entire list or leaf-list at once can be extremely inefficient. Pagination mechanisms are needed to allow a client to iterate through a large list, in a manner that is most efficient for the application. This document Updates RFC 8040 to declare "list" and "leaf-list" as resource targets for the RESTCONF operations such as GET, POST operations. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 6, 2021. Watsen, et al. Expires May 6, 2021 [Page 1] Internet-Draft RESTCONF Pagination Support November 2020 Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 2. RESTCONF Extensions . . . . . . . . . . . . . . . . . . . . . 4 2.1. RESTCONF Media Type . . . . . . . . . . . . . . . . . . . 4 2.2. New Resource Target . . . . . . . . . . . . . . . . . . . 4 3. YANG Module for List Pagination . . . . . . . . . . . . . . . 4 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 4.1. The "IETF XML" Registry . . . . . . . . . . . . . . . . . 6 4.2. The "YANG Module Names" Registry . . . . . . . . . . . . 6 4.3. The "Media Types" Registry . . . . . . . . . . . . . . . 6 4.3.1. Media Type "application/yang-collection+xml" . . . . 6 4.3.2. Media Type "application/yang-data+xml-list" . . . . . 8 5. Security Considerations . . . . . . . . . . . . . . . . . . . 9 5.1. The "ietf-restconf-list-pagination" YANG Module . . . . . 9 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 6.1. Normative References . . . . . . . . . . . . . . . . . . 9 6.2. Informative References . . . . . . . . . . . . . . . . . 10 Appendix A. Example YANG Module . . . . . . . . . . . . . . . . 12 Appendix B. Example Data . . . . . . . . . . . . . . . . . . . . 12 Appendix C. Example Queries . . . . . . . . . . . . . . . . . . 12 C.1. No Query Parameter . . . . . . . . . . . . . . . . . . . 12 C.1.1. XML . . . . . . . . . . . . . . . . . . . . . . . . . 12 C.1.2. JSON . . . . . . . . . . . . . . . . . . . . . . . . 13 C.2. Just the "count" Parameter . . . . . . . . . . . . . . . 13 C.3. Just the "skip" Parameter . . . . . . . . . . . . . . . . 14 C.4. Just the "direction" Parameter . . . . . . . . . . . . . 15 C.5. Just the "sort" Parameter . . . . . . . . . . . . . . . . 16 C.6. Combination of "where" + "count" . . . . . . . . . . . . 17 C.7. Combination of "where" + "skip" + "count" . . . . . . . . 18 C.8. Combination of "where" + "sort" + "skip","count" . . . . 19 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 20 Watsen, et al. Expires May 6, 2021 [Page 2] Internet-Draft RESTCONF Pagination Support November 2020 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 1. Introduction There is a need for standard mechanisms to control the filtering, sorting, and retrieval of data from the server. A server may contain many instances of a particular YANG list. Retrieval of the entire list or leaf-list at once can be extremely inefficient. Pagination mechanisms are needed to allow a client to iterate through a large list, in a manner that is most efficient for the application. The RPC operation defined in [I-D.wwlh-netconf-list-pagination-nc] can be used by the RESTCONF protocol [RFC8040] if the RESTCONF server implements the "ietf-yang-list-pagination" module [I-D.wwlh-netconf- list-pagination-nc]. This document Updates RFC 8040 to declare "list" and "leaf-list" as resource targets for the RESTCONF operations such as GET, POST operations. 1.1. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. The following terms are defined in [RFC8342] [RFC7950] and are not redefined here: o server o startup configuration datastore o candidate configuration datastore o running configuration datastore o intended configuration datastore o operational state datastore o conventional configuration datastore o datastore schema o RPC operation Watsen, et al. Expires May 6, 2021 [Page 3] Internet-Draft RESTCONF Pagination Support November 2020 The following terms are defined in this document as follows: 2. RESTCONF Extensions This section describes the RESTCONF extensions needed to support the Pagination mechanism. 2.1. RESTCONF Media Type The following resource type is defined in this document: +-----------------+-----------------------------+ | Resource | Media Type | +-----------------+-----------------------------+ | YANG-Collection | application/yang-data+xml-list | | YANG-Collection | application/yang-data+xml-json | +-----------------+-----------------------------+ Media Type 2.2. New Resource Target The "get-list-pagination" RPC operation defined in "ietf-yang-list- pagination" module [I-D.wwlh-netconf-list-pagination-nc] can use the RESTCONF target resource URI to identify the resource that will be filtered and returned. A collection resource contains a set of data resources. It is used to represent all instances or a subset of all instances in a YANG list or leaf-list. A collection resource can be used as target resource and retrieved with the GET method or POST method. The "ietf-restconf-list-pagination" YANG module contains the "yang-collection" yang data template with the "yang-data"extension which specifies the syntax and semantics of a "yang-collection" media type. 3. YANG Module for List Pagination The "ietf-restconf-list-pagination" module defines conceptual definitions within groupings, which are not meant to be implemented as datastore contents by a server. file "ietf-restconf-list-pagination@2020-11-02.yang" =============== NOTE: '\' line wrapping per RFC 8792 ================ module ietf-restconf-list-pagination { namespace "urn:ietf:params:xml:ns:yang:ietf-restconf-list-paginati\ on"; prefix rlpg; Watsen, et al. Expires May 6, 2021 [Page 4] Internet-Draft RESTCONF Pagination Support November 2020 import ietf-restconf { prefix rc; } organization "IETF NETCONF (Network Configuration) Working Group"; contact "WG Web: WG List: "; description "This module contains conceptual YANG specifications for the RESTCONF Collection resource type. Note that the YANG definitions within this module do not represent configuration data of any kind. The YANG grouping statements provide a normative syntax for XML and JSON message encoding purposes. Copyright (c) 2015 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Simplified BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX; see the RFC itself for full legal notices."; revision 2015-01-30 { description "Initial revision."; reference "RFC XXXX: RESTCONF Collection Resource."; } rc:yang-data yang-collection { uses collection; } grouping collection { description "Conceptual container representing the yang-collection resource type."; container yang-collection { description "Container representing the yang-collection Watsen, et al. Expires May 6, 2021 [Page 5] Internet-Draft RESTCONF Pagination Support November 2020 resource type."; } } } 4. IANA Considerations 4.1. The "IETF XML" Registry This document registers one URI in the "ns" subregistry of the IETF XML Registry [RFC3688] maintained at . Following the format in [RFC3688], the following registration is requested: URI: urn:ietf:params:xml:ns:yang:ietf-restconf-list-pagination Registrant Contact: The IESG. XML: N/A, the requested URI is an XML namespace. 4.2. The "YANG Module Names" Registry This document registers one YANG module in the YANG Module Names registry [RFC6020] maintained at . Following the format defined in [RFC6020], the below registration is requested: name: ietf-restconf-list-pagination namespace: urn:ietf:params:xml:ns:yang:ietf-restconf-list-pagination prefix: rlpg RFC: xxxx 4.3. The "Media Types" Registry This document registers two media types in the "application" subregistry of the Media Types registry [RFC6838] [RFC4855] maintained at Following the format defined in [RFC4855], the below registrations are requested: 4.3.1. Media Type "application/yang-collection+xml" Type name: application Subtype name: yang-collection+xml Watsen, et al. Expires May 6, 2021 [Page 6] Internet-Draft RESTCONF Pagination Support November 2020 Required parameters: None Optional parameters: None Encoding considerations: 8-bit The "utf-8" charset is always used for this type. Each conceptual YANG data node is encoded according to the XML Encoding Rules and Canonical Format for the specific YANG data node type defined in RFC 7950. Security considerations: Security considerations related to the generation and consumption of RESTCONF messages are discussed in Section 5 of RFC 8072. Additional security considerations are specific to the semantics of particular YANG data models. Each YANG module is expected to specify security considerations for the YANG data defined in that module. Interoperability considerations: RFC 8072 specifies the format of conforming messages and the interpretation thereof. Published specification: RFC 8072 Applications that use this media type: Instance document data parsers used within a protocol or automation tool that utilize the YANG Patch data structure. Fragment identifier considerations: The syntax and semantics of fragment identifiers are the same as the syntax and semantics specified for the "application/xml" media type. Additional information: Deprecated alias names for this type: N/A Magic number(s): N/A File extension(s): None Macintosh file type code(s): "TEXT" Person & email address to contact for further information: See the Authors' Addresses section of RFC 8072. Intended usage: COMMON Restrictions on usage: N/A Watsen, et al. Expires May 6, 2021 [Page 7] Internet-Draft RESTCONF Pagination Support November 2020 Author: See the Authors' Addresses section of RFC 8072. Change controller: Internet Engineering Task Force (mailto:iesg@ietf.org). Provisional registration? (standards tree only): no 4.3.2. Media Type "application/yang-data+xml-list" Type name: application Subtype name: yang-collection+json Required parameters: None Optional parameters: None Encoding considerations: 8-bit The "utf-8" charset is always used for this type. Each conceptual YANG data node is encoded according to the XML Encoding Rules and Canonical Format for the specific YANG data node type defined in RFC 7950. Security considerations: Security considerations related to the generation and consumption of RESTCONF messages are discussed in Section 5 of RFC 8072. Additional security considerations are specific to the semantics of particular YANG data models. Each YANG module is expected to specify security considerations for the YANG data defined in that module. Interoperability considerations: RFC 8072 specifies the format of conforming messages and the interpretation thereof. Published specification: RFC 8072 Applications that use this media type: Instance document data parsers used within a protocol or automation tool that utilize the YANG Patch data structure. Fragment identifier considerations: The syntax and semantics of fragment identifiers are the same as the syntax and semantics specified for the "application/xml" media type. Additional information: Deprecated alias names for this type: N/A Magic number(s): N/A Watsen, et al. Expires May 6, 2021 [Page 8] Internet-Draft RESTCONF Pagination Support November 2020 File extension(s): None Macintosh file type code(s): "TEXT" Person & email address to contact for further information: See the Authors' Addresses section of RFC 8072. Intended usage: COMMON Restrictions on usage: N/A Author: See the Authors' Addresses section of RFC 8072. Change controller: Internet Engineering Task Force (mailto:iesg@ietf.org). Provisional registration? (standards tree only): no 5. Security Considerations 5.1. The "ietf-restconf-list-pagination" YANG Module The YANG module defined in this document extends the base operations for NETCONF [RFC6241] and RESTCONF [RFC8040]. The lowest NETCONF layer is the secure transport layer, and the mandatory-to-implement secure transport is Secure Shell (SSH) [RFC6242]. The lowest RESTCONF layer is HTTPS, and the mandatory-to-implement secure transport is TLS [RFC8446]. The Network Configuration Access Control Model (NACM) [RFC8341] provides the means to restrict access for particular NETCONF users to a preconfigured subset of all available NETCONF protocol operations and content. The security constraints for the base RESTCONF protocol (see Section 12 of [RFC8040]) apply to the new RESTCONF target resources defined in this document. 6. References 6.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . Watsen, et al. Expires May 6, 2021 [Page 9] Internet-Draft RESTCONF Pagination Support November 2020 [RFC3688] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, DOI 10.17487/RFC3688, January 2004, . [RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)", RFC 6020, DOI 10.17487/RFC6020, October 2010, . [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, . [RFC6242] Wasserman, M., "Using the NETCONF Protocol over Secure Shell (SSH)", RFC 6242, DOI 10.17487/RFC6242, June 2011, . [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, August 2016, . [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8342] Bjorklund, M., Schoenwaelder, J., Shafer, P., Watsen, K., and R. Wilton, "Network Management Datastore Architecture (NMDA)", RFC 8342, DOI 10.17487/RFC8342, March 2018, . 6.2. Informative References [RFC4855] Casner, S., "Media Type Registration of RTP Payload Formats", RFC 4855, DOI 10.17487/RFC4855, February 2007, . [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013, . Watsen, et al. Expires May 6, 2021 [Page 10] Internet-Draft RESTCONF Pagination Support November 2020 [RFC8341] Bierman, A. and M. Bjorklund, "Network Configuration Access Control Model", STD 91, RFC 8341, DOI 10.17487/RFC8341, March 2018, . [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, . Watsen, et al. Expires May 6, 2021 [Page 11] Internet-Draft RESTCONF Pagination Support November 2020 Appendix A. Example YANG Module The examples within this document use the "example-module" YANG module defined in Appendix A of [I-D.wwlh-netconf-list-pagination- nc]. Appendix B. Example Data FIXME Appendix C. Example Queries C.1. No Query Parameter Client requests all list entries: C.1.1. XML Client requests an XML-based Response: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... Watsen, et al. Expires May 6, 2021 [Page 12] Internet-Draft RESTCONF Pagination Support November 2020 C.1.2. JSON Client requests a JSON-based Response: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.2. Just the "count" Parameter In this example, the client requests the first two "skill" resources for a given admin resource. =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 13] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.3. Just the "skip" Parameter In this example, the client requests the next two skills resource, i.e., two skills resource starting from two. =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?skip=2&count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 14] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.4. Just the "direction" Parameter In this example, the client requests the first two "skill" resources in the "forward" direction for a given admin: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?direction=forward&count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 15] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.5. Just the "sort" Parameter In this example, the client requests the first 2 "skill" resources sorted by name for a given admin: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?sort=name&count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 16] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.6. Combination of "where" + "count" In this example, the client requests the first 2 "skill" resources from the selected skill resource list for a given admin: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?where=???&count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 17] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.7. Combination of "where" + "skip" + "count" In this example, the client requests the first 2 "skill" resources from the selected "skill" resource list for a given admin: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?where=???&skip=2&count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 18] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... C.8. Combination of "where" + "sort" + "skip","count" In this example, the client requests the first 2 "skill" resources from the selected skill resources list for a given admin: =============== NOTE: '\' line wrapping per RFC 8792 ================ GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\ =Bob/skill?where=???&sort=name&skip=2&count=2 HTTP/1.1 Host: example.com Accept: application/yang-data+xml-list Response from the RESTCONF server: Watsen, et al. Expires May 6, 2021 [Page 19] Internet-Draft RESTCONF Pagination Support November 2020 HTTP/1.1 200 OK Date: Thu, 26 Jan 2017 20:56:30 GMT Server: example-server Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT Content-Type: application/yang-data+xml-list Problem Solving 98 ... Conflict Resolution 93 ... Acknowledgements This work has benefited from the discussions of restconf resource collection over the years, in particular, [I-D.ietf-netconf-restconf- collection] which provides enhanced filtering features for the retrieval of data nodes with the GET method. The authors would like to thank the following for lively discussions on list (ordered by first name): Andy Bierman, Martin Bjoerklund, Robert Varga Authors' Addresses Kent Watsen Watsen Network EMail: kent+ietf@watsen.net Qin Wu Huawei 101 Software Avenue, Yuhua District Nanjing, Jiangsu 210012 China EMail: bill.wu@huawei.com Watsen, et al. Expires May 6, 2021 [Page 20] Internet-Draft RESTCONF Pagination Support November 2020 Olof Hagsand Netgate EMail: olof@hagsand.se Hongwei Li HPE EMail: flycoolman@gmail.com Watsen, et al. Expires May 6, 2021 [Page 21]