Internet-Draft RESTCONF Pagination Support November 2020
Watsen, et al. Expires 6 May 2021 [Page]
Workgroup:
NETCONF Working Group
Internet-Draft:
draft-wwlh-netconf-list-pagination-rc-01
Published:
Intended Status:
Standards Track
Expires:
Authors:
K. Watsen
Watsen Network
Q. Wu
Huawei
O. Hagsand
Netgate
H. Li
HPE

RESTCONF Extensions to Support List Pagination

Abstract

In some circumstance, a server may contain many instances of a particular YANG list or leaf-list. Retrieval of the entire list or leaf-list at once can be extremely inefficient.

Pagination mechanisms are needed to allow a client to iterate through a large list, in a manner that is most efficient for the application.

This document Updates RFC 8040 to declare "list" and "leaf-list" as resource targets for the RESTCONF operations such as GET, POST operations.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 6 May 2021.

Table of Contents

1. Introduction

There is a need for standard mechanisms to control the filtering, sorting, and retrieval of data from the server. A server may contain many instances of a particular YANG list. Retrieval of the entire list or leaf-list at once can be extremely inefficient.

Pagination mechanisms are needed to allow a client to iterate through a large list, in a manner that is most efficient for the application. The RPC operation defined in [I-D.wwlh-netconf-list-pagination-nc] can be used by the RESTCONF protocol [RFC8040] if the RESTCONF server implements the "ietf-yang-list-pagination" module [I-D.wwlh-netconf-list-pagination-nc].

This document Updates RFC 8040 to declare "list" and "leaf-list" as resource targets for the RESTCONF operations such as GET, POST operations.

1.1. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

The following terms are defined in [RFC8342] [RFC7950] and are not redefined here:

  • server
  • startup configuration datastore
  • candidate configuration datastore
  • running configuration datastore
  • intended configuration datastore
  • operational state datastore
  • conventional configuration datastore
  • datastore schema
  • RPC operation

The following terms are defined in this document as follows:

2. RESTCONF Extensions

This section describes the RESTCONF extensions needed to support the Pagination mechanism.

2.1. RESTCONF Media Type

The following resource type is defined in this document:

               +-----------------+-----------------------------+
               | Resource        | Media Type                  |
               +-----------------+-----------------------------+
               | YANG-Collection | application/yang-data+xml-list |
               | YANG-Collection | application/yang-data+xml-json |
               +-----------------+-----------------------------+
                           Media Type

2.2. New Resource Target

The "get-list-pagination" RPC operation defined in "ietf-yang-list-pagination" module [I-D.wwlh-netconf-list-pagination-nc] can use the RESTCONF target resource URI to identify the resource that will be filtered and returned. A collection resource contains a set of data resources. It is used to represent all instances or a subset of all instances in a YANG list or leaf-list. A collection resource can be used as target resource and retrieved with the GET method or POST method. The "ietf-restconf-list-pagination" YANG module contains the "yang-collection" yang data template with the "yang-data"extension which specifies the syntax and semantics of a "yang-collection" media type.

3. YANG Module for List Pagination

The "ietf-restconf-list-pagination" module defines conceptual definitions within groupings, which are not meant to be implemented as datastore contents by a server.

<CODE BEGINS> file "ietf-restconf-list-pagination@2020-11-02.yang"

=============== NOTE: '\' line wrapping per RFC 8792 ================

module ietf-restconf-list-pagination {
  namespace "urn:ietf:params:xml:ns:yang:ietf-restconf-list-paginati\
on";
  prefix rlpg;

  import ietf-restconf {
    prefix rc;
  }

  organization
    "IETF NETCONF (Network Configuration) Working Group";
  contact
    "WG Web:   <http://tools.ietf.org/wg/netconf/>
     WG List:  <mailto:netconf@ietf.org>";

  description
    "This module contains conceptual YANG specifications
     for the RESTCONF Collection resource type.
     Note that the YANG definitions within this module do not
     represent configuration data of any kind.
     The YANG grouping statements provide a normative syntax
     for XML and JSON message encoding purposes.

     Copyright (c) 2015 IETF Trust and the persons identified as
     authors of the code.  All rights reserved.

     Redistribution and use in source and binary forms, with or
     without modification, is permitted pursuant to, and subject
     to the license terms contained in, the Simplified BSD License
     set forth in Section 4.c of the IETF Trust's Legal Provisions
     Relating to IETF Documents
     (http://trustee.ietf.org/license-info).

     This version of this YANG module is part of RFC XXXX; see
     the RFC itself for full legal notices.";

  revision 2015-01-30 {
    description
      "Initial revision.";
    reference
      "RFC XXXX: RESTCONF Collection Resource.";
  }
  rc:yang-data yang-collection {
    uses collection;
  }

  grouping collection {
    description
      "Conceptual container representing the
       yang-collection resource type.";
    container yang-collection {
      description
        "Container representing the yang-collection
         resource type.";
    }
  }
}

<CODE ENDS>

4. IANA Considerations

4.1. The "IETF XML" Registry

This document registers one URI in the "ns" subregistry of the IETF XML Registry [RFC3688] maintained at https://www.iana.org/assignments/xml-registry/xml-registry.xhtml#ns. Following the format in [RFC3688], the following registration is requested:

  URI: urn:ietf:params:xml:ns:yang:ietf-restconf-list-pagination
  Registrant Contact: The IESG.
  XML: N/A, the requested URI is an XML namespace.

4.2. The "YANG Module Names" Registry

This document registers one YANG module in the YANG Module Names registry [RFC6020] maintained at https://www.iana.org/assignments/yang-parameters/yang-parameters.xhtml. Following the format defined in [RFC6020], the below registration is requested:

  name: ietf-restconf-list-pagination
  namespace: urn:ietf:params:xml:ns:yang:ietf-restconf-list-pagination
  prefix: rlpg
  RFC: xxxx

4.3. The "Media Types" Registry

This document registers two media types in the "application" subregistry of the Media Types registry [RFC6838] [RFC4855] maintained at https://www.iana.org/assignments/media-types/media-types.xhtml#application Following the format defined in [RFC4855], the below registrations are requested:

4.3.1. Media Type "application/yang-collection+xml"

   Type name: application

   Subtype name: yang-collection+xml

   Required parameters: None

   Optional parameters: None

   Encoding considerations: 8-bit
      The "utf-8" charset is always used for this type.  Each
      conceptual YANG data node is encoded according to the
      XML Encoding Rules and Canonical Format for the specific
      YANG data node type defined in RFC 7950.

   Security considerations: Security considerations related
      to the generation and consumption of RESTCONF messages
      are discussed in Section 5 of RFC 8072.  Additional
      security considerations are specific to the semantics
      of particular YANG data models.  Each YANG module is
      expected to specify security considerations for the
      YANG data defined in that module.

   Interoperability considerations: RFC 8072 specifies the
      format of conforming messages and the interpretation
      thereof.

   Published specification: RFC 8072

   Applications that use this media type: Instance document
      data parsers used within a protocol or automation tool
      that utilize the YANG Patch data structure.

   Fragment identifier considerations: The syntax and
      semantics of fragment identifiers are the same
      as the syntax and semantics specified for the
      "application/xml" media type.

   Additional information:

      Deprecated alias names for this type: N/A
      Magic number(s): N/A
      File extension(s): None
      Macintosh file type code(s): "TEXT"

   Person & email address to contact for further
      information: See the Authors' Addresses section
      of RFC 8072.

   Intended usage: COMMON

   Restrictions on usage: N/A

   Author: See the Authors' Addresses section of RFC 8072.

   Change controller: Internet Engineering Task Force
      (mailto:iesg@ietf.org).

   Provisional registration? (standards tree only): no

4.3.2. Media Type "application/yang-data+xml-list"

   Type name: application

   Subtype name: yang-collection+json

   Required parameters: None

   Optional parameters: None

   Encoding considerations: 8-bit
      The "utf-8" charset is always used for this type.
      Each conceptual YANG data node is encoded according to the
      XML Encoding Rules and Canonical Format for the specific
      YANG data node type defined in RFC 7950.

   Security considerations: Security considerations related
      to the generation and consumption of RESTCONF messages
      are discussed in Section 5 of RFC 8072.
      Additional security considerations are specific to the
      semantics of particular YANG data models.  Each YANG module
      is expected to specify security considerations for the
      YANG data defined in that module.

   Interoperability considerations: RFC 8072 specifies the format
      of conforming messages and the interpretation thereof.

   Published specification: RFC 8072

   Applications that use this media type: Instance document
      data parsers used within a protocol or automation tool
      that utilize the YANG Patch data structure.

   Fragment identifier considerations: The syntax and semantics
      of fragment identifiers are the same as the syntax and semantics
      specified for the "application/xml" media type.

   Additional information:

      Deprecated alias names for this type: N/A
      Magic number(s): N/A
      File extension(s): None
      Macintosh file type code(s): "TEXT"

   Person & email address to contact for further information: See
      the Authors' Addresses section of RFC 8072.

   Intended usage: COMMON

   Restrictions on usage: N/A

   Author: See the Authors' Addresses section of RFC 8072.

   Change controller: Internet Engineering Task Force
      (mailto:iesg@ietf.org).

   Provisional registration? (standards tree only): no

5. Security Considerations

5.1. The "ietf-restconf-list-pagination" YANG Module

The YANG module defined in this document extends the base operations for NETCONF [RFC6241] and RESTCONF [RFC8040]. The lowest NETCONF layer is the secure transport layer, and the mandatory-to-implement secure transport is Secure Shell (SSH) [RFC6242]. The lowest RESTCONF layer is HTTPS, and the mandatory-to-implement secure transport is TLS [RFC8446].

The Network Configuration Access Control Model (NACM) [RFC8341] provides the means to restrict access for particular NETCONF users to a preconfigured subset of all available NETCONF protocol operations and content.

The security constraints for the base RESTCONF protocol (see Section 12 of [RFC8040]) apply to the new RESTCONF target resources defined in this document.

6. References

6.1. Normative References

[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
[RFC3688]
Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, DOI 10.17487/RFC3688, , <https://www.rfc-editor.org/info/rfc3688>.
[RFC6020]
Bjorklund, M., Ed., "YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)", RFC 6020, DOI 10.17487/RFC6020, , <https://www.rfc-editor.org/info/rfc6020>.
[RFC6241]
Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, , <https://www.rfc-editor.org/info/rfc6241>.
[RFC6242]
Wasserman, M., "Using the NETCONF Protocol over Secure Shell (SSH)", RFC 6242, DOI 10.17487/RFC6242, , <https://www.rfc-editor.org/info/rfc6242>.
[RFC7950]
Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, , <https://www.rfc-editor.org/info/rfc7950>.
[RFC8040]
Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF Protocol", RFC 8040, DOI 10.17487/RFC8040, , <https://www.rfc-editor.org/info/rfc8040>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
[RFC8342]
Bjorklund, M., Schoenwaelder, J., Shafer, P., Watsen, K., and R. Wilton, "Network Management Datastore Architecture (NMDA)", RFC 8342, DOI 10.17487/RFC8342, , <https://www.rfc-editor.org/info/rfc8342>.

6.2. Informative References

[RFC4855]
Casner, S., "Media Type Registration of RTP Payload Formats", RFC 4855, DOI 10.17487/RFC4855, , <https://www.rfc-editor.org/info/rfc4855>.
[RFC6838]
Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, , <https://www.rfc-editor.org/info/rfc6838>.
[RFC8341]
Bierman, A. and M. Bjorklund, "Network Configuration Access Control Model", STD 91, RFC 8341, DOI 10.17487/RFC8341, , <https://www.rfc-editor.org/info/rfc8341>.
[RFC8446]
Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, , <https://www.rfc-editor.org/info/rfc8446>.

Appendix A. Example YANG Module

The examples within this document use the "example-module" YANG module defined in Appendix A of [I-D.wwlh-netconf-list-pagination-nc].

Appendix B. Example Data

FIXME

Appendix C. Example Queries

C.1. No Query Parameter

Client requests all list entries:

C.1.1. XML

Client requests an XML-based Response:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.1.2. JSON

Client requests a JSON-based Response:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.2. Just the "count" Parameter

In this example, the client requests the first two "skill" resources for a given admin resource.

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.3. Just the "skip" Parameter

In this example, the client requests the next two skills resource, i.e., two skills resource starting from two.

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?skip=2&count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.4. Just the "direction" Parameter

In this example, the client requests the first two "skill" resources in the "forward" direction for a given admin:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?direction=forward&count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.5. Just the "sort" Parameter

In this example, the client requests the first 2 "skill" resources sorted by name for a given admin:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?sort=name&count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.6. Combination of "where" + "count"

In this example, the client requests the first 2 "skill" resources from the selected skill resource list for a given admin:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?where=???&count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.7. Combination of "where" + "skip" + "count"

In this example, the client requests the first 2 "skill" resources from the selected "skill" resource list for a given admin:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?where=???&skip=2&count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

C.8. Combination of "where" + "sort" + "skip","count"

In this example, the client requests the first 2 "skill" resources from the selected skill resources list for a given admin:

=============== NOTE: '\' line wrapping per RFC 8792 ================

GET /restconf/ds/ietf-datastores:running/example-module:admins/admin\
=Bob/skill?where=???&sort=name&skip=2&count=2  HTTP/1.1
Host: example.com
Accept: application/yang-data+xml-list

Response from the RESTCONF server:

HTTP/1.1 200 OK
Date: Thu, 26 Jan 2017 20:56:30 GMT
Server: example-server
Last-Modified: Thu, 26 Jan 2017 20:55:30 GMT
Content-Type: application/yang-data+xml-list

<xml-list xmlns="urn:ietf:params:xml:ns:yang:ietf-list-pagination">
  <skill xmlns="http://example.com/ns/example-module">
    <name>Problem Solving</name>
    <rank>98</rank>
    ...
  </skill>
  <skill xmlns="http://example.com/ns/example-module">
    <name>Conflict Resolution</name>
    <rank>93</rank>
    ...
  </skill>
</xml-list>

Acknowledgements

This work has benefited from the discussions of restconf resource collection over the years, in particular, [I-D.ietf-netconf-restconf-collection] which provides enhanced filtering features for the retrieval of data nodes with the GET method. The authors would like to thank the following for lively discussions on list (ordered by first name): Andy Bierman, Martin Bjoerklund, Robert Varga

Authors' Addresses

Kent Watsen
Watsen Network
Qin Wu
Huawei
101 Software Avenue, Yuhua District
Nanjing
Jiangsu, 210012
China
Olof Hagsand
Netgate
Hongwei Li
HPE