NETCONF Working Group P. Andersson Internet-Draft Cisco Systems Intended status: Standards Track K. Watsen Expires: 2 September 2024 Watsen Networks Q. Wu Huawei Technologies O. Hagsand SUNET H. Li Hewlett Packard Enterprise 1 March 2024 List Pagination Snapshots for YANG-driven Protocols draft-awwhl-netconf-list-pagination-snapshot-00 Abstract List pagination for YANG-driven protocols are defined in [I-D.ietf-netconf-list-pagination]. Operational data can have very large data sets. These data sets can furthermore have big churn, a lot of additions or deletions to the data set. In order to support a stable pagination of such data sets, snapshots can be used. This document defines snapshot support for pagination of "config false" nodes of type "list" and "leaf-list". The snapshot support for individual nodes is signaled via the "ietf-system-capabilities" module. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 2 September 2024. Andersson, et al. Expires 2 September 2024 [Page 1] Internet-Draft List Pagination Snapshots March 2024 Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Conventions . . . . . . . . . . . . . . . . . . . . . . . 4 1.3. Adherence to the NMDA . . . . . . . . . . . . . . . . . . 4 2. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 4 4. Snapshot support . . . . . . . . . . . . . . . . . . . . . . 5 5. The "ietf-list-pagination-snapshot" Module . . . . . . . . . 5 5.1. Data Model Overview . . . . . . . . . . . . . . . . . . . 6 5.2. YANG Module . . . . . . . . . . . . . . . . . . . . . . . 6 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 6.1. The "IETF XML" Registry . . . . . . . . . . . . . . . . . 10 6.2. The "YANG Module Names" Registry . . . . . . . . . . . . 10 6.3. The "RESTCONF Capability URNs" Registry . . . . . . . . . 10 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 7.1. Regarding the "ietf-list-pagination-snapshot" YANG Module . . . . . . . . . . . . . . . . . . . . . . . . . 11 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 8.1. Normative References . . . . . . . . . . . . . . . . . . 11 8.2. Informative References . . . . . . . . . . . . . . . . . 13 Appendix A. Vector Tests . . . . . . . . . . . . . . . . . . . . 14 A.1. Example Data Set . . . . . . . . . . . . . . . . . . . . 14 A.2. Example Queries . . . . . . . . . . . . . . . . . . . . . 14 A.2.1. The "snapshot" Parameter . . . . . . . . . . . . . . 15 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 16 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 1. Introduction The following open questions have been identified for list-pagination with snapshots. Andersson, et al. Expires 2 September 2024 [Page 2] Internet-Draft List Pagination Snapshots March 2024 The requirements that are necessory to resolve for a complete solution: * What should be in the snapshot? The discussions have touched on include entire list content, take a snapshot of list keys etc. * How should a client return to a taken snapshot? I.e. one RESTCONF request starts paginating and allocates a snapshot, how does the client return to that snapshot for the next page? The snapshot would need some id and a method to fetch it later. For instance a new query parameter to identify a snapshot, and a snapshot metadata id? * What is the lifecycle of a snapshot for pagination? * Should the client be able to signal that the snapshot should be deallocated? * Should it the snapshot have some timeout after which it is deallocated? * What happens when a server can't take a snapshot due to resource constraints? * Should snapshots be implicitly deallocated when the pagination reaches the last page? * Security considerations for protecting against DoS when a lot of (possibly huge) snapshots can be taken. 1.1. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. The following terms are defined in [RFC7950] and are not redefined here: client, data model, data tree, feature, extension, module, leaf, leaf-list, and server. Andersson, et al. Expires 2 September 2024 [Page 3] Internet-Draft List Pagination Snapshots March 2024 1.2. Conventions Various examples in this document use "BASE64VALUE=" as a placeholder value for binary data that has been base64 encoded (per Section 9.8 of [RFC7950]). This placeholder value is used because real base64 encoded structures are often many lines long and hence distracting to the example being presented. 1.3. Adherence to the NMDA This document is compliant with the Network Management Datastore Architecture (NMDA) [RFC8342]. The "ietf-list-pagination-snapshot" module only defines a YANG identity, grouping, and augments a couple leafs into a "config false" node defined by the "ietf-system- capabilities" module. 2. Solution Overview The solution presented in this document extends the pagination functionality in [I-D.ietf-netconf-list-pagination]. The snapshot functionality defined by the document conforms to "config false" "list" and "leaf-list" nodes. The "snapshot" query parameter (see Section 3) enables clients to ask create a snapshot. The support for snapshots is signaled via [RFC9196] (see Section 4). 3. The "snapshot" Query Parameter Description The "snapshot" query parameter indicates that the client requests the server to take a snapshot of a "config false" target before starting the pagination. Default Value If this query parameter is unspecified, it defaults to false. Allowed Values The allowed values are true or false. If snapshots are not supported the "snapshot-not-supported" SHOULD be produced in the error-app-tag in the error output. Conformance The "snapshot" query parameter MAY be supported for "config false" lists and leaf-lists. Andersson, et al. Expires 2 September 2024 [Page 4] Internet-Draft List Pagination Snapshots March 2024 3.1. NETCONF For the NETCONF protocol, the "snapshot" query parameter is added to the protocol by augmenting "lpgsnap:snapshot-param-grouping" to the get, get-config, and get-data RPCs. 3.2. RESTCONF The RESTCONF protocol specific functionality and conformance is defined in this section. If the target node does not support snapshots, then a "501 Not Implemented" status-line MUST be returned with the error-type value "application" and error-tag value "invalid-value", and SHOULD also include the "snapshot-not-supported" identity as error-app-tag value. +----------+---------+--------------------------------------------+ | Name | Methods | Description | +----------+---------+--------------------------------------------+ | snapshot | GET, | Indicates that the server should take a | | | HEAD | snapshot before paginating the result set. | +----------+---------+--------------------------------------------+ The "snapshot" query parameter is allowed for GET and HEAD methods on "list" and "leaf-list" data resources. A "400 Bad Request" status- line MUST be returned if used with any other method or resource type. The error-tag value "operation-not-supprted" is used in this case. 4. Snapshot support A server MAY support snapshots when paginating a "config false" list or leaf-list. In order to enable servers to identify which nodes may be used to take snapshots when paginating the "ietf-list-pagination- snapshot" module (see Section 5) augments an empty leaf node called "snapshot" into the "per-node-capabilities" node defined in the "ietf-system-capabilities" module (see [RFC9196]). Note that it is possible for a client to request the server to take a snapshot when paginating with the "snapshot" query parameter (see Section 3. 5. The "ietf-list-pagination-snapshot" Module The "ietf-list-pagination-snapshot" module is used by servers to indicate that they support pagination on YANG "list" and "leaf-list" nodes, and to provide an ability to indicate which "config false" list and/or "leaf-list" nodes are constrained and, if so, which nodes may be used in "where" and "sort-by" expressions. Andersson, et al. Expires 2 September 2024 [Page 5] Internet-Draft List Pagination Snapshots March 2024 5.1. Data Model Overview The following tree diagram [RFC8340] illustrates the "ietf-list- pagination-snapshot" module: module: ietf-list-pagination-snapshot augment /nc:get/nc:input: +---w snapshot? boolean augment /nc:get-config/nc:input: +---w snapshot? boolean augment /ncds:get-data/ncds:input: +---w snapshot? boolean augment /sysc:system-capabilities/sysc:datastore-capabilities /sysc:per-node-capabilities: +--ro snapshot? empty Comments: As shown, this module augments an optional leaf into the "per-node- capabilities" list node of the "ietf-system-capabilities" module. 5.2. YANG Module This YANG module has normative references to [RFC7952] and [RFC9196]. file "ietf-list-pagination-snapshot@2024-03-01.yang" module ietf-list-pagination-snapshot { yang-version 1.1; namespace "urn:ietf:params:xml:ns:yang:ietf-list-pagination-snapshot"; prefix lpgsnap; import ietf-datastores { prefix ds; reference "RFC 8342: Network Management Datastore Architecture (NMDA)"; } import ietf-netconf { prefix nc; reference "RFC 6241: Network Configuration Protocol (NETCONF)"; } import ietf-netconf-nmda { prefix ncds; Andersson, et al. Expires 2 September 2024 [Page 6] Internet-Draft List Pagination Snapshots March 2024 reference "RFC 8526: NETCONF Extensions to Support the Network Management Datastore Architecture"; } import ietf-system-capabilities { prefix sysc; reference "RFC 9691: YANG Modules Describing Capabilities for Systems and Datastore Update Notifications"; } import ietf-list-pagination { prefix lpg; reference "draft-ietf-list-pagination: List Pagination for YANG-driven Protocols"; } organization "IETF NETCONF (Network Configuration) Working Group"; contact "WG Web: https://datatracker.ietf.org/wg/netconf WG List: NETCONF WG list "; description "This module is used by servers to indicate they support snapshot pagination on 'config false' nodes of type 'list' and 'leaf-list'. It also defines a grouping for the snapshot parameter. Copyright (c) 2024 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Revised BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX (https://www.rfc-editor.org/info/rfcXXXX); see the RFC itself for full legal notices. The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', Andersson, et al. Expires 2 September 2024 [Page 7] Internet-Draft List Pagination Snapshots March 2024 'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document are to be interpreted as described in BCP 14 (RFC 2119) (RFC 8174) when, and only when, they appear in all capitals, as shown here."; revision 2024-03-01 { description "Initial revision."; reference "RFC XXXX: List Pagination Snapshots for YANG-driven Protocols"; } // Identities identity snapshot-not-supported { base lpg:list-pagination-error; description "Snapshot is not supported for the target. Either it is not a 'config false' list or leaf-list, or it is disabled."; } // Groupings grouping snapshot-param-grouping { description "This grouping may be used by protocol-specific YANG modules to define a protocol-specific query parameter."; leaf snapshot { type boolean; description "The 'snapshot' parameter indicates that the client requests the server to take a snapshot of the 'config false' list or leaf-list target before paginating."; } } // Protocol-accessible nodes augment "/nc:get/nc:input" { description "Allow the 'get' operation to use the 'snapshot' query parameter for YANG list or leaf-list that is to be retrieved."; uses snapshot-param-grouping; } Andersson, et al. Expires 2 September 2024 [Page 8] Internet-Draft List Pagination Snapshots March 2024 augment "/nc:get-config/nc:input" { description "Allow the 'get-config' operation to use the 'snapshot' query parameter for YANG list or leaf-list that is to be retrieved."; uses snapshot-param-grouping; } augment "/ncds:get-data/ncds:input" { description "Allow the 'get-data' operation to use the 'snapshot' query parameter for YANG list or leaf-list that is to be retrieved."; uses snapshot-param-grouping; } augment "/sysc:system-capabilities/sysc:datastore-capabilities" + "/sysc:per-node-capabilities" { // Ensure the following node is only used for the // datastore. when "/sysc:system-capabilities/sysc:datastore-capabilities" + "/sysc:datastore = 'ds:operational'"; description "Defines some leafs that MAY be used by the server to describe constraints imposed of the 'where' filters and 'sort-by' parameters used in list pagination queries."; leaf snapshot { type empty; description "Indicates that snapshots are supported for the targeted 'config false' list or leaf-list node."; } } } 6. IANA Considerations Andersson, et al. Expires 2 September 2024 [Page 9] Internet-Draft List Pagination Snapshots March 2024 6.1. The "IETF XML" Registry This document registers one URI in the "ns" subregistry of the IETF XML Registry [RFC3688] maintained at https://www.iana.org/assignments/xml-registry/xml-registry.xhtml#ns. Following the format in [RFC3688], the following registration is requested: URI: urn:ietf:params:xml:ns:yang:ietf-list-pagination-snapshot Registrant Contact: The IESG. XML: N/A, the requested URI is an XML namespace. 6.2. The "YANG Module Names" Registry This document registers one YANG module in the YANG Module Names registry [RFC6020] maintained at https://www.iana.org/assignments/ yang-parameters/yang-parameters.xhtml. Following the format defined in [RFC6020], the below registration is requested: name: ietf-list-pagination-snapshot namespace: urn:ietf:params:xml:ns:yang:ietf-list-pagination-snapshot prefix: lpg RFC: XXXX 6.3. The "RESTCONF Capability URNs" Registry This document registers one capability in the RESTCONF Capability URNs [RFC8040] maintained at https://www.iana.org/assignments/ restconf-capability-urns/restconf-capability-urns.xhtml. Following the instructions defined in Section 11.4 of [RFC8040], the below registrations are requested: All the registrations are to use this document (RFC XXXX) for the "Reference" value. Index Capability Identifier --------------------------------------------------------------------- :snapshot urn:ietf:params:restconf:capability:snapshot:1.0 7. Security Considerations Andersson, et al. Expires 2 September 2024 [Page 10] Internet-Draft List Pagination Snapshots March 2024 7.1. Regarding the "ietf-list-pagination-snapshot" YANG Module The YANG module specified in this document defines a schema for data that is designed to be accessed via network management protocols such as NETCONF [RFC6241] or RESTCONF [RFC8040]. The lowest NETCONF layer is the secure transport layer, and the mandatory-to-implement secure transport is Secure Shell (SSH) [RFC6242]. The lowest RESTCONF layer is HTTPS, and the mandatory-to-implement secure transport is TLS [RFC8446]. The Network Configuration Access Control Model (NACM) [RFC8341] provides the means to restrict access for particular NETCONF or RESTCONF users to a preconfigured subset of all available NETCONF or RESTCONF protocol operations and content. All protocol-accessible data nodes in the extension to "ietf-system- capabilities" module are read-only and cannot be modified. Access control may be configured to avoid exposing any read-only data that is defined by the augmenting module documentation as being security sensitive. The security considerations for the base NETCONF protocol operations (see Section 9 of [RFC6241] and Section 6 of [RFC8526]) apply to the extension made to operations , , and defined in this document. 8. References 8.1. Normative References [I-D.ietf-netconf-list-pagination] Watsen, K., Wu, Q., Andersson, P., Hagsand, O., and H. Li, "List Pagination for YANG-driven Protocols", Work in Progress, Internet-Draft, draft-ietf-netconf-list- pagination-03, 1 March 2024, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3688] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, DOI 10.17487/RFC3688, January 2004, . Andersson, et al. Expires 2 September 2024 [Page 11] Internet-Draft List Pagination Snapshots March 2024 [RFC5646] Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying Languages", BCP 47, RFC 5646, DOI 10.17487/RFC5646, September 2009, . [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, . [RFC6242] Wasserman, M., "Using the NETCONF Protocol over Secure Shell (SSH)", RFC 6242, DOI 10.17487/RFC6242, June 2011, . [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, August 2016, . [RFC7952] Lhotka, L., "Defining and Using Metadata with YANG", RFC 7952, DOI 10.17487/RFC7952, August 2016, . [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8341] Bierman, A. and M. Bjorklund, "Network Configuration Access Control Model", STD 91, RFC 8341, DOI 10.17487/RFC8341, March 2018, . [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, . [RFC8526] Bjorklund, M., Schoenwaelder, J., Shafer, P., Watsen, K., and R. Wilton, "NETCONF Extensions to Support the Network Management Datastore Architecture", RFC 8526, DOI 10.17487/RFC8526, March 2019, . [RFC9196] Lengyel, B., Clemm, A., and B. Claise, "YANG Modules Describing Capabilities for Systems and Datastore Update Notifications", RFC 9196, DOI 10.17487/RFC9196, February 2022, . Andersson, et al. Expires 2 September 2024 [Page 12] Internet-Draft List Pagination Snapshots March 2024 8.2. Informative References [I-D.ietf-netconf-list-pagination-nc] Watsen, K., Wu, Q., Hagsand, O., Li, H., and P. Andersson, "NETCONF Extensions to Support List Pagination", Work in Progress, Internet-Draft, draft-ietf-netconf-list- pagination-nc-02, 22 October 2023, . [I-D.ietf-netconf-list-pagination-rc] Watsen, K., Wu, Q., Hagsand, O., Li, H., and P. Andersson, "RESTCONF Extensions to Support List Pagination", Work in Progress, Internet-Draft, draft-ietf-netconf-list- pagination-rc-02, 22 October 2023, . [I-D.ietf-netconf-restconf-collection] Bierman, A., Björklund, M., and K. Watsen, "RESTCONF Collection Resource", Work in Progress, Internet-Draft, draft-ietf-netconf-restconf-collection-00, 30 January 2015, . [RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)", RFC 6020, DOI 10.17487/RFC6020, October 2010, . [RFC6365] Hoffman, P. and J. Klensin, "Terminology Used in Internationalization in the IETF", BCP 166, RFC 6365, DOI 10.17487/RFC6365, September 2011, . [RFC8340] Bjorklund, M. and L. Berger, Ed., "YANG Tree Diagrams", BCP 215, RFC 8340, DOI 10.17487/RFC8340, March 2018, . [RFC8342] Bjorklund, M., Schoenwaelder, J., Shafer, P., Watsen, K., and R. Wilton, "Network Management Datastore Architecture (NMDA)", RFC 8342, DOI 10.17487/RFC8342, March 2018, . [RFC8525] Bierman, A., Bjorklund, M., Schoenwaelder, J., Watsen, K., and R. Wilton, "YANG Library", RFC 8525, DOI 10.17487/RFC8525, March 2019, . Andersson, et al. Expires 2 September 2024 [Page 13] Internet-Draft List Pagination Snapshots March 2024 Appendix A. Vector Tests This normative appendix section illustrates every notable edge condition conceived during this document's production. Test inputs and outputs are provided in a manner that is both generic and concise. Management protocol specific documents need only reproduce as many of these tests as necessary to convey pecularities presented by the protocol. Implementations are RECOMMENDED to implement the tests presented in this document, in addition to any tests that may be presented in protocol specific documents. The vector tests assume the "example-social" YANG module and example data set defined [I-D.ietf-netconf-list-pagination]. A.1. Example Data Set The examples assume the server's operational state as follows. The following data enables snapshot support for the audit-log list node. ds:operational /es:audit-logs/es:audit-log A.2. Example Queries The following sections present example queries for the the snapshot query parameter. All the vector tests are presented in a protocol-independent manner. JSON is used only for its conciseness. Andersson, et al. Expires 2 September 2024 [Page 14] Internet-Draft List Pagination Snapshots March 2024 A.2.1. The "snapshot" Parameter The "snapshot" parameter may be used on "config false" target nodes. | If this parameter is omitted, the default value is false. REQUEST Target: /example-social:audit-logs/audit-log Pagination Parameters: Where: - Sort-by: - Direction: - Offset: - Limit: - Snapshot: true RESPONSE Andersson, et al. Expires 2 September 2024 [Page 15] Internet-Draft List Pagination Snapshots March 2024 { "example-social:audit-log": [ { "member-id": "alice", ... }, { "member-id": "bob", ... }, { "member-id": "eric", ... }, { "member-id": "alice", ... }, { "member-id": "bob", ... }, { "member-id": "alice", ... }, { "member-id": "bob", ... } ] } Acknowledgements The authors would like to thank the following for lively discussions on list (ordered by first name): Andy Bierman, Tom Petch, and Quifang Ma. Authors' Addresses Per Andersson Cisco Systems Email: perander@cisco.com Kent Watsen Watsen Networks Andersson, et al. Expires 2 September 2024 [Page 16] Internet-Draft List Pagination Snapshots March 2024 Email: kent+ietf@watsen.net Qin Wu Huawei Technologies Email: bill.wu@huawei.com Olof Hagsand SUNET Email: olof@hagsand.se Hongwei Li Hewlett Packard Enterprise Email: flycoolman@gmail.com Andersson, et al. Expires 2 September 2024 [Page 17]