JSONPath WG S. Gössner, Ed. Internet-Draft Fachhochschule Dortmund Intended status: Standards Track G. Normington, Ed. Expires: 13 July 2023 C. Bormann, Ed. Universität Bremen TZI 9 January 2023 JSONPath: Query expressions for JSON draft-ietf-jsonpath-base-09 Abstract JSONPath defines a string syntax for selecting and extracting JSON (RFC 8259) values from a JSON value. About This Document This note is to be removed before publishing as an RFC. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-ietf-jsonpath-base/. Discussion of this document takes place on the JSON Path Working Group mailing list (mailto:jsonpath@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/jsonpath/. Subscribe at https://www.ietf.org/mailman/listinfo/jsonpath/. Source for this draft and an issue tracker can be found at https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 13 July 2023. Gössner, et al. Expires 13 July 2023 [Page 1] Internet-Draft JSONPath January 2023 Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.1. JSON Values as Trees of Nodes . . . . . . . . . . . . 6 1.2. History . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3. JSON Values . . . . . . . . . . . . . . . . . . . . . . . 7 1.4. Overview of JSONPath Expressions . . . . . . . . . . . . 7 1.4.1. Identifiers . . . . . . . . . . . . . . . . . . . . . 7 1.4.2. Segments . . . . . . . . . . . . . . . . . . . . . . 8 1.4.3. Selectors . . . . . . . . . . . . . . . . . . . . . . 8 1.4.4. Summary . . . . . . . . . . . . . . . . . . . . . . . 8 1.5. JSONPath Examples . . . . . . . . . . . . . . . . . . . . 10 2. JSONPath Syntax and Semantics . . . . . . . . . . . . . . . . 11 2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3. Semantics . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4. Root Identifier . . . . . . . . . . . . . . . . . . . . . 14 2.5. Selectors . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5.1. Name Selector . . . . . . . . . . . . . . . . . . . . 15 2.5.2. Wildcard Selector . . . . . . . . . . . . . . . . . . 18 2.5.3. Index selector . . . . . . . . . . . . . . . . . . . 20 2.5.4. Array Slice selector . . . . . . . . . . . . . . . . 21 2.5.5. Filter selector . . . . . . . . . . . . . . . . . . . 25 2.6. Function Extensions . . . . . . . . . . . . . . . . . . . 33 2.6.1. Type System for Function Expressions . . . . . . . . 34 2.6.2. Type Correctness of Function Expressions . . . . . . 36 2.6.3. length Function Extension . . . . . . . . . . . . . . 37 2.6.4. count Function Extension . . . . . . . . . . . . . . 38 2.6.5. match Function Extension . . . . . . . . . . . . . . 38 2.6.6. search Function Extension . . . . . . . . . . . . . . 38 2.7. Segments . . . . . . . . . . . . . . . . . . . . . . . . 40 2.7.1. Child Segment . . . . . . . . . . . . . . . . . . . . 40 2.7.2. Descendant Segment . . . . . . . . . . . . . . . . . 42 Gössner, et al. Expires 13 July 2023 [Page 2] Internet-Draft JSONPath January 2023 2.8. Semantics of null . . . . . . . . . . . . . . . . . . . . 45 2.9. Normalized Paths . . . . . . . . . . . . . . . . . . . . 46 3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 48 3.1. Registration of Media Type application/jsonpath . . . . . 48 3.2. Function Extensions . . . . . . . . . . . . . . . . . . . 49 4. Security Considerations . . . . . . . . . . . . . . . . . . . 51 4.1. Attack Vectors on JSONPath Implementations . . . . . . . 51 4.2. Attack Vectors on How JSONPath Queries are Formed . . . . 51 4.3. Attacks on Security Mechanisms that Employ JSONPath . . . 52 5. References . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.1. Normative References . . . . . . . . . . . . . . . . . . 52 5.2. Informative References . . . . . . . . . . . . . . . . . 53 Appendix A. Inspired by XPath . . . . . . . . . . . . . . . . . 54 A.1. JSONPath and XPath . . . . . . . . . . . . . . . . . . . 55 Appendix B. JSON Pointer . . . . . . . . . . . . . . . . . . . . 58 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 58 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 59 1. Introduction JSON [RFC8259] is a popular representation format for structured data values. JSONPath defines a string syntax for selecting and extracting JSON values from a JSON value. JSONPath is not intended as a replacement for, but as a more powerful companion to, JSON Pointer [RFC6901]. See Appendix B. 1.1. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. The grammatical rules in this document are to be interpreted as ABNF, as described in [RFC5234]. ABNF terminal values in this document define Unicode code points rather than their UTF-8 encoding. For example, the Unicode PLACE OF INTEREST SIGN (U+2318) would be defined in ABNF as %x2318. Gössner, et al. Expires 13 July 2023 [Page 3] Internet-Draft JSONPath January 2023 The terminology of [RFC8259] applies except where clarified below. The terms "Primitive" and "Structured" are used to group different kinds of values as in Section 1 of [RFC8259]; JSON Objects and Arrays are structured, all other values are primitive. Definitions for "Object", "Array", "Number", and "String" remain unchanged. Importantly "object" and "array" in particular do not take on a generic meaning, such as they would in a general programming context. Additional terms used in this document are defined below. Value: As per [RFC8259], a structure conforming to the generic data model of JSON, i.e., composed of components such as structured values, namely JSON objects and arrays, and primitive data, namely numbers and text strings as well as the special values null, true, and false. [RFC8259] focuses on the textual representation of JSON values and does not fully define the value abstraction assumed here. Member: A name/value pair in an object. (A member is not itself a value.) Name: The name (a string) in a name/value pair constituting a member. This is also used in [RFC8259], but that specification does not formally define it. It is included here for completeness. Element: A value in a JSON array. Index: An integer that identifies a specific element in an array. Query: Short name for a JSONPath expression. Argument: Short name for the value a JSONPath expression is applied to. Location: the position of a value within the argument. This can be thought of as a sequence of names and indexes navigating to the value through the objects and arrays in the argument, with the empty sequence indicating the argument itself. A location can be represented as a Normalized Path (defined below). Node: The pair of a value along with its location within the argument. Root Node: The unique node whose value is the entire argument. Root Node Identifier: The expression $ which refers to the root node of the argument. Gössner, et al. Expires 13 July 2023 [Page 4] Internet-Draft JSONPath January 2023 Current Node Identifier: The expression @ which refers to the current node in the context of the evaluation of a filter expression (described later). Children (of a node): If the node is an array, the nodes of its elements. If the node is an object, the nodes of its member values. If the node is neither an array nor an object, it has no children. Descendants (of a node): The children of the node, together with the children of its children, and so forth recursively. More formally, the descendants relation between nodes is the transitive closure of the children relation. Depth (of a descendant node within a value): The number of ancestors of the node within the value. The root node of the value has depth zero, the children of the root node have depth one, their children have depth two, and so forth. Segment: One of the constructs which select children ([]) or descendants (..[]) of an input value. Nodelist: A list of nodes. While a nodelist can be represented in JSON, e.g. as an array, this document does not require or assume any particular representation. Normalized Path: A simple form of JSONPath expression that identifies a node in a value by providing a query that results in exactly that node. Similar to, but syntactically different from, a JSON Pointer [RFC6901]. Unicode Scalar Value: Any Unicode [UNICODE] code point except high- surrogate and low-surrogate code points. In other words, integers in either of the inclusive base 16 ranges 0 to D7FF and E000 to 10FFFF. JSON string values are sequences of Unicode scalar values. Singular Nodelist: A nodelist containing at most one node. Singular Path: A JSONPath expression built from segments each of which, regardless of the input value, produces a Singular Nodelist. Selector: A single item within a segment that takes the input value and produces a nodelist consisting of child nodes of the input value. Gössner, et al. Expires 13 July 2023 [Page 5] Internet-Draft JSONPath January 2023 1.1.1. JSON Values as Trees of Nodes This document models the argument as a tree of JSON values, each with its own node. A node is either the root node or one of its descendants. This document models the result of applying a query to the argument as a nodelist (a list of nodes). So nodes are the selectable parts of the argument. The only parts of an object that can be selected by a query are the member values. Member names and members (name/value pairs) cannot be selected. So member values have nodes, but members and member names do not. Similarly, member values are children of an object, but members and member names are not. 1.2. History This section is informative. This document picks up Stefan Gössner's popular JSONPath proposal dated 2007-02-21 [JSONPath-orig], builds on the experience from the widespread deployment of its implementations, and provides a normative specification for it. Appendix A describes how JSONPath was inspired by XML's XPath [XPath]. JSONPath was intended as a light-weight companion to JSON implementations in programming languages such as PHP and JavaScript, so instead of defining its own expression language, like XPath did, JSONPath delegated parts of a query to the underlying runtime, e.g., JavaScript's eval() function. As JSONPath was implemented in more environments, JSONPath expressions became decreasingly portable. For example, regular expression processing was often delegated to a convenient regular expression engine. This document aims to remove such implementation-specific dependencies and serve as a common JSONPath specification that can be used across programming languages and environments. This means that backwards compatibility is not always achieved; a design principle of this document is to go with a "consensus" between implementations even if it is rough, as long as that does not jeopardize the objective of obtaining a usable, stable JSON query language. Gössner, et al. Expires 13 July 2023 [Page 6] Internet-Draft JSONPath January 2023 1.3. JSON Values The JSON value a JSONPath query is applied to is, by definition, a valid JSON value. A JSON value is often constructed by parsing a JSON text. The parsing of a JSON text into a JSON value and what happens if a JSON text does not represent valid JSON are not defined by this document. Sections 4 and 8 of [RFC8259] identify specific situations that may conform to the grammar for JSON texts but are not interoperable uses of JSON, as they may cause unpredictable behavior. This document does not attempt to define predictable behavior for JSONPath queries in these situations. Specifically, the "Semantics" subsections of Sections 2.5.1, 2.5.2, 2.5.5, and 2.7.2 describe behavior that becomes unpredictable when the JSON value for one of the objects under consideration was constructed out of JSON text that exhibits multiple members for a single object that share the same member name ("duplicate names", see Section 4 of [RFC8259]). Also, selecting a child by name (Section 2.5.1) and comparing strings (Section "Comparisons" in Section 2.5.5) assume these strings are sequences of Unicode scalar values, becoming unpredictable if they are not (Section 8.2 of [RFC8259]). 1.4. Overview of JSONPath Expressions This section is informative. A JSONPath expression is applied to a JSON value, known as the argument. The output is a nodelist. A JSONPath expression consists of an identifier followed by a series of zero or more segments each of which contains one or more selectors. 1.4.1. Identifiers The root node identifier $ refers to the root node of the argument, i.e., to the argument as a whole. Every JSONPath expression begins with the root node identifier. The current node identifier @ refers to the current node in the context of the evaluation of a filter expression (described later). Gössner, et al. Expires 13 July 2023 [Page 7] Internet-Draft JSONPath January 2023 1.4.2. Segments Segments select children ([]) or descendants (..[]) of an input value. Segments can use _bracket notation_, for example: $['store']['book'][0]['title'] or the more compact _dot notation_, for example: $.store.book[0].title A JSONPath expression may use a combination of bracket and dot notations. This document treats the bracket notations as canonical and defines the shorthand dot notation in terms of bracket notation. Examples and descriptions use shorthands where convenient. 1.4.3. Selectors A wildcard * (Section 2.5.2) in the expression [*] selects all children of a node and in the expression ..[*] selects all descendants of a node. An array slice start:end:step (Section 2.5.4) selects a series of elements from an array, giving a start position, an end position, and an optional step value that moves the position from the start to the end. Filter expressions ? select certain children of an object or array, as in: $.store.book[?@.price < 10].title 1.4.4. Summary Table 1 provides a brief overview of JSONPath syntax. Gössner, et al. Expires 13 July 2023 [Page 8] Internet-Draft JSONPath January 2023 +=================+==============================================+ | Syntax Element | Description | +=================+==============================================+ | $ | root node identifier (Section 2.4) | +-----------------+----------------------------------------------+ | @ | current node identifier (Section 2.5.5) | | | (valid only within filter selectors) | +-----------------+----------------------------------------------+ | [] | child segment (Section 2.7.1) selects zero | | | or more children of a node; contains one or | | | more selectors, separated by commas | +-----------------+----------------------------------------------+ | .name | shorthand for ['name'] | +-----------------+----------------------------------------------+ | .* | shorthand for [*] | +-----------------+----------------------------------------------+ | ..[] | descendant segment (Section 2.7.2): selects | | | zero or more descendants of a node; contains | | | one or more selectors, separated by commas | +-----------------+----------------------------------------------+ | ..name | shorthand for ..['name'] | +-----------------+----------------------------------------------+ | ..* | shorthand for ..[*] | +-----------------+----------------------------------------------+ | 'name' | name selector (Section 2.5.1): selects a | | | named child of an object | +-----------------+----------------------------------------------+ | * | wildcard selector (Section 2.5.1): selects | | | all children of a node | +-----------------+----------------------------------------------+ | 3 | index selector (Section 2.5.3): selects an | | | indexed child of an array (from 0) | +-----------------+----------------------------------------------+ | 0:100:5 | array slice selector (Section 2.5.4): | | | start:end:step for arrays | +-----------------+----------------------------------------------+ | ? | filter selector (Section 2.5.5): selects | | | particular children using a boolean | | | expression | +-----------------+----------------------------------------------+ | length(@.foo) | function extension (Section 2.6): invokes a | | | function in a filter expression | +-----------------+----------------------------------------------+ Table 1: Overview of JSONPath syntax Gössner, et al. Expires 13 July 2023 [Page 9] Internet-Draft JSONPath January 2023 1.5. JSONPath Examples This section is informative. It provides examples of JSONPath expressions. The examples are based on the simple JSON value shown in Figure 1, representing a bookstore (that also has a bicycle). { "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 }, { "category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99 }, { "category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99 } ], "bicycle": { "color": "red", "price": 399 } } } Figure 1: Example JSON value Table 2 shows some JSONPath queries that might be applied to this example and their intended results. Gössner, et al. Expires 13 July 2023 [Page 10] Internet-Draft JSONPath January 2023 +========================+=======================================+ | JSONPath | Intended result | +========================+=======================================+ | $.store.book[*].author | the authors of all books in the store | +------------------------+---------------------------------------+ | $..author | all authors | +------------------------+---------------------------------------+ | $.store.* | all things in store, which are some | | | books and a red bicycle | +------------------------+---------------------------------------+ | $.store..price | the prices of everything in the store | +------------------------+---------------------------------------+ | $..book[2] | the third book | +------------------------+---------------------------------------+ | $..book[-1] | the last book in order | +------------------------+---------------------------------------+ | $..book[0,1] | the first two books | | $..book[:2] | | +------------------------+---------------------------------------+ | $..book[?(@.isbn)] | all books with an ISBN number | +------------------------+---------------------------------------+ | $..book[?(@.price<10)] | all books cheaper than 10 | +------------------------+---------------------------------------+ | $..* | all member values and array elements | | | contained in input value | +------------------------+---------------------------------------+ Table 2: Example JSONPath expressions and their intended results when applied to the example JSON value 2. JSONPath Syntax and Semantics 2.1. Overview A JSONPath expression is a string which, when applied to a JSON value, the _argument_, selects zero or more nodes of the argument and outputs these nodes as a nodelist. A query MUST be encoded using UTF-8. The grammar for queries given in this document assumes that its UTF-8 form is first decoded into Unicode code points as described in [RFC3629]; implementation approaches that lead to an equivalent result are possible. A string to be used as a JSONPath query needs to be _well-formed_ and _valid_. A string is a well-formed JSONPath query if it conforms to the ABNF syntax in this document. A well-formed JSONPath query is valid if it also fulfills all semantic requirements posed by this document, which are: Gössner, et al. Expires 13 July 2023 [Page 11] Internet-Draft JSONPath January 2023 1. Integer numbers in the JSONPath query that are relevant to the JSONPath processing (e.g., index values and steps) MUST be within the range of exact values defined in I-JSON [RFC7493], namely within the interval [-(2^53)+1, (2^53)-1]. 2. Uses of function extensions must be correctly typed, as described in Section 2.6. A JSONPath implementation MUST raise an error for any query which is not well-formed and valid. The well-formedness and the validity of JSONPath queries are independent of the JSON value the query is applied to; no further errors relating to the well-formedness and the validity of a JSONPath query can be raised during application of the query to a value. Obviously, an implementation can still fail when executing a JSONPath query, e.g., because of resource depletion, but this is not modeled in this document. However, the implementation MUST NOT silently malfunction. Specifically, if a valid JSONPath query is evaluated against a structured value whose size does not fit in the range of exact values, interfering with the correct interpretation of the query, the implementation MUST provide an indication of overflow. (Readers familiar with the HTTP error model may be reminded of 400 type errors when pondering well-formedness and validity, while resource depletion and related errors are comparable to 500 type errors.) 2.2. Syntax Syntactically, a JSONPath query consists of a root identifier ($), which stands for a nodelist that contains the root node of the argument, followed by a possibly empty sequence of _segments_. json-path = root-identifier segments segments = *(S segment) The syntax and semantics of segments are defined in Section 2.7. 2.3. Semantics In this document, the semantics of a JSONPath query define the required results and do not prescribe the internal workings of an implementation. The semantics are that a valid query is executed against a value, the _argument_, and produces a nodelist (i.e., a list of zero or more nodes of the value). Gössner, et al. Expires 13 July 2023 [Page 12] Internet-Draft JSONPath January 2023 The query is a root identifier followed by a sequence of zero or more segments, each of which is applied to the result of the previous root identifier or segment and provides input to the next segment. These results and inputs take the form of a nodelist. Segments can be added to a query to drill further into the structure of the input value. The nodelist resulting from the root identifier contains a single node, the argument. The nodelist resulting from the last segment is presented as the result of the query. Depending on the specific API, it might be presented as an array of the JSON values at the nodes, an array of Normalized Paths referencing the nodes, or both -- or some other representation as desired by the implementation. Note that an empty nodelist is a valid query result. A segment operates on each of the nodes in its input nodelist in turn, and the resultant nodelists are concatenated to produce the result of the segment. A node may be selected more than once and appears that number of times in the nodelist. Duplicate nodes are not removed. A syntactically valid segment MUST NOT produce errors when executing the query. This means that some operations that might be considered erroneous, such as using an index lying outside the range of an array, simply result in fewer nodes being selected. Consider this example. With the argument {"a":[{"b":0},{"b":1},{"c":2}]}, the query $.a[*].b selects the following list of nodes: 0, 1 (denoted here by their value). The query consists of $ followed by three segments: .a, [*], and .b. Firstly, $ produces a nodelist consisting of just the argument. Next, .a selects from any object input node and selects the node of any member value of the input node corresponding to the member name "a". The result is again a list of one node: [{"b":0},{"b":1},{"c":2}]. Next, [*] selects from any array input node all its elements (for an object input node, it would select all its member values, but not the member names). The result is a list of three nodes: {"b":0}, {"b":1}, and {"c":2}. Gössner, et al. Expires 13 July 2023 [Page 13] Internet-Draft JSONPath January 2023 Finally, .b selects from any object input node with a member name b and selects the node of the member value of the input node corresponding to that name. The result is a list containing 0, 1. This is the concatenation of three lists, two of length one containing 0, 1, respectively, and one of length zero. As a consequence of this approach, if any of the segments produces an empty nodelist, then the whole query produces an empty nodelist. If a query may produce a nodelist with more than one possible ordering, a particular implementation may also produce distinct orderings in successive runs of the query. In what follows, the semantics of each segment are defined for each kind of input node. 2.4. Root Identifier Syntax Every JSONPath query MUST begin with the root identifier $. root-identifier = "$" Semantics The root identifier $ represents the root node of the argument and produces a nodelist consisting of that root node. Examples JSON: {"k": "v"} Queries: +=======+============+=============+===========+ | Query | Result | Result Path | Comment | +=======+============+=============+===========+ | $ | {"k": "v"} | $ | Root node | +-------+------------+-------------+-----------+ Table 3: Root identifier examples Gössner, et al. Expires 13 July 2023 [Page 14] Internet-Draft JSONPath January 2023 2.5. Selectors Selectors appear only inside child segments (Section 2.7.1) and descendant segments (Section 2.7.2). A selector produces a nodelist consisting of zero or more children of the input value. There are various kinds of selectors which produce children of objects, children of arrays, or children of either objects or arrays. selector = ( name-selector / index-selector / slice-selector / filter-selector ) The syntax and semantics of each kind of selector are defined below. 2.5.1. Name Selector Syntax A name selector '' selects at most one object member value. In contrast to JSON, the JSONPath syntax allows strings to be enclosed in _single_ or _double_ quotes. Gössner, et al. Expires 13 July 2023 [Page 15] Internet-Draft JSONPath January 2023 name-selector = string-literal string-literal = %x22 *double-quoted %x22 / ; "string" %x27 *single-quoted %x27 ; 'string' double-quoted = unescaped / %x27 / ; ' ESC %x22 / ; \" ESC escapable single-quoted = unescaped / %x22 / ; " ESC %x27 / ; \' ESC escapable ESC = %x5C ; \ backslash unescaped = %x20-21 / ; s. RFC 8259 %x23-26 / ; omit " %x28-5B / ; omit ' %x5D-10FFFF ; omit \ escapable = ( %x62 / %x66 / %x6E / %x72 / %x74 / ; \b \f \n \r \t ; b / ; BS backspace U+0008 ; t / ; HT horizontal tab U+0009 ; n / ; LF line feed U+000A ; f / ; FF form feed U+000C ; r / ; CR carriage return U+000D "/" / ; / slash (solidus) U+002F "\" / ; \ backslash (reverse solidus) U+005C (%x75 hexchar) ; uXXXX U+XXXX ) hexchar = non-surrogate / (high-surrogate "\" %x75 low-surrogate) non-surrogate = ((DIGIT / "A"/"B"/"C" / "E"/"F") 3HEXDIG) / ("D" %x30-37 2HEXDIG ) high-surrogate = "D" ("8"/"9"/"A"/"B") 2HEXDIG low-surrogate = "D" ("C"/"D"/"E"/"F") 2HEXDIG HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" Note: double-quoted strings follow the JSON string syntax (Section 7 of [RFC8259]); single-quoted strings follow an analogous pattern (Section "Syntax"). Gössner, et al. Expires 13 July 2023 [Page 16] Internet-Draft JSONPath January 2023 Semantics A name-selector string MUST be converted to a member name M by removing the surrounding quotes and replacing each escape sequence with its equivalent Unicode character, as in the table below: +=================+===================+=============================+ | Escape Sequence | Unicode Character | Description | +=================+===================+=============================+ | \b | U+0008 | BS backspace | +-----------------+-------------------+-----------------------------+ | \t | U+0009 | HT horizontal tab | +-----------------+-------------------+-----------------------------+ | \n | U+000A | LF line feed | +-----------------+-------------------+-----------------------------+ | \f | U+000C | FF form feed | +-----------------+-------------------+-----------------------------+ | \r | U+000D | CR carriage return | +-----------------+-------------------+-----------------------------+ | \" | U+0022 | quotation mark | +-----------------+-------------------+-----------------------------+ | \' | U+0027 | apostrophe | +-----------------+-------------------+-----------------------------+ | \/ | U+002F | slash (solidus) | +-----------------+-------------------+-----------------------------+ | \\ | U+005C | backslash (reverse | | | | solidus) | +-----------------+-------------------+-----------------------------+ | \uXXXX | U+XXXX | unicode character | +-----------------+-------------------+-----------------------------+ Table 4: Escape Sequence Replacements Applying the name-selector to an object node selects a member value whose name equals the member name M, or selects nothing if there is no such member value. Nothing is selected from a value that is not an object. Note that processing the name selector requires comparing the member name string M with member name strings in the JSON to which the selector is being applied. Two strings MUST be considered equal if and only if they are identical sequences of Unicode scalar values. In other words, normalization operations MUST NOT be applied to either the member name string M from the JSONPath or to the member name strings in the JSON prior to comparison. Gössner, et al. Expires 13 July 2023 [Page 17] Internet-Draft JSONPath January 2023 Examples JSON: { "o": {"j j": {"k.k": 3}}, "'": {"@": 2} } Queries: +===================+========+======================+=============+ | Query | Result | Result Paths | Comment | +===================+========+======================+=============+ | $.o['j j']['k.k'] | 3 | $['o']['j j']['k.k'] | Named value | | | | | in nested | | | | | object | +-------------------+--------+----------------------+-------------+ | $.o["j j"]["k.k"] | 3 | $['o']['j j']['k.k'] | Named value | | | | | in nested | | | | | object | +-------------------+--------+----------------------+-------------+ | $["'"]["@"] | 2 | $['\'']['@'] | Unusual | | | | | member | | | | | names | +-------------------+--------+----------------------+-------------+ Table 5: Name selector examples 2.5.2. Wildcard Selector Syntax The wildcard selector consists of an asterisk. wildcard = "*" Semantics A wildcard selector selects the nodes of all children of an object or array. The order in which the children of an object appear in the resultant nodelist is not stipulated, since JSON objects are unordered. Children of an array appear in array order in the resultant nodelist. The wildcard selector selects nothing from a primitive JSON value (that is, a number, a string, true, false, or null). Gössner, et al. Expires 13 July 2023 [Page 18] Internet-Draft JSONPath January 2023 Examples JSON: { "o": {"j": 1, "k": 2}, "a": [5, 3] } Queries: The following examples show the wildcard selector in use by a child segment. +========+==========+=============+===================+ | Query | Result | Result | Comment | | | | Paths | | +========+==========+=============+===================+ | $[*] | {"j": 1, | $['o'] | Object values | | | "k": 2} | $['a'] | | | | [5, 3] | | | +--------+----------+-------------+-------------------+ | $.o[*] | 1 | $['o']['j'] | Object values | | | 2 | $['o']['k'] | | +--------+----------+-------------+-------------------+ | $.o[*] | 2 | $['o']['k'] | Alternative | | | 1 | $['o']['j'] | result | +--------+----------+-------------+-------------------+ | $.o[*, | 1 | $['o']['j'] | Non-deterministic | | *] | 2 | $['o']['k'] | ordering | | | 2 | $['o']['k'] | | | | 1 | $['o']['j'] | | +--------+----------+-------------+-------------------+ | $.a[*] | 5 | $['a'][0] | Array members | | | 3 | $['a'][1] | | +--------+----------+-------------+-------------------+ Table 6: Wildcard selector examples The example above with the query $.o[*, *] shows that the wildcard selector may produce nodelists in distinct orders each time it appears in the child segment, when it is applied to an object node with two or more members (but not when it is applied to object nodes with less than two members or to array nodes). Gössner, et al. Expires 13 July 2023 [Page 19] Internet-Draft JSONPath January 2023 2.5.3. Index selector Syntax An index selector matches at most one array element value. index-selector = int ; decimal integer int = "0" / (["-"] DIGIT1 *DIGIT) ; - optional DIGIT1 = %x31-39 ; 1-9 non-zero digit Applying the numerical index-selector selects the corresponding element. JSONPath allows it to be negative (see Section "Semantics"). Notes: 1. An index-selector is an integer (in base 10, as in JSON numbers). 2. As in JSON numbers, the syntax does not allow octal- like integers with leading zeros such as 01 or -01. Semantics A non-negative index-selector applied to an array selects an array element using a zero-based index. For example, the selector 0 selects the first and the selector 4 selects the fifth element of a sufficiently long array. Nothing is selected, and it is not an error, if the index lies outside the range of the array. Nothing is selected from a value that is not an array. A negative index-selector counts from the array end. For example, the selector -1 selects the last and the selector -2 selects the penultimate element of an array with at least two elements. As with non-negative indexes, it is not an error if such an element does not exist; this simply means that no element is selected. Examples JSON: ["a","b"] Queries: The following examples show the index selector in use by a child segment. Gössner, et al. Expires 13 July 2023 [Page 20] Internet-Draft JSONPath January 2023 +=======+========+==============+================================+ | Query | Result | Result Paths | Comment | +=======+========+==============+================================+ | $[1] | "b" | $[1] | Element of array | +-------+--------+--------------+--------------------------------+ | $[-2] | "a" | $[0] | Element of array, from the end | +-------+--------+--------------+--------------------------------+ Table 7: Index selector examples 2.5.4. Array Slice selector Syntax The array slice selector has the form ::. It matches elements from arrays starting at index , ending at -- but not including -- , while incrementing by step with a default of 1. slice-selector = [start S] ":" S [end S] [":" [S step ]] start = int ; included in selection end = int ; not included in selection step = int ; default: 1 B = %x20 / ; Space %x09 / ; Horizontal tab %x0A / ; Line feed or New line %x0D ; Carriage return S = *B ; optional blank space RS = 1*B ; required blank space The slice selector consists of three optional decimal integers separated by colons. Semantics The slice selector was inspired by the slice operator of ECMAScript 4 (ES4), which was deprecated in 2014, and that of Python. Informal Introduction This section is informative. Array slicing is inspired by the behavior of the Array.prototype.slice method of the JavaScript language as defined by the ECMA-262 standard [ECMA-262], with the addition of the step parameter, which is inspired by the Python slice expression. Gössner, et al. Expires 13 July 2023 [Page 21] Internet-Draft JSONPath January 2023 The array slice expression start:end:step selects elements at indices starting at start, incrementing by step, and ending with end (which is itself excluded). So, for example, the expression 1:3 (where step defaults to 1) selects elements with indices 1 and 2 (in that order) whereas 1:5:2 selects elements with indices 1 and 3. When step is negative, elements are selected in reverse order. Thus, for example, 5:1:-2 selects elements with indices 5 and 3, in that order and ::-1 selects all the elements of an array in reverse order. When step is 0, no elements are selected. (This is the one case that differs from the behavior of Python, which raises an error in this case.) The following section specifies the behavior fully, without depending on JavaScript or Python behavior. Detailed Semantics A slice expression selects a subset of the elements of the input array, in the same order as the array or the reverse order, depending on the sign of the step parameter. It selects no nodes from a node that is not an array. A slice is defined by the two slice parameters, start and end, and an iteration delta, step. Each of these parameters is optional. len is the length of the input array. The default value for step is 1. The default values for start and end depend on the sign of step, as follows: +===========+=========+==========+ | Condition | start | end | +===========+=========+==========+ | step >= 0 | 0 | len | +-----------+---------+----------+ | step < 0 | len - 1 | -len - 1 | +-----------+---------+----------+ Table 8: Default array slice start and end values Slice expression parameters start and end are not directly usable as slice bounds and must first be normalized. Normalization for this purpose is defined as: Gössner, et al. Expires 13 July 2023 [Page 22] Internet-Draft JSONPath January 2023 FUNCTION Normalize(i, len): IF i >= 0 THEN RETURN i ELSE RETURN len + i END IF The result of the array index expression i applied to an array of length len is defined to be the result of the array slicing expression Normalize(i, len):Normalize(i, len)+1:1. Slice expression parameters start and end are used to derive slice bounds lower and upper. The direction of the iteration, defined by the sign of step, determines which of the parameters is the lower bound and which is the upper bound: FUNCTION Bounds(start, end, step, len): n_start = Normalize(start, len) n_end = Normalize(end, len) IF step >= 0 THEN lower = MIN(MAX(n_start, 0), len) upper = MIN(MAX(n_end, 0), len) ELSE upper = MIN(MAX(n_start, -1), len-1) lower = MIN(MAX(n_end, -1), len-1) END IF RETURN (lower, upper) The slice expression selects elements with indices between the lower and upper bounds. In the following pseudocode, a(i) is the i+1th element of the array a (i.e., a(0) is the first element, a(1) the second, and so forth). Gössner, et al. Expires 13 July 2023 [Page 23] Internet-Draft JSONPath January 2023 IF step > 0 THEN i = lower WHILE i < upper: SELECT a(i) i = i + step END WHILE ELSE if step < 0 THEN i = upper WHILE lower < i: SELECT a(i) i = i + step END WHILE END IF When step = 0, no elements are selected and the result array is empty. To be valid, the slice expression parameters MUST be in the I-JSON range of exact values, see Section 2.1. Examples JSON: ["a", "b", "c", "d", "e", "f", "g"] Queries: Gössner, et al. Expires 13 July 2023 [Page 24] Internet-Draft JSONPath January 2023 +===========+========+========+==========+ | Query | Result | Result | Comment | | | | Paths | | +===========+========+========+==========+ | $[1:3] | "b" | $[1] | Slice | | | "c" | $[2] | with | | | | | default | | | | | step | +-----------+--------+--------+----------+ | $[1:5:2] | "b" | $[1] | Slice | | | "d" | $[3] | with | | | | | step 2 | +-----------+--------+--------+----------+ | $[5:1:-2] | "f" | $[5] | Slice | | | "d" | $[3] | with | | | | | negative | | | | | step | +-----------+--------+--------+----------+ | $[::-1] | "g" | $[6] | Slice in | | | "f" | $[5] | reverse | | | "e" | $[4] | order | | | "d" | $[3] | | | | "c" | $[2] | | | | "b" | $[1] | | | | "a" | $[0] | | +-----------+--------+--------+----------+ Table 9: Array slice selector examples 2.5.5. Filter selector Syntax The filter selector has the form ?. It iterates over structured values, i.e., arrays and objects. filter-selector = "?" S boolean-expr During the iteration process the node of each array element or object member value being visited is known as the current node. A boolean expression, usually involving the current node, is evaluated and the current node is selected if and only if the expression yields true. The current node is accessible via the current node identifier @. An existence expression may test the result of a function expression (see Section 2.6). Gössner, et al. Expires 13 July 2023 [Page 25] Internet-Draft JSONPath January 2023 boolean-expr = logical-or-expr logical-or-expr = logical-and-expr *(S "||" S logical-and-expr) ; disjunction ; binds less tightly than conjunction logical-and-expr = basic-expr *(S "&&" S basic-expr) ; conjunction ; binds more tightly than disjunction basic-expr = paren-expr / relation-expr exist-expr exist-expr = [logical-not-op S] filter-path ; path existence or non-existence filter-path = rel-path / json-path / function-expression rel-path = current-node-identifier segments current-node-identifier = "@" Parentheses MAY be used within boolean-expr for grouping. paren-expr = [logical-not-op S] "(" S boolean-expr S ")" ; parenthesized expression logical-not-op = "!" ; logical NOT operator relation-expr = comp-expr ; comparison test Comparisons are restricted to primitive values (that is, numbers, strings, true, false, and null). These can be notated as literal values, or they can be derived from Singular Paths, each of which selects at most one node. Function expressions (see Section 2.6) used in comparison expressions return a primitive value or at most one node. comp-expr = comparable S comp-op S comparable comparable = number / string-literal / ; primitive ... true / false / null / ; values only singular-path / ; Singular Path value function-expression comp-op = "==" / "!=" / ; comparison ... "<" / ">" / ; operators "<=" / ">=" singular-path = rel-singular-path / abs-singular-path / function-expression rel-singular-path = current-node-identifier singular-path-segments abs-singular-path = root-identifier singular-path-segments singular-path-segments = *(S (name-segment / index-segment)) name-segment = "[" name-selector "]" / dot-member-name-shorthand index-segment = "[" index-selector "]" Gössner, et al. Expires 13 July 2023 [Page 26] Internet-Draft JSONPath January 2023 Alphabetic characters in ABNF are case-insensitive, so "e" can be either "e" or "E". true, false, and null are lower-case only (case-sensitive). number = (int / "-0") [ frac ] [ exp ] ; decimal number frac = "." 1*DIGIT ; decimal fraction exp = "e" [ "-" / "+" ] 1*DIGIT ; decimal exponent true = %x74.72.75.65 ; true false = %x66.61.6c.73.65 ; false null = %x6e.75.6c.6c ; null The following table lists filter expression operators in order of precedence from highest (binds most tightly) to lowest (binds least tightly). +============+===============+===========+ | Precedence | Operator type | Syntax | +============+===============+===========+ | 5 | Grouping | (...) | +------------+---------------+-----------+ | 4 | Logical NOT | ! | +------------+---------------+-----------+ | 3 | Relations | == != | | | | < <= > >= | +------------+---------------+-----------+ | 2 | Logical AND | && | +------------+---------------+-----------+ | 1 | Logical OR | || | +------------+---------------+-----------+ Table 10: Filter expression operator precedence Semantics The filter selector works with arrays and objects exclusively. Its result is a list of _zero_, _one_, _multiple_ or _all_ of their array elements or member values, respectively. Applied to primitive values, it will select nothing. The order in which the children of an object appear in the resultant nodelist is not stipulated, since JSON objects are unordered. Children of an array appear in array order in the resultant nodelist. Gössner, et al. Expires 13 July 2023 [Page 27] Internet-Draft JSONPath January 2023 Existence Tests A path by itself in a Boolean context is an existence test which yields true if the path selects at least one node and yields false if the path does not select any nodes. Existence tests differ from comparisons in that: * they work with arbitrary relative or absolute paths (not just Singular Paths). * they work with paths that select structured values. To test the value of a node selected by a path, an explicit comparison is necessary. For example, to test whether the node selected by the path @.foo has the value null, use @.foo == null (see Section 2.8) rather than the negated existence test !@.foo (which yields false if @.foo selects a node, regardless of the node's value). Comparisons The comparison operators == and < are defined first and then these are used to define !=, <=, >, and >=. When a path resulting in an empty nodelist appears on either side of a comparison: * a comparison using the operator == yields true if and only if the comparison is between two paths each of which result in an empty nodelist. * a comparison using the operator < yields false. When any path on either side of a comparison results in a nodelist consisting of a single node, each such path is replaced by the value of its node and then: * a comparison using the operator == yields true if and only if the comparison is between: - equal primitive values, - equal arrays, that is arrays of the same length where each element of the first array is equal to the corresponding element of the second array, or - equal objects with no duplicate names, that is where: Gössner, et al. Expires 13 July 2023 [Page 28] Internet-Draft JSONPath January 2023 o both objects have the same collection of names (with no duplicates), and o for each of those names, the values associated with the name by the objects are equal. * a comparison using the operator < yields true if and only if the comparison is between values which are both numbers or both strings and which satisfy the comparison: - numbers expected to interoperate as per Section 2.2 of I-JSON [RFC7493] MUST compare using the normal mathematical ordering; numbers not expected to interoperate as per I-JSON MAY compare using an implementation specific ordering - the empty string compares less than any non-empty string - a non-empty string compares less than another non-empty string if and only if the first string starts with a lower Unicode scalar value than the second string or if both strings start with the same Unicode scalar value and the remainder of the first string compares less than the remainder of the second string. Note that comparisons using the operator < yield false if either value being compared is an object, array, boolean, or null. !=, <=, >, and >= are defined in terms of the other comparison operators. For any a and b: * The comparison a != b yields true if and only if a == b yields false. * The comparison a <= b yields true if and only if a < b yields true or a == b yields true. * The comparison a > b yields true if and only if b < a yields true. * The comparison a >= b yields true if and only if b < a yields true or a == b yields true. Boolean Operators The logical AND, OR, and NOT operators have the normal semantics of Boolean algebra and obey its laws (see, for example, [BOOLEAN-LAWS]). Gössner, et al. Expires 13 July 2023 [Page 29] Internet-Draft JSONPath January 2023 Function Extensions Filter selectors may use function extensions, which are covered in Section 2.6. Examples The first set of examples shows some comparison expressions and their result with a given JSON value as input. JSON: { "obj": {"x": "y"}, "arr": [2, 3] } +========================+========+============================+ | Comparison | Result | Comment | +========================+========+============================+ | $.absent1 == $.absent2 | true | Empty nodelists | +------------------------+--------+----------------------------+ | $.absent1 <= $.absent2 | true | == implies <= | +------------------------+--------+----------------------------+ | $.absent == 'g' | false | Empty nodelist | +------------------------+--------+----------------------------+ | $.absent1 != $.absent2 | false | Empty nodelists | +------------------------+--------+----------------------------+ | $.absent != 'g' | true | Empty nodelist | +------------------------+--------+----------------------------+ | 1 <= 2 | true | Numeric comparison | +------------------------+--------+----------------------------+ | 1 > 2 | false | Strict, numeric comparison | +------------------------+--------+----------------------------+ | 13 == '13' | false | Type mismatch | +------------------------+--------+----------------------------+ | 'a' <= 'b' | true | String comparison | +------------------------+--------+----------------------------+ | 'a' > 'b' | false | Strict, string comparison | +------------------------+--------+----------------------------+ | $.obj == $.arr | false | Type mismatch | +------------------------+--------+----------------------------+ | $.obj != $.arr | true | Type mismatch | +------------------------+--------+----------------------------+ | $.obj == $.obj | true | Object comparison | +------------------------+--------+----------------------------+ | $.obj != $.obj | false | Object comparison | +------------------------+--------+----------------------------+ Gössner, et al. Expires 13 July 2023 [Page 30] Internet-Draft JSONPath January 2023 | $.arr == $.arr | true | Array comparison | +------------------------+--------+----------------------------+ | $.arr != $.arr | false | Array comparison | +------------------------+--------+----------------------------+ | $.obj == 17 | false | Type mismatch | +------------------------+--------+----------------------------+ | $.obj != 17 | true | Type mismatch | +------------------------+--------+----------------------------+ | $.obj <= $.arr | false | Objects and arrays are not | | | | ordered | +------------------------+--------+----------------------------+ | $.obj < $.arr | false | Objects and arrays are not | | | | ordered | +------------------------+--------+----------------------------+ | $.obj <= $.obj | true | == implies <= | +------------------------+--------+----------------------------+ | $.arr <= $.arr | true | == implies <= | +------------------------+--------+----------------------------+ | 1 <= $.arr | false | Arrays are not ordered | +------------------------+--------+----------------------------+ | 1 >= $.arr | false | Arrays are not ordered | +------------------------+--------+----------------------------+ | 1 > $.arr | false | Arrays are not ordered | +------------------------+--------+----------------------------+ | 1 < $.arr | false | Arrays are not ordered | +------------------------+--------+----------------------------+ | true <= true | true | == implies <= | +------------------------+--------+----------------------------+ | true > true | false | Booleans are not ordered | +------------------------+--------+----------------------------+ Table 11: Comparison examples The second set of examples shows some complete JSONPath queries that make use of filter selectors, and the results of evaluating these queries on a given JSON value as input. (Note that two of the queries employ function extensions; please see Sections 2.6.5 and 2.6.6 below for details about these.) JSON: { "a": [3, 5, 1, 2, 4, 6, {"b": "j"}, {"b": "k"}, {"b": {}}, {"b": "kilo"}], "o": {"p": 1, "q": 2, "r": 3, "s": 5, "t": {"u": 6}}, "e": "f" } Gössner, et al. Expires 13 July 2023 [Page 31] Internet-Draft JSONPath January 2023 +==================+==============+=============+===================+ | Query | Result | Result | Comment | | | | Paths | | +==================+==============+=============+===================+ | $.a[@.b == | {"b": | $['a'][9] | Member value | | 'kilo'] | "kilo"} | | comparison | +------------------+--------------+-------------+-------------------+ | $.a[?@>3.5] | 5 | $['a'][1] | Array value | | | 4 | $['a'][4] | comparison | | | 6 | $['a'][5] | | +------------------+--------------+-------------+-------------------+ | $.a[?@.b] | {"b": "j"} | $['a'][6] | Array value | | | {"b": "k"} | $['a'][7] | existence | | | {"b": {}} | $['a'][8] | | | | {"b": | $['a'][9] | | | | "kilo"} | | | +------------------+--------------+-------------+-------------------+ | $[?@.*] | [3, 5, 1, | $['a'] | Existence of non- | | | 2, 4, 6, | $['o'] | singular paths | | | {"b": "j"}, | | | | | {"b": "k"}, | | | | | {"b": {}}, | | | | | {"b": | | | | | "kilo"}] | | | | | {"p": 1, | | | | | "q": 2, | | | | | "r": 3, | | | | | "s": 5, | | | | | "t": {"u": | | | | | 6}} | | | +------------------+--------------+-------------+-------------------+ | $[?@[?@.b]] | [3, 5, 1, | $['a'] | Nested filters | | | 2, 4, 6, | | | | | {"b": "j"}, | | | | | {"b": "k"}, | | | | | {"b": {}}, | | | | | {"b": | | | | | "kilo"}] | | | +------------------+--------------+-------------+-------------------+ | $.o[?@<3, ?@<3] | 1 | $['o']['p'] | Non-deterministic | | | 2 | $['o']['q'] | ordering | | | 2 | $['o']['q'] | | | | 1 | $['o']['p'] | | +------------------+--------------+-------------+-------------------+ | $.a[?@<2 || @.b | 1 | $['a'][2] | Array value | | == "k"] | {"b": "k"} | $['a'][7] | logical OR | +------------------+--------------+-------------+-------------------+ | $.a[?match(@.b, | {"b": "j"} | $['a'][6] | Array value | Gössner, et al. Expires 13 July 2023 [Page 32] Internet-Draft JSONPath January 2023 | "[jk]")] | {"b": "k"} | $['a'][7] | regular | | | | | expression match | +------------------+--------------+-------------+-------------------+ | $.a[?search(@.b, | {"b": "j"} | $['a'][6] | Array value | | "[jk]")] | {"b": "k"} | $['a'][7] | regular | | | {"b": | $['a'][9] | expression search | | | "kilo"} | | | +------------------+--------------+-------------+-------------------+ | $.o[?@>1 && @<4] | 2 | $['o']['q'] | Object value | | | 3 | $['o']['r'] | logical AND | +------------------+--------------+-------------+-------------------+ | $.o[?@>1 && @<4] | 3 | $['o']['r'] | Alternative | | | 2 | $['o']['q'] | result | +------------------+--------------+-------------+-------------------+ | $.o[?@.u || @.x] | {"u": 6} | $['o']['t'] | Object value | | | | | logical OR | +------------------+--------------+-------------+-------------------+ | $.a[?(@.b == | 3 | $['a'][0] | Comparison of | | $.x)] | 5 | $['a'][1] | paths with no | | | 1 | $['a'][2] | values | | | 2 | $['a'][3] | | | | 4 | $['a'][4] | | | | 6 | | | +------------------+--------------+-------------+-------------------+ | $[?(@ == @)] | | | Comparison of | | | | | structured values | +------------------+--------------+-------------+-------------------+ Table 12: Filter selector examples The example above with the query $.o[?@<3, ?@<3] shows that the filter selector may produce nodelists in distinct orders each time it appears in the child segment. 2.6. Function Extensions Beyond the filter expression functionality defined in the preceding subsections, JSONPath defines an extension point that can be used to add filter expression functionality: "Function Extensions". A function extension defines a registered name (see Section 3.2) that can be applied to a sequence of zero or more arguments, producing a result. Gössner, et al. Expires 13 July 2023 [Page 33] Internet-Draft JSONPath January 2023 function-name = function-name-first *function-name-char function-name-first = LCALPHA function-name-char = DIGIT / function-name-first / "_" LCALPHA = %x61-7A ; "a".."z" function-expression = function-name "(" S [function-argument *(S "," S function-argument)] S ")" function-argument = filter-path / comparable A function argument is a filter-path or a comparable. According to Section 2.5.5, a function-expression is valid as a filter-path or a comparable. Any function expressions in a query must be well-formed (by conforming to the above ABNF) and correctly typed, otherwise the JSONPath implementation MUST raise an error (see Section 2.1). To define which function expressions are correctly typed, a type system is first introduced. 2.6.1. Type System for Function Expressions Each argument and result of a function extension must have a declared type. A type is a set of instances. A type is a subtype of another type if its set of instances (possibly after coercion) is a subset of the set of instances of the other type. Table 13 defines the available types in terms of abstract instances, where n denotes a node, v denotes a value, and nl denotes a non-empty nodelist. The table also lists the subtypes of each type. Gössner, et al. Expires 13 July 2023 [Page 34] Internet-Draft JSONPath January 2023 +=====================+===================+==================+ | Type | Abstract | Subtypes | | | Instances | | +=====================+===================+==================+ | OptionalNodeOrValue | Node(n), | OptionalNode, | | | Value(v), Nothing | OptionalValue, | | | | Value, Absent | +---------------------+-------------------+------------------+ | OptionalNode | Node(n), Nothing | Absent | +---------------------+-------------------+------------------+ | OptionalValue | Value(v), Nothing | Value, | | | | OptionalBoolean, | | | | Absent | +---------------------+-------------------+------------------+ | Value | Value(v) | Boolean | +---------------------+-------------------+------------------+ | OptionalBoolean | Value(true), | Boolean, Absent | | | Value(false), | | | | Nothing | | +---------------------+-------------------+------------------+ | Boolean | Value(true), | | | | Value(false) | | +---------------------+-------------------+------------------+ | Absent | Nothing | | +---------------------+-------------------+------------------+ | OptionalNodes | Nodes(nl), | OptionalNode, | | | Nothing | Absent | +---------------------+-------------------+------------------+ Table 13: Function extension type system Notes: * OptionalNodeOrValue is an abstraction of a comparable (which may appear on either side of a comparison or as a function argument). * OptionalNode is an abstraction of a Singular Path. * Value is an abstraction of a primitive value. * Boolean is an abstraction of a primitive value that is either true or false. * OptionalValue is an abstraction of a primitive value that may also be absent. * Absent is an abstraction of an empty nodelist. Gössner, et al. Expires 13 July 2023 [Page 35] Internet-Draft JSONPath January 2023 * OptionalNodes is an abstraction of a filter-path (which appears in an existence test or as a function argument). The abstract instances above can be obtained from the concrete representations in Table 14. +===================+=====================================+ | Abstract Instance | Concrete Representations | +===================+=====================================+ | Node(n) | Singular Path resulting in a | | | nodelist containing just the node n | +-------------------+-------------------------------------+ | Value(v) | JSON value v | +-------------------+-------------------------------------+ | Nothing | Singular Path or filter-path | | | resulting in an empty nodelist | +-------------------+-------------------------------------+ | Nodes(nl) | filter-path resulting in the non- | | | empty nodelist nl | +-------------------+-------------------------------------+ Table 14: Concrete representations of abstract instances The following subtype relationships depend on coercion: * OptionalNode is a subtype of OptionalValue via coercion since the OptionalNode instance Node(n) can be coerced to the OptionalValue instance Value(v), where v is the value of the node n. * OptionalNode is a subtype of OptionalNodes via coercion since the OptionalNode instance Node(n) can be coerced to the OptionalNodes instance Nodes(l), where l is a nodelist consisting of just the node n. The type correctness of function expressions can now be defined in terms of this type system. 2.6.2. Type Correctness of Function Expressions A function expression is correctly typed if all the following are true: * If it occurs as a filter-path in an existence test, the function is defined to have result type OptionalNodes or one of its subtypes, or to have result type OptionalBoolean or one of its subtypes. Gössner, et al. Expires 13 July 2023 [Page 36] Internet-Draft JSONPath January 2023 * If it occurs as a comparable in a comparison, the function is defined to have result type OptionalNodeOrValue or one of its subtypes. * For it and any function expression it contains, each argument of the function matches the defined type of the argument according to one of the following rules: - The argument is a function expression with defined result type that is the same as, or a subtype of, the defined type of the argument. - The argument is a literal primitive value and the defined type of the argument is Value or any type of which Value is a subtype. - The argument is a Singular Path and the defined type of the argument is OptionalNode or any type of which OptionalNode is a subtype. - The argument is a filter-path or a Singular Path and the defined type of the argument is OptionalNodes. 2.6.3. length Function Extension Arguments: 1. Value Result: Value (unsigned integer) The "length" function extension provides a way to compute the length of a value and make that available for further processing in the filter expression: $[?length(@.authors) >= 5] Its only argument is a value (possibly taken from a singular path as in the example above). The result also is a value, an unsigned integer. * If the argument value is a string, the result is the number of Unicode scalar values in the string. * If the argument value is an array, the result is the number of elements in the array. * If the argument value is an object, the result is the number of members in the object. Gössner, et al. Expires 13 July 2023 [Page 37] Internet-Draft JSONPath January 2023 * For any other argument value, the result is one. 2.6.4. count Function Extension Arguments: 1. OptionalNodes Result: Value (unsigned integer) The "count" function extension provides a way to obtain the number of nodes in a nodelist and make that available for further processing in the filter expression: $[?count(@.*.author) >= 5] Its only argument is a nodelist. The result is a value, an unsigned integer, that gives the number of nodes in the nodelist. Note that there is no deduplication of the nodelist. // Well, that can be discussed. 2.6.5. match Function Extension Arguments: 1. OptionalNodeOrValue (string) 2. Value (string conforming to [I-D.draft-ietf-jsonpath-iregexp]) Result: OptionalBoolean (true, false, or Nothing) The "match" function extension provides a way to check whether (the entirety of, see Section 2.6.6 below) a given string matches a given regular expression, which is in [I-D.draft-ietf-jsonpath-iregexp] form. $[?match(@.date, "1974-05-..")] Its first argument is a string that is matched against the iregexp contained in the string that is the second argument. The result is true if the string matches the iregexp and false otherwise. The result is Nothing if the first argument is not a string or the second argument is not a string conforming to [I-D.draft-ietf-jsonpath-iregexp]. 2.6.6. search Function Extension Arguments: 1. OptionalNodeOrValue (string) Gössner, et al. Expires 13 July 2023 [Page 38] Internet-Draft JSONPath January 2023 2. Value (string conforming to [I-D.draft-ietf-jsonpath-iregexp]) Result: OptionalBoolean (true, false, or Nothing) The "search" function extension provides a way to check whether a given string contains a substring that matches a given regular expression, which is in [I-D.draft-ietf-jsonpath-iregexp] form. The result is Nothing if the first argument is not a string or the second argument is not a string conforming to [I-D.draft-ietf-jsonpath-iregexp]. $[?search(@.author, "[BR]ob")] Its first argument is a string that is searched for at least one substring that matches the iregexp contained in the string that is the second argument. The result is true if such a substring exists, false otherwise. Examples +======================+================================+ | Query | Comment | +======================+================================+ | $[?length(@) < 3] | Valid typing | +----------------------+--------------------------------+ | $[?length(@.*) < 3] | Invalid typing since @.* is a | | | non-singular path | +----------------------+--------------------------------+ | $[?count(@.*) == 1] | Valid typing | +----------------------+--------------------------------+ | $[?count(1) == 1] | Invalid typing since 1 is not | | | a path | +----------------------+--------------------------------+ | $[?count(foo(@.*)) | Valid typing, where foo is a | | == 1] | function extension with | | | argument of type OptionalNodes | | | and result type OptionalNodes | +----------------------+--------------------------------+ | $[?match(@.timezone, | Valid typing | | 'Europe/.*')] | | +----------------------+--------------------------------+ | $[?match(@.timezone, | Valid typing | | 'Europe/.*') == | | | true] | | +----------------------+--------------------------------+ Table 15: Function expression examples Gössner, et al. Expires 13 July 2023 [Page 39] Internet-Draft JSONPath January 2023 2.7. Segments Segments apply one or more selectors to an input value and concatenate the results into a single nodelist. It turns out that the more segments there are in a query, the greater the depth in the input value of the nodes of the resultant nodelist: * A query with N segments, where N >= 0, produces a nodelist consisting of nodes at depth in the input value of N or greater. * A query with N segments, where N >= 0, all of which are child segments (Section 2.7.1), produces a nodelist consisting of nodes precisely at depth N in the input value. There are two kinds of segment: child segments and descendant segments. segment = child-segment / descendant-segment The syntax and semantics of each kind of segment are defined below. 2.7.1. Child Segment Syntax The child segment consists of a non-empty, comma-separated sequence of selectors enclosed in square brackets. Shorthand notations are also provided for when there is a single wildcard or name selector. Gössner, et al. Expires 13 July 2023 [Page 40] Internet-Draft JSONPath January 2023 child-segment = (child-longhand / dot-wildcard-shorthand / dot-member-name-shorthand) child-longhand = "[" S selector 1*(S "," S selector) S "]" dot-wildcard-shorthand = "." wildcard dot-member-name-shorthand = "." dot-member-name dot-member-name = name-first *name-char name-first = ALPHA / "_" / ; _ %x80-10FFFF ; any non-ASCII Unicode character name-char = DIGIT / name-first DIGIT = %x30-39 ; 0-9 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z The dot-wildcard-shorthand is shorthand for [*]. A dot-member-name-shorthand of the form . is shorthand for [''], but can only be used with member names that are composed of certain characters. Thus, for example, $.foo.bar is shorthand for $['foo']['bar'] (and not for $['foo.bar']). Semantics A child segment contains a sequence of selectors, each of which selects zero or more children of the input value. Selectors of different kinds may be combined within a single child segment. The resulting nodelist of a child segment is the concatenation of the nodelists from each of its selectors in the order that the selectors appear in the list. Note that any node matched by more than one selector is kept as many times in the nodelist. Where a selector can produce a nodelist in more than one possible order, each occurrence of the selector in the child segment may evaluate to produce a nodelist in a distinct order. So a child segment drills down one more level into the structure of the input value. Gössner, et al. Expires 13 July 2023 [Page 41] Internet-Draft JSONPath January 2023 Examples JSON: ["a", "b", "c", "d", "e", "f", "g"] Queries: +========+========+========+============+ | Query | Result | Result | Comment | | | | Paths | | +========+========+========+============+ | $[0, | "a" | $[0] | Indices | | 3] | "d" | $[3] | | +--------+--------+--------+------------+ | $[0:2, | "a" | $[0] | Slice and | | 5] | "b" | $[1] | index | | | "f" | $[5] | | +--------+--------+--------+------------+ | $[0, | "a" | $[0] | Duplicated | | 0] | "a" | $[0] | entries | +--------+--------+--------+------------+ Table 16: Child segment examples 2.7.2. Descendant Segment Syntax The descendant segment consists of a double dot .. followed by a child segment (descendant-segment). Shortand notations are also provided that correspond to the shorthand forms of the child segment. descendant-segment = (descendant-child / descendant-wildcard-shorthand / descendant-member-name-shorthand) descendant-child = ".." child-segment descendant-wildcard-shorthand = ".." wildcard descendant-member-name-shorthand = ".." dot-member-name The descendant-wildcard-shorthand is shorthand for ..[*]. A descendant-member-name-shorthand of the form .. is shorthand for ..['']. Gössner, et al. Expires 13 July 2023 [Page 42] Internet-Draft JSONPath January 2023 Note that .. on its own is not a valid segment. Semantics A descendant segment produces zero or more descendants of the input value. A descendant selector visits the input value and each of its descendants such that: * nodes of any array are visited in array order, and * nodes are visited before their descendants. The order in which the children of an object are visited is not stipulated, since JSON objects are unordered. Suppose the nodes, in the order visited, are D1, ..., Dn (where n >= 1). Note that D1 is the input value. For each i such that 1 <= i <= n, the nodelist Ri is defined to be a result of applying the child segment [] to the node Di. The result of the descendant selector is the concatenation of R1, ..., Rn (in that order). So a descendant segment drills down one or more levels into the structure of the input value. Examples JSON: { "o": {"j": 1, "k": 2}, "a": [5, 3, [{"j": 4}, {"k": 6}]] } Queries: Gössner, et al. Expires 13 July 2023 [Page 43] Internet-Draft JSONPath January 2023 +==========+================+===================+===================+ | Query | Result | Result Paths | Comment | +==========+================+===================+===================+ | $..j | 1 | $['o']['j'] | Object values | | | 4 | $['a'][2][0]['j'] | | +----------+----------------+-------------------+-------------------+ | $..j | 4 | $['a'][2][0]['j'] | Alternative | | | 1 | $['o']['j'] | result | +----------+----------------+-------------------+-------------------+ | $..[0] | 5 | $['a'][0] | Array values | | | {"j": 4} | $['a'][2][0] | | +----------+----------------+-------------------+-------------------+ | $..[0] | {"j": 4} | $['a'][2][0] | Alternative | | | 5 | $['a'][0] | result | +----------+----------------+-------------------+-------------------+ | $..[*] | {"j": 1, | $['o'] | All values | | $..* | "k" : 2} | $['a'] | | | | [5, 3, | $['o']['j'] | | | | [{"j": 4}, | $['o']['k'] | | | | {"k": 6}]] | $['a'][0] | | | | 1 | $['a'][1] | | | | 2 | $['a'][2] | | | | 5 | $['a'][2][0] | | | | 3 | $['a'][2][0]['j'] | | | | [{"j": 4}, | $['a'][2][1] | | | | {"k": 6}] | $['a'][2][1]['k'] | | | | {"j": 4} | | | | | 4 | | | | | {"k": 6} | | | | | 6 | | | +----------+----------------+-------------------+-------------------+ | $..o | {"j": 1, | $['o'] | Input value is | | | "k": 2} | | visited | +----------+----------------+-------------------+-------------------+ | $.o..[*, | 1 | $['o']['j'] | Non-deterministic | | *] | 2 | $['o']['k'] | ordering | | | 2 | $['o']['k'] | | | | 1 | $['o']['j'] | | +----------+----------------+-------------------+-------------------+ | $.a..[0, | 5 | $['a'][0] | Multiple segments | | 1] | 3 | $['a'][1] | | | | {"j": 4} | $['a'][0][2][0] | | | | {"k": 6} | $['a'][0][2][1] | | +----------+----------------+-------------------+-------------------+ Table 17: Descendant segment examples Gössner, et al. Expires 13 July 2023 [Page 44] Internet-Draft JSONPath January 2023 Note: The ordering of the results for the $..[*] and $..* examples above is not guaranteed, except that: * {"j": 1, "k": 2} must appear before 1 and 2, * [5, 3, [{"j": 4}, {"k": 6}]] must appear before 5, 3, and [{"j": 4}, {"k": 6}], * 5 must appear before 3 which must appear before [{"j": 4}, {"k": 6}], * 5 and 3 must appear before {"j": 4}, 4, , {"k": 6}, and 6, * [{"j": 4}, {"k": 6}] must appear before {"j": 4} and {"k": 6}, * {"j": 4} must appear before 4, and * {"k": 6} must appear before 6. The example above with the query $.o..[*, *] shows that a selector may produce nodelists in distinct orders each time it appears in the descendant segment. The example above with the query $.a..[0, 1] shows that the child segment [0, 1] is applied to each node in turn (rather than the nodes being visited once per selector, which is the case for some JSONPath implementations that do not conform to this specification). 2.8. Semantics of null Note that JSON null is treated the same as any other JSON value: it is not taken to mean "undefined" or "missing". Examples JSON: {"a": null, "b": [null], "c": [{}], "null": 1} Queries: Gössner, et al. Expires 13 July 2023 [Page 45] Internet-Draft JSONPath January 2023 +===================+========+===========+====================+ | Query | Result | Result | Comment | | | | Paths | | +===================+========+===========+====================+ | $.a | null | $['a'] | Object value | +-------------------+--------+-----------+--------------------+ | $.a[0] | | | null used as array | +-------------------+--------+-----------+--------------------+ | $.a.d | | | null used as | | | | | object | +-------------------+--------+-----------+--------------------+ | $.b[0] | null | $['b'][0] | Array value | +-------------------+--------+-----------+--------------------+ | $.b[*] | null | $['b'][0] | Array value | +-------------------+--------+-----------+--------------------+ | $.b[?@] | null | $['b'][0] | Existence | +-------------------+--------+-----------+--------------------+ | $.b[?@==null] | null | $['b'][0] | Comparison | +-------------------+--------+-----------+--------------------+ | $.c[?(@.d==null)] | | | Comparison with | | | | | "missing" value | +-------------------+--------+-----------+--------------------+ | $.null | 1 | $['null'] | Not JSON null at | | | | | all, just a member | | | | | name string | +-------------------+--------+-----------+--------------------+ Table 18: Examples involving (or not involving) null 2.9. Normalized Paths A Normalized Path is a canonical representation of the location of a node in a value and uniquely identifies the node in the value. Specifically, a Normalized Path is a JSONPath query with restricted syntax (defined below), e.g., $['book'][3], which when applied to the value results in a nodelist consisting of just the node identified by the Normalized Path. Note that a Normalized Path represents the identity of a node _in a specific value_. There is precisely one Normalized Path identifying any particular node in a value. A canonical representation of a nodelist is as a JSON arrays of strings, where the strings are Normalized Paths. Normalized Paths provide a predictable format that simplifies testing and post-processing of nodelists, e.g., to remove duplicate nodes. Normalized Paths are used in this document as result paths in examples. Gössner, et al. Expires 13 July 2023 [Page 46] Internet-Draft JSONPath January 2023 Normalized Paths use the canonical bracket notation, rather than dot notation. Single quotes are used to delimit string member names. This reduces the number of characters that need escaping when Normalized Paths appear in double quote delimited strings, e.g., in JSON texts. Certain characters are escaped, in one and only one way; all other characters are unescaped. Note: Normalized Paths are Singular Paths, but not all Singular Paths are Normalized Paths. For example, $[-3] is a Singular Path, but is not a Normalized Path. The Normalized Path equivalent to $[-3] would have an index equal to the array length minus 3. (The array length must be at least 3 if $[-3] is to identify a node.) normalized-path = root-identifier *(normal-index-segment) normal-index-segment = "[" normal-selector "]" normal-selector = normal-name-selector / normal-index-selector normal-name-selector = %x27 *normal-single-quoted %x27 ; 'string' normal-single-quoted = normal-unescaped / ESC normal-escapable normal-unescaped = %x20-26 / ; omit control codes %x28-5B / ; omit ' %x5D-10FFFF ; omit \ normal-escapable = ( %x62 / %x66 / %x6E / %x72 / %x74 / ; \b \f \n \r \t ; b / ; BS backspace U+0008 ; t / ; HT horizontal tab U+0009 ; n / ; LF line feed U+000A ; f / ; FF form feed U+000C ; r / ; CR carriage return U+000D "'" / ; ' apostrophe U+0027 "\" / ; \ backslash (reverse solidus) U+005C (%x75 normal-hexchar) ; certain values u00xx U+00XX ) normal-hexchar = "0" "0" ( ("0" %x30-37) / ; "00"-"07" ("0" %x62) / ; "0b" ; omit U+0008-U+000A ("0" %x65-66) / ; "0e"-"0f" ; omit U+000C-U+000D ("1" normal-HEXDIG) ) normal-HEXDIG = DIGIT / %x61-66 ; "0"-"9", "a"-"f" normal-index-selector = "0" / (DIGIT1 *DIGIT) ; non-negative decimal integer Gössner, et al. Expires 13 July 2023 [Page 47] Internet-Draft JSONPath January 2023 Since there can only be one Normalized Path identifying a given node, the syntax stipulates which characters are escaped and which are not. So the definition of normal-hexchar is designed for hex escaping of characters which are not straightforwardly-printable, for example U+000B LINE TABULATION, but for which no standard JSON escape, such as \n, is available. Examples +=============+=================+==========================+ | Path | Normalized Path | Comment | +=============+=================+==========================+ | $.a | $['a'] | Object value | +-------------+-----------------+--------------------------+ | $[1] | $[1] | Array index | +-------------+-----------------+--------------------------+ | $[-3] | $[2] | Negative array index for | | | | an array of length 5 | +-------------+-----------------+--------------------------+ | $.a.b[1:2] | $['a']['b'][1] | Nested structure | +-------------+-----------------+--------------------------+ | $["\u000B"] | $['\u000b'] | Unicode escape | +-------------+-----------------+--------------------------+ | $["\u0061"] | $['a'] | Unicode character | +-------------+-----------------+--------------------------+ Table 19: Normalized Path examples 3. IANA Considerations 3.1. Registration of Media Type application/jsonpath IANA is requested to register the following media type [RFC6838]: Type name: application Subtype name: jsonpath Required parameters: N/A Optional parameters: N/A Encoding considerations: binary (UTF-8) Security considerations: See the Security Considerations section of RFCXXXX. Interoperability considerations: N/A Gössner, et al. Expires 13 July 2023 [Page 48] Internet-Draft JSONPath January 2023 Published specification: RFCXXXX Applications that use this media type: Applications that need to convey queries in JSON data Fragment identifier considerations: N/A Additional information: Deprecated alias names for this type: N/A Magic number(s): N/A File extension(s): N/A Macintosh file type code(s): N/A Person & email address to contact for further information: iesg@ietf.org Intended usage: COMMON Restrictions on usage: N/A Author: JSONPath WG Change controller: IESG Provisional registration? (standards tree only): no 3.2. Function Extensions This specification defines a new "Function Extensions sub-registry" in a new "JSONPath Parameters registry", with the policy "expert review" (Section 4.5 of [BCP26]). The experts are instructed to be frugal in the allocation of function extension names that are suggestive of generally applicable semantics, keeping them in reserve for functions that are likely to enjoy wide use and can make good use of their conciseness. The expert is also instructed to direct the registrant to provide a specification (Section 4.6 of [BCP26]), but can make exceptions, for instance when a specification is not available at the time of registration but is likely forthcoming. If the expert becomes aware of function extensions that are deployed and in use, they may also initiate a registration on their own if they deem such a registration can avert potential future collisions. Each entry in the registry must include: Gössner, et al. Expires 13 July 2023 [Page 49] Internet-Draft JSONPath January 2023 Function Name: a lower case ASCII [STD80] string that starts with a letter and can contain letters, digits and underscore characters afterwards ([a-z][_a-z0-9]*). Brief description: a brief description Input: A comma-separated list of zero or more kinds (value or nodes) of the arguments expected for this function extension Output: The kind (value or nodes) of the result for this function extension Change Controller: (see Section 2.3 of [BCP26]) Reference: a reference document that provides a description of the function extension Initial entries in this sub-registry are as listed in Table 20: +========+=============+==========+======+============+===========+ |Function| Brief | Input |Output| Change | Reference | |Name | description | | | Controller | | +========+=============+==========+======+============+===========+ |length | length of | value |value | IESG | Section | | | array | | | | 2.6 of | | | | | | | RFCthis | +--------+-------------+----------+------+------------+-----------+ |count | size of | nodelist |value | IESG | Section | | | nodelist | | | | 2.6 of | | | | | | | RFCthis | +--------+-------------+----------+------+------------+-----------+ |match | regular | value, |value | IESG | Section | | | expression | value | | | 2.6 of | | | full match | | | | RFCthis | +--------+-------------+----------+------+------------+-----------+ |search | regular | value, |value | IESG | Section | | | expression | value | | | 2.6 of | | | substring | | | | RFCthis | | | match | | | | | +--------+-------------+----------+------+------------+-----------+ Table 20: Initial Entries in the Function Extensions Subregistry Gössner, et al. Expires 13 July 2023 [Page 50] Internet-Draft JSONPath January 2023 4. Security Considerations Security considerations for JSONPath can stem from * attack vectors on JSONPath implementations, * attack vectors on how JSONPath queries are formed, and * the way JSONPath is used in security-relevant mechanisms. 4.1. Attack Vectors on JSONPath Implementations Historically, JSONPath has often been implemented by feeding parts of the query to an underlying programming language engine, e.g., JavaScript's eval() function. This approach is well known to lead to injection attacks and would require perfect input validation to prevent these attacks (see Section 12 of [RFC8259] for similar considerations for JSON itself). Instead, JSONPath implementations need to implement the entire syntax of the query without relying on the parsers of programming language engines. Attacks on availability may attempt to trigger unusually expensive runtime performance exhibited by certain implementations in certain cases. (See Section 10 of [RFC8949] for issues in hash-table implementations, and Section 8 of [I-D.draft-ietf-jsonpath-iregexp] for performance issues in regular expression implementations.) Implementers need to be aware that good average performance is not sufficient as long as an attacker can choose to submit specially crafted JSONPath queries or arguments that trigger surprisingly high, possibly exponential, CPU usage or, for example via a naive recursive implementation of the descendant segment, stack overflow. Implementations need to have appropriate resource management to mitigate these attacks. 4.2. Attack Vectors on How JSONPath Queries are Formed JSONPath queries are often not static, but formed from variables that provide index values, member names, or values to compare with in a filter expression. These variables need to be translated into the form they take in a JSONPath query, e.g., by escaping string delimiters, or by only allowing specific constructs such as .name to be formed when the given values allow that. Failure to perform these translations correctly can lead to unexpected failures, which can lead to Availability, Confidentiality, and Integrity breaches, in particular if an adversary has control over the values (e.g., by entering them into a Web form). The resulting class of attacks, _injections_ (e.g., SQL injections), is consistently found among the top causes of application security vulnerabilities and requires Gössner, et al. Expires 13 July 2023 [Page 51] Internet-Draft JSONPath January 2023 particular attention. 4.3. Attacks on Security Mechanisms that Employ JSONPath Where JSONPath is used as a part of a security mechanism, attackers can attempt to provoke unexpected or unpredictable behavior, or take advantage of differences in behavior between JSONPath implementations. Unexpected or unpredictable behavior can arise from an argument with certain constructs described as unpredictable by [RFC8259]. Predictable behavior can be expected, except in relation to the ordering of objects, for any argument conforming with [RFC7493]. Other attacks can target the behavior of underlying technologies such as UTF-8 (see Section 10 of [RFC3629]) and the Unicode character set. 5. References 5.1. Normative References [BCP26] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, . [I-D.draft-ietf-jsonpath-iregexp] Bormann, C. and T. Bray, "I-Regexp: An Interoperable Regexp Format", Work in Progress, Internet-Draft, draft- ietf-jsonpath-iregexp-02, 17 October 2022, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 2003, . [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, January 2008, . Gössner, et al. Expires 13 July 2023 [Page 52] Internet-Draft JSONPath January 2023 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013, . [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, DOI 10.17487/RFC7493, March 2015, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, December 2017, . [STD80] Cerf, V., "ASCII format for network interchange", STD 80, RFC 20, DOI 10.17487/RFC0020, October 1969, . [UNICODE] The Unicode Consortium, "The Unicode® Standard: Version 14.0 - Core Specification", September 2021, . 5.2. Informative References [BOOLEAN-LAWS] "Boolean algebra laws", n.d., . [E4X] ISO, "Information technology — ECMAScript for XML (E4X) specification", ISO/IEC 22537:2006 , 2006. [ECMA-262] Ecma International, "ECMAScript Language Specification, Standard ECMA-262, Third Edition", December 1999, . [JSONPath-orig] Gössner, S., "JSONPath — XPath for JSON", 21 February 2007, . Gössner, et al. Expires 13 July 2023 [Page 53] Internet-Draft JSONPath January 2023 [RFC6901] Bryan, P., Ed., Zyp, K., and M. Nottingham, Ed., "JavaScript Object Notation (JSON) Pointer", RFC 6901, DOI 10.17487/RFC6901, April 2013, . [RFC8949] Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", STD 94, RFC 8949, DOI 10.17487/RFC8949, December 2020, . [SLICE] "Slice notation", n.d., . [XPath] Berglund, A., Ed., Chamberlin, D., Ed., Simeon, J., Ed., Robie, J., Ed., Fernandez, M., Ed., Kay, M., Ed., and S. Boag, Ed., "XML Path Language (XPath) 2.0 (Second Edition)", W3C REC REC-xpath20-20101214, W3C REC- xpath20-20101214, 14 December 2010, . Appendix A. Inspired by XPath This appendix is informative. At the time JSONPath was invented, XML was noted for the availability of powerful tools to analyze, transform and selectively extract data from XML documents. [XPath] is one of these tools. In 2007, the need for something solving the same class of problems for the emerging JSON community became apparent, specifically for: * Finding data interactively and extracting them out of [RFC8259] JSON values without special scripting. * Specifying the relevant parts of the JSON data in a request by a client, so the server can reduce the amount of data in its response, minimizing bandwidth usage. (Note that XPath has evolved since 2007, and recent versions even nominally support operating inside JSON values. This appendix only discusses the more widely used version of XPath that was available in 2007.) JSONPath picks up the overall feeling of XPath, but maps the concepts to syntax (and partially semantics) that would be familiar to someone using JSON in a dynamic language. Gössner, et al. Expires 13 July 2023 [Page 54] Internet-Draft JSONPath January 2023 E.g., in popular dynamic programming languages such as JavaScript, Python and PHP, the semantics of the XPath expression /store/book[1]/title can be realized in the expression x.store.book[0].title or, in bracket notation, x['store']['book'][0]['title'] with the variable x holding the argument. The JSONPath language was designed to: * be naturally based on those language characteristics; * cover only the most essential parts of XPath 1.0; * be lightweight in code size and memory consumption; * be runtime efficient. A.1. JSONPath and XPath JSONPath expressions apply to JSON values in the same way as XPath expressions are used in combination with an XML document. JSONPath uses $ to refer to the root node of the argument, similar to XPath's / at the front. JSONPath expressions move further down the hierarchy using _dot notation_ ($.store.book[0].title) or the _bracket notation_ ($['store']['book'][0]['title']), a lightweight/limited, and a more heavyweight syntax replacing XPath's / within query expressions. Both JSONPath and XPath use * for a wildcard. The descendant operators, starting with .., borrowed from [E4X], are similar to XPath's //. The array slicing construct [start:end:step] is unique to JSONPath, inspired by [SLICE] from ECMASCRIPT 4. Filter expressions are supported via the syntax ?() as in $.store.book[?(@.price < 10)].title Gössner, et al. Expires 13 July 2023 [Page 55] Internet-Draft JSONPath January 2023 Table 21 extends Table 1 by providing a comparison with similar XPath concepts. +==========+==================+===================================+ | XPath | JSONPath | Description | +==========+==================+===================================+ | / | $ | the root XML element | +----------+------------------+-----------------------------------+ | . | @ | the current XML element | +----------+------------------+-----------------------------------+ | / | . or [] | child operator | +----------+------------------+-----------------------------------+ | .. | n/a | parent operator | +----------+------------------+-----------------------------------+ | // | ..name, | descendants (JSONPath borrows | | | ..[index], ..*, | this syntax from E4X) | | | or ..[*] | | +----------+------------------+-----------------------------------+ | * | * | wildcard: All XML elements | | | | regardless of their names | +----------+------------------+-----------------------------------+ | @ | n/a | attribute access: JSON values do | | | | not have attributes | +----------+------------------+-----------------------------------+ | [] | [] | subscript operator used to | | | | iterate over XML element | | | | collections and for predicates | +----------+------------------+-----------------------------------+ | | | [,] | Union operator (results in a | | | | combination of node sets); called | | | | list operator in JSONPath, allows | | | | combining member names, array | | | | indices, and slices | +----------+------------------+-----------------------------------+ | n/a | [start:end:step] | array slice operator borrowed | | | | from ES4 | +----------+------------------+-----------------------------------+ | [] | ?() | applies a filter (script) | | | | expression | +----------+------------------+-----------------------------------+ | seamless | n/a | expression engine | +----------+------------------+-----------------------------------+ | () | n/a | grouping | +----------+------------------+-----------------------------------+ Table 21: XPath syntax compared to JSONPath Gössner, et al. Expires 13 July 2023 [Page 56] Internet-Draft JSONPath January 2023 For further illustration, Table 22 shows some XPath expressions and their JSONPath equivalents. +======================+========================+===================+ | XPath | JSONPath | Result | +======================+========================+===================+ | /store/book/author | $.store.book[*].author | the authors of | | | | all books in | | | | the store | +----------------------+------------------------+-------------------+ | //author | $..author | all authors | +----------------------+------------------------+-------------------+ | /store/* | $.store.* | all things in | | | | store, which | | | | are some books | | | | and a red | | | | bicycle | +----------------------+------------------------+-------------------+ | /store//price | $.store..price | the prices of | | | | everything in | | | | the store | +----------------------+------------------------+-------------------+ | //book[3] | $..book[2] | the third book | +----------------------+------------------------+-------------------+ | //book[last()] | $..book[-1] | the last book | | | | in order | +----------------------+------------------------+-------------------+ | //book[position()<3] | $..book[0,1] | the first two | | | $..book[:2] | books | +----------------------+------------------------+-------------------+ | //book[isbn] | $..book[?(@.isbn)] | filter all | | | | books with isbn | | | | number | +----------------------+------------------------+-------------------+ | //book[price<10] | $..book[?(@.price<10)] | filter all | | | | books cheaper | | | | than 10 | +----------------------+------------------------+-------------------+ | //* | $..* | all elements in | | | | XML document; | | | | all member | | | | values and | | | | array elements | | | | contained in | | | | input value | +----------------------+------------------------+-------------------+ Table 22: Example XPath expressions and their JSONPath equivalents Gössner, et al. Expires 13 July 2023 [Page 57] Internet-Draft JSONPath January 2023 XPath has a lot more functionality (location paths in unabbreviated syntax, operators and functions) than listed in this comparison. Moreover, there are significant differences in how the subscript operator works in XPath and JSONPath: * Square brackets in XPath expressions always operate on the _node set_ resulting from the previous path fragment. Indices always start at 1. * With JSONPath, square brackets operate on the _object_ or _array_ addressed by the previous path fragment. Array indices always start at 0. Appendix B. JSON Pointer This appendix is informative. JSONPath is not intended as a replacement for, but as a more powerful companion to, JSON Pointer [RFC6901]. The purposes of the two standards are different. JSON Pointer is for identifying a single value within a JSON value whose structure is known. JSONPath can identify a single value within a JSON value, for example by using a Normalized Path. But JSONPath is also a query syntax that can be used to search for and extract multiple values from JSON values whose structure is known only in a general way. A Normalized JSONPath can be converted into a JSON Pointer by converting the syntax, without knowledge of any JSON value. The inverse is not generally true: a numeric path component in a JSON Pointer may identify a member value of an object or an element of an array. For conversion to a JSONPath query, knowledge of the structure of the JSON value is needed to distinguish these cases. Acknowledgements This document is based on Stefan Gössner's original online article defining JSONPath [JSONPath-orig]. The books example was taken from http://coli.lili.uni- bielefeld.de/~andreas/Seminare/sommer02/books.xml -- a dead link now. Contributors Gössner, et al. Expires 13 July 2023 [Page 58] Internet-Draft JSONPath January 2023 Marko Mikulicic InfluxData, Inc. Pisa Italy Email: mmikulicic@gmail.com Edward Surov TheSoul Publishing Ltd. Limassol Cyprus Email: esurov.tsp@gmail.com Greg Dennis Auckland New Zealand Email: gregsdennis@yahoo.com URI: https://github.com/gregsdennis Authors' Addresses Stefan Gössner (editor) Fachhochschule Dortmund Sonnenstraße 96 D-44139 Dortmund Germany Email: stefan.goessner@fh-dortmund.de Glyn Normington (editor) Winchester United Kingdom Email: glyn.normington@gmail.com Carsten Bormann (editor) Universität Bremen TZI Postfach 330440 D-28359 Bremen Germany Phone: +49-421-218-63921 Email: cabo@tzi.org Gössner, et al. Expires 13 July 2023 [Page 59]