Internet-Draft CBOR EDN Literals July 2023
Bormann Expires 24 January 2024 [Page]
Workgroup:
Network Working Group
Internet-Draft:
draft-ietf-cbor-edn-literals-02
Published:
Intended Status:
Informational
Expires:
Author:
C. Bormann
Universität Bremen TZI

Application-Oriented Literals in CBOR Extended Diagnostic Notation

Abstract

The Concise Binary Object Representation, CBOR (RFC 8949), defines a "diagnostic notation" in order to be able to converse about CBOR data items without having to resort to binary data.

​This document specifies how to add application-oriented extensions to the diagnostic notation. It then defines two such extensions for text representations of epoch-based date/times and of Constrained Resource Identifiers (draft-ietf-core-href).

To facilitate tool interoperation, this document also specifies a formal ABNF definition for extended diagnostic notation (EDN) that accommodates application-oriented literals.

About This Document

This note is to be removed before publishing as an RFC.

The latest revision of this draft can be found at https://cbor-wg.github.io/edn-literal/. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-ietf-cbor-edn-literals/.

Discussion of this document takes place on the cbor Working Group mailing list (mailto:cbor@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/cbor/. Subscribe at https://www.ietf.org/mailman/listinfo/cbor/.

Source for this draft and an issue tracker can be found at https://github.com/cbor-wg/edn-literal.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 24 January 2024.

Table of Contents

1. Introduction

For the Concise Binary Object Representation, CBOR, Section 8 of [RFC8949] in conjunction with Appendix G of [RFC8610] defines a "diagnostic notation" in order to be able to converse about CBOR data items without having to resort to binary data. Diagnostic notation is based on JSON, with extensions for representing CBOR constructs such as binary data and tags. (Standardizing this together with the actual interchange format does not serve to create another interchange format, but enables the use of a shared diagnostic notation in tools for and documents about CBOR.)

This document specifies how to add application-oriented extensions to the diagnostic notation. It then defines two such extensions for text representations of epoch-based date/times and of Constrained Resource Identifiers [I-D.ietf-core-href].

To facilitate tool interoperation, this document also specifies a formal ABNF definition for extended diagnostic notation (EDN) that accommodates application-oriented literals. (See Appendix A.1 for an overall ABNF grammar as well as the ABNF definitions in Appendix A.2 for grammars for both the byte string presentations predefined in [RFC8949] and the application-extensions).

Note that Section 3 and Appendix A.2.4 about CRIs may move to the [I-D.ietf-core-href] specification, depending on the relative speed of approval; the later document gets the section.

1.1. Terminology

Section 8 of [RFC8949] defines the original CBOR diagnostic notation, and Appendix G of [RFC8610] supplies a number of extensions to the diagnostic notation that result in the Extended Diagnostic Notation (EDN). The diagnostic notation extensions include popular features such as embedded CBOR (encoded CBOR data items in byte strings) and comments. A simple diagnostic notation extension for CBOR sequences was added in Section 4.2 of [RFC8742]. As diagnostic notation is not used in the kind of interchange situations where backward compatibility would pose a significant obstacle, there is little point in not using these extensions.

Therefore, when we refer to "diagnostic notation", we mean to include the original notation from Section 8 of [RFC8949] as well as the extensions from Appendix G of [RFC8610], Section 4.2 of [RFC8742], and the present document. However, we stick to the abbreviation "EDN" as it has become quite popular and is more sharply distinguishable from other meanings than "DN" would be.

In a similar vein, the term "ABNF" in this document refers to the language defined in [RFC5234] as extended in [RFC7405], even if the latter extensions are not currently used in this document. The term "CDDL" refers to the data definition language defined in [RFC8610] and its registered extensions (such as those in [RFC9165]), as well as [I-D.ietf-cbor-update-8610-grammar].

2. Application-Oriented Extension Literals

This document extends the syntax used in diagnostic notation for byte string literals to also be available for application-oriented extensions.

As per Section 8 of [RFC8949], the diagnostic notation can notate byte strings in a number of [RFC4648] base encodings, where the encoded text is enclosed in single quotes, prefixed by an identifier (>h< for base16, >b32< for base32, >h32< for base32hex, >b64< for base64 or base64url).

This syntax can be thought to establish a name space, with the names "h", "b32", "h32", and "b64" taken, but other names being unallocated. The present specification defines additional names for this namespace, which we call application-extension identifiers. For the quoted string, the same rules apply as for byte strings. In particular, the escaping rules of JSON strings are applied equivalently for application-oriented extensions, e.g., \\ stands for a single backslash and \' stands for a single quote.

An application-extension identifier is a name consisting of a lower-case ASCII letter (a-z) and zero or more additional ASCII characters that are either lower-case letters or digits (a-z0-9).

Application-extension identifiers are registered in a registry (Section 5.1). Prefixing a single-quoted string, an application-extension identifier is used to build an application-oriented extension literal, which stands for a CBOR data item the value of which is derived from the text given in the single-quoted string using a procedure defined in the specification for an application-extension identifier.

Examples for application-oriented extensions to CBOR diagnostic notation can be found in the following sections.

In addition, this document finally registers a media type identifier and a content-format for CBOR diagnostic notation. This does not elevate its status as an interchange format, but recognizes that interaction between tools is often smoother if media types can be used.

3. The "cri" Extension

The application-extension identifier "cri" is used to notate a Constrained Resource Identifier literal as per [I-D.ietf-core-href].

The text of the literal is a URI Reference as per [RFC3986] or an IRI Reference as per [RFC3987].

The value of the literal is a CRI that can be converted to the text of the literal using the procedure of Section 6.1 of [I-D.ietf-core-href]. Note that there may be more than one CRI that can be converted to the URI/IRI given; implementations are expected to favor the simplest variant available and make non-surprising choices otherwise.

As an example, the CBOR diagnostic notation

cri'https://example.com/bottarga/shaved'

is equivalent to

[-4, ["example", "com"], ["bottarga", "shaved"]]

See Appendix A.2.4 for an ABNF definition for the content of CRI literals.

4. The "dt" Extension

The application-extension identifier "dt" is used to notate a date/time literal that can be used as an Epoch-Based Date/Time as per Section 3.4.2 of [RFC8949].

The text of the literal is a Standard Date/Time String as per Section 3.4.1 of [RFC8949].

The value of the literal is a number representing the result of a conversion of the given Standard Date/Time String to an Epoch-Based Date/Time. If fractional seconds are given in the text (production time-secfrac in Figure 4), the value is a floating-point number; the value is an integer number otherwise.

As an example, the CBOR diagnostic notation

dt'1969-07-21T02:56:16Z'

is equivalent to

-14159024

See Appendix A.2.3 for an ABNF definition for the content of DT literals.

5. IANA Considerations

5.1. CBOR Diagnostic Notation application extension identifiers registry

IANA is requested to create a registry [[where?]] for application-extension identifiers, with the initial content shown in Table 1.

Table 1: Initial Content of application extension identifier registry
application-extension identifier description reference
h Reserved RFC8949
b32 Reserved RFC8949
h32 Reserved RFC8949
b64 Reserved RFC8949
cri Constrained Resource Identifier RFCthis
dt Date/Time RFCthis

(Define policy: probably specification required?; detailed template)

5.2. Media Type

IANA is requested to add the following Media-Type to the "Media Types" registry [IANA.media-types].

Table 2: New Media Type application/cbor-diagnostic
Name Template Reference
cbor-diagnostic application/cbor-diagnostic RFC XXXX, Section 5.2
Type name:

application

Subtype name:

cbor-diagnostic

Required parameters:

N/A

Optional parameters:

N/A

Encoding considerations:

binary (UTF-8)

Security considerations:

Section 6 of RFC XXXX

Interoperability considerations:

none

Published specification:

Section 5.2 of RFC XXXX

Applications that use this media type:

Tools interchanging a human-readable form of CBOR

Fragment identifier considerations:

The syntax and semantics of fragment identifiers is as specified for "application/cbor". (At publication of RFC XXXX, there is no fragment identification syntax defined for "application/cbor".)

Additional information:


Deprecated alias names for this type:

N/A

Magic number(s):

N/A

File extension(s):

.diag

Macintosh file type code(s):

N/A

Person & email address to contact for further information:

CBOR WG mailing list (cbor@ietf.org), or IETF Applications and Real-Time Area (art@ietf.org)

Intended usage:

COMMON

Restrictions on usage:

none

Author/Change controller:

IETF

Provisional registration:

no

5.3. Content-Format

IANA is requested to register a Content-Format number in the "CoAP Content-Formats" sub-registry, within the "Constrained RESTful Environments (CoRE) Parameters" Registry [IANA.core-parameters], as follows:

Table 3: New Content-Format
Content-Type Content Coding ID Reference
application/cbor-diagnostic - TBD1 RFC XXXX

TBD1 is to be assigned from the space 256..999.

6. Security considerations

The security considerations of [RFC8949] and [RFC8610] apply.

Anything else meaningful to say here?

7. References

7.1. Normative References

[I-D.ietf-cbor-update-8610-grammar]
Bormann, C., "Updates to the CDDL grammar of RFC 8610", Work in Progress, Internet-Draft, draft-ietf-cbor-update-8610-grammar-00, , <https://datatracker.ietf.org/doc/html/draft-ietf-cbor-update-8610-grammar-00>.
[I-D.ietf-core-href]
Bormann, C. and H. Birkholz, "Constrained Resource Identifiers", Work in Progress, Internet-Draft, draft-ietf-core-href-13, , <https://datatracker.ietf.org/doc/html/draft-ietf-core-href-13>.
[IANA.media-types]
IANA, "Media Types", <https://www.iana.org/assignments/media-types>.
[RFC3339]
Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, DOI 10.17487/RFC3339, , <https://www.rfc-editor.org/rfc/rfc3339>.
[RFC3986]
Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, DOI 10.17487/RFC3986, , <https://www.rfc-editor.org/rfc/rfc3986>.
[RFC3987]
Duerst, M. and M. Suignard, "Internationalized Resource Identifiers (IRIs)", RFC 3987, DOI 10.17487/RFC3987, , <https://www.rfc-editor.org/rfc/rfc3987>.
[RFC5234]
Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, , <https://www.rfc-editor.org/rfc/rfc5234>.
[RFC7405]
Kyzivat, P., "Case-Sensitive String Support in ABNF", RFC 7405, DOI 10.17487/RFC7405, , <https://www.rfc-editor.org/rfc/rfc7405>.
[RFC8259]
Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, , <https://www.rfc-editor.org/rfc/rfc8259>.
[RFC8610]
Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, , <https://www.rfc-editor.org/rfc/rfc8610>.
[RFC8742]
Bormann, C., "Concise Binary Object Representation (CBOR) Sequences", RFC 8742, DOI 10.17487/RFC8742, , <https://www.rfc-editor.org/rfc/rfc8742>.
[RFC8949]
Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", STD 94, RFC 8949, DOI 10.17487/RFC8949, , <https://www.rfc-editor.org/rfc/rfc8949>.
[RFC9165]
Bormann, C., "Additional Control Operators for the Concise Data Definition Language (CDDL)", RFC 9165, DOI 10.17487/RFC9165, , <https://www.rfc-editor.org/rfc/rfc9165>.

7.2. Informative References

[IANA.core-parameters]
IANA, "Constrained RESTful Environments (CoRE) Parameters", <https://www.iana.org/assignments/core-parameters>.
[RFC4648]
Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10.17487/RFC4648, , <https://www.rfc-editor.org/rfc/rfc4648>.

Appendix A. ABNF Definitions

A.1. Overall ABNF Definition for Extended Diagnostic Notation

This appendix provides an overall ABNF definition for the syntax of CBOR extended diagnostic notation.

To complete the parsing of an app-string with prefix, say, p, the processed sqstr inside it is further parsed using the ABNF definition specified for the production app-string-p in Appendix A.2.

For simplicity, the internal parsing for the built-in EDN prefixes is specified in the same way. ABNF definitions for h'' and b64'' are provided in Appendix A.2.1 and Appendix A.2.2. However, the prefixes b32'' and h32'' are not in wide use and an ABNF definition in this document could therefore not be based on implementation experience.

seq             = S [item S *("," S item S)] S
item            = map / array / tagged
                / basenumber / decnumber / infin / simple
                / tstr / bstr / embedded / streamstring

sign            = "+" / "-"
decnumber       = [sign] 1*DIGIT ["." 1*DIGIT] ["e" [sign] 1*DIGIT]
basenumber      = [sign] "0" ("x" 1*HEXDIG
                            / "o" 1*ODIGIT
                            / "b" 1*BDIGIT)
infin           = %s"Infinity"
                / %s"-Infinity"
                / %s"NaN"
simple          = %s"false"
                / %s"true"
                / %s"null"
                / %s"undefined"
                / %s"simple(" S item S ")"
uint            = "0" / DIGIT1 *DIGIT
tagged          = uint "(" S item S ")"

app-prefix      = lcalpha *lcalnum ; including h and b64
app-string      = app-prefix sqstr
sqstr           = "'" *single-quoted "'"
bstr            = app-string / sqstr ; app could be any type
tstr            = DQUOTE *double-quoted DQUOTE
embedded        = "<<" seq ">>"

array           = "[" spec [item S *("," S item S)] "]"
map             = "{" spec [kp S *("," S kp S)] "}"
kp              = item S ":" S item

; We allow %x09 HT in prose, but not in strings
blank           = %x09 / %x0A / %x0D / %x20
non-slash       = blank / %x21-2e / %x2f-10FFFF
S               = *blank *("/" *non-slash "/" *blank )

; note that there must be at least one string to distinguish
streamstring    = "(" spec1 tstr S *("," S tstr S) ")"
                / "(" spec1 sqstr S *("," S sqstr S) ")"
spec            = S ["_" S]
spec1           = S "_" S

double-quoted   = unescaped
                / "'"
                / "\" DQUOTE
                / "\" escapable

single-quoted   = unescaped
                / DQUOTE
                / "\" "'"
                / "\" escapable

escapable       = %s"b" ; BS backspace U+0008
                / %s"f" ; FF form feed U+000C
                / %s"n" ; LF line feed U+000A
                / %s"r" ; CR carriage return U+000D
                / %s"t" ; HT horizontal tab U+0009
                / "/"   ; / slash (solidus) U+002F (JSON!)
                / "\"   ; \ backslash (reverse solidus) U+005C
                / (%s"u" hexchar) ;  uXXXX      U+XXXX

hexchar         = non-surrogate
                / (high-surrogate "\" %s"u" low-surrogate)
non-surrogate   = ((DIGIT / "A"/"B"/"C" / "E"/"F") 3HEXDIG)
                / ("D" ODIGIT 2HEXDIG )
high-surrogate  = "D" ("8"/"9"/"A"/"B") 2HEXDIG
low-surrogate   = "D" ("C"/"D"/"E"/"F") 2HEXDIG

; Note that no other C0 characters are allowed, including %x09 HT
unescaped       = %x0A ; new line
                / %x0D ; carriage return -- ignored on input
                / %x20-21
                     ; omit 0x22 "
                / %x23-26
                     ; omit 0x27 '
                / %x28-5B
                     ; omit 0x5C \
                / %x5D-10FFFF

DQUOTE          = %x22    ; " double quote
DIGIT           = %x30-39 ; 0-9
DIGIT1          = %x31-39 ; 1-9
ODIGIT          = %x30-37 ; 0-7
BDIGIT          = %x30-31 ; 0-1
HEXDIG          = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
; Note: double-quoted strings as in "A" are case-insensitive in ABNF
lcalpha         = %x61-7A ; a-z
lcalnum         = lcalpha / DIGIT
ALPHA           = %x41-5A / lcalpha   ; A-Z / a-z
Figure 1

A.2. ABNF Definitions for app-string Content

This appendix provides ABNF definitions for application-oriented extension literals defined in [RFC8949] and in this specification. These grammars describe the decoded content of the sqstr components that combine with the application-extension identifiers to form application-oriented extension literals. Each of these may make use of rules defined in Figure 1.

A.2.1. h: ABNF Definition of Hexadecimal representation of a byte string

The syntax of the content of byte strings represented in hex, such as h'', h'0815, or h'/head/ 63 /contents/ 66 6f 6f' (another representation of << "foo" >>), is described by the ABNF in Figure 2. This syntax accommodates both lower case and upper case hex digits, as well as blank space (including comments) around each hex digit.

A.2.2. b64: ABNF Definition of Base64 representation of a byte string

The syntax of the content of byte strings represented in base64 is described by the ABNF in Figure 2.

This syntax allows both the classic Section 4 of [RFC4648] and the URL-safe Section 5 of [RFC4648] alphabet to be used. It accommodates, but does not require base64 padding. Note that inclusion of classic base64 makes it impossible to have comments in b64, as "/" is valid base64-classic.

app-string-b64  = B *(4(b64dig B))
                  [b64dig B b64dig B ["==" / b64dig B ["="]] B]
b64dig          = ALPHA / DIGIT / "-" / "_" / "+" / "/"
B               = *iblank
iblank          = %x0A / %x20  ; Not HT or CR (gone)
Figure 3: ABNF definition of Base64 Representation of a Byte String

A.2.3. dt: ABNF Definition of RFC 3339 Representation of a Date/Time

The syntax of the content of dt literals can be described by the ABNF for date-time from [RFC3339] as summarized in Section 3 of [RFC9165]:

app-string-dt   = date-time

date-fullyear   = 4DIGIT
date-month      = 2DIGIT  ; 01-12
date-mday       = 2DIGIT  ; 01-28, 01-29, 01-30, 01-31 based on
                          ; month/year
time-hour       = 2DIGIT  ; 00-23
time-minute     = 2DIGIT  ; 00-59
time-second     = 2DIGIT  ; 00-58, 00-59, 00-60 based on leap sec
                          ; rules
time-secfrac    = "." 1*DIGIT
time-numoffset  = ("+" / "-") time-hour ":" time-minute
time-offset     = "Z" / time-numoffset

partial-time    = time-hour ":" time-minute ":" time-second
                  [time-secfrac]
full-date       = date-fullyear "-" date-month "-" date-mday
full-time       = partial-time time-offset

date-time       = full-date "T" full-time
DIGIT           =  %x30-39 ; 0-9
Figure 4: ABNF Definition of RFC3339 Representation of a Date/Time

A.2.4. cri: ABNF Definition of URI Representation of a CRI

The syntax of the content of cri literals can be described by the ABNF for URI-reference in Section 4.1 of [RFC3986]:

app-string-cri = URI-reference
; ABNF from RFC 3986:

URI           = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

hier-part     = "//" authority path-abempty
                 / path-absolute
                 / path-rootless
                 / path-empty

URI-reference = URI / relative-ref

absolute-URI  = scheme ":" hier-part [ "?" query ]

relative-ref  = relative-part [ "?" query ] [ "#" fragment ]

relative-part = "//" authority path-abempty
                 / path-absolute
                 / path-noscheme
                 / path-empty

scheme        = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

authority     = [ userinfo "@" ] host [ ":" port ]
userinfo      = *( unreserved / pct-encoded / sub-delims / ":" )
host          = IP-literal / IPv4address / reg-name
port          = *DIGIT

IP-literal    = "[" ( IPv6address / IPvFuture  ) "]"

IPvFuture     = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )

IPv6address   =                            6( h16 ":" ) ls32
                 /                       "::" 5( h16 ":" ) ls32
                 / [               h16 ] "::" 4( h16 ":" ) ls32
                 / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32
                 / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32
                 / [ *3( h16 ":" ) h16 ] "::"    h16 ":"   ls32
                 / [ *4( h16 ":" ) h16 ] "::"              ls32
                 / [ *5( h16 ":" ) h16 ] "::"              h16
                 / [ *6( h16 ":" ) h16 ] "::"

h16           = 1*4HEXDIG
ls32          = ( h16 ":" h16 ) / IPv4address
IPv4address   = dec-octet "." dec-octet "." dec-octet "." dec-octet
dec-octet     = DIGIT                 ; 0-9
                 / %x31-39 DIGIT         ; 10-99
                 / "1" 2DIGIT            ; 100-199
                 / "2" %x30-34 DIGIT     ; 200-249
                 / "25" %x30-35          ; 250-255

reg-name      = *( unreserved / pct-encoded / sub-delims )

path          = path-abempty    ; begins with "/" or is empty
                 / path-absolute   ; begins with "/" but not "//"
                 / path-noscheme   ; begins with a non-colon segment
                 / path-rootless   ; begins with a segment
                 / path-empty      ; zero characters

path-abempty  = *( "/" segment )
path-absolute = "/" [ segment-nz *( "/" segment ) ]
path-noscheme = segment-nz-nc *( "/" segment )
path-rootless = segment-nz *( "/" segment )
path-empty    = 0<pchar>

segment       = *pchar
segment-nz    = 1*pchar
segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" )
                 ; non-zero-length segment without any colon ":"

pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"

query         = *( pchar / "/" / "?" )

fragment      = *( pchar / "/" / "?" )

pct-encoded   = "%" HEXDIG HEXDIG

unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"
reserved      = gen-delims / sub-delims
gen-delims    = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims    = "!" / "$" / "&" / "'" / "(" / ")"
                 / "*" / "+" / "," / ";" / "="
Figure 5: ABNF Definition of URI Representation of a CRI

Appendix B. EDN and CDDL

EDN was designed as a language to provide a human-readable representation of an instance, i.e., a single CBOR data item or CBOR sequence. CDDL was designed as a language to describe an (often large) set of such instances (which itself constitutes a language), in the form of a data definition or grammar (or sometimes schema).

The two languages share some similarities, not the least because they have mutually inspired each other. But they have very different roots:

For engineers that are using both EDN and CDDL, it is easy to write "CDDLisms" or "EDNisms" into their drafts that are meant to be in the other language. (This is one more of the many reasons to always validate formal language instances with tools.)

Important differences include:

Acknowledgements

The concept of application-oriented extensions to diagnostic notation, as well as the definition for the "dt" extension were inspired by the CoRAL work by Klaus Hartke.

Author's Address

Carsten Bormann
Universität Bremen TZI
Postfach 330440
D-28359 Bremen
Germany