Network Working Group P. Hallam-Baker Internet-Draft Comodo Group Inc. Intended status: Informational August 27, 2018 Expires: February 28, 2019 Data At Rest Encryption Part 2: DARE Container draft-hallambaker-dare-container-02 Abstract This document describes DARE Container, a message and file syntax that allows an append-only sequence of data frames to be represented with cryptographic integrity, signature and encryption enhancements. The format supports data integrity checks using digest chains and Merkle trees. The simplest supports efficient write operations and efficient read operations in either the forward or reverse direction. Support for efficient random-access reads may be provided through the use of binary trees or index records appended to the end of the file. This document is also available online at http://mathmesh.com/Documents/draft-hallambaker-dare-container.html [1] . Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on February 28, 2019. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents Hallam-Baker Expires February 28, 2019 [Page 1] Internet-Draft DARE Data Container August 2018 (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Container Format . . . . . . . . . . . . . . . . . . . . 3 1.2. Write . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3. Read Access . . . . . . . . . . . . . . . . . . . . . . . 5 1.4. Encryption and Authentication . . . . . . . . . . . . . . 5 1.5. Integrity and Signature . . . . . . . . . . . . . . . . . 5 1.6. Redaction . . . . . . . . . . . . . . . . . . . . . . . . 6 1.7. Alternative approaches . . . . . . . . . . . . . . . . . 6 1.8. Efficiency . . . . . . . . . . . . . . . . . . . . . . . 7 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1. Related Specifications . . . . . . . . . . . . . . . . . 7 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 8 3. Container Navigation . . . . . . . . . . . . . . . . . . . . 8 3.1. Tree . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2. Position Index . . . . . . . . . . . . . . . . . . . . . 9 3.3. Metadata Index . . . . . . . . . . . . . . . . . . . . . 10 4. Integrity Mechanisms . . . . . . . . . . . . . . . . . . . . 10 4.1. Digest Chain calculation . . . . . . . . . . . . . . . . 10 4.2. Binary Merkle tree calculation . . . . . . . . . . . . . 10 5. Reference . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.1. Container Headers . . . . . . . . . . . . . . . . . . . . 11 5.1.1. Structure: ContainerHeaderFirst . . . . . . . . . . . 11 5.1.2. Structure: ContainerHeader . . . . . . . . . . . . . 11 5.2. Content Metadata Structure . . . . . . . . . . . . . . . 12 5.2.1. Structure: ContentMeta . . . . . . . . . . . . . . . 12 5.3. Index Structures . . . . . . . . . . . . . . . . . . . . 13 5.3.1. Structure: ContainerIndex . . . . . . . . . . . . . . 13 5.3.2. Structure: IndexPosition . . . . . . . . . . . . . . 13 5.3.3. Structure: KeyValue . . . . . . . . . . . . . . . . . 13 5.3.4. Structure: IndexMeta . . . . . . . . . . . . . . . . 13 5.4. Signature . . . . . . . . . . . . . . . . . . . . . . . . 14 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 9. Appendix A: Examples and Test Vectors . . . . . . . . . . . . 14 9.1. Simple container . . . . . . . . . . . . . . . . . . . . 15 9.2. Payload and chain digests . . . . . . . . . . . . . . . . 16 9.3. Merkle Tree . . . . . . . . . . . . . . . . . . . . . . . 17 Hallam-Baker Expires February 28, 2019 [Page 2] Internet-Draft DARE Data Container August 2018 9.4. Signed container . . . . . . . . . . . . . . . . . . . . 19 9.5. Encrypted container . . . . . . . . . . . . . . . . . . . 21 10. Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . 25 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 11.1. Normative References . . . . . . . . . . . . . . . . . . 25 11.2. Informative References . . . . . . . . . . . . . . . . . 26 11.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 26 1. Introduction DARE Container is a message and file syntax that allows a sequence of data frames to be represented with cryptographic integrity, signature, and encryption enhancements to be constructed in an append only format. DARE Container was developed in response to needs that arose out of the design of the Mathematical Mesh [draft-hallambaker-mesh-architecture] . It is built on the binary encodings of JSON data objects, JSON-B and JSON-C [draft-hallambaker-jsonbcd] and the DARE Message format [draft-hallambaker-dare-message] . The format is designed to meet the requirements of a wide range of use cases including: o Recording transactions in persistent storage. o Synchronizing transaction logs between hosts. o File archive. o Message spool. o Signing and encrypting single data items. o Incremental encryption and authentication of server logs. 1.1. Container Format A DARE Container consists of a sequence of variable length frames. Each frame consists of a forward length indicator, the framed data and a reverse length indicator. The reverse length indicator is written out backwards allowing the length and thus the frame to be read in the reverse direction: Hallam-Baker Expires February 28, 2019 [Page 3] Internet-Draft DARE Data Container August 2018 [[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-dare- container.html [2].]] JBCD Bidirectional Frame Each frame contains a single DARE Message consisting of a Header, Payload and Trailer (if required). The first frame in a container describes the container format options and defaults. These include the range of encoding options for frame metadata supported and the container profiles to which the container conforms. All internal data formats are 64 bit clean allowing for containers of up to 18 exabytes to be written. Five container types are currently specified but this may be reduced if the Digest and Tree types are withdrawn. These are: Simple The container does not provide any index or content integrity checks. Tree Frame headers contain entries that specify the start position of previous frames at the apex of the immediately enclosing binary tree. This enables efficient random access to any frame in the file. Digest Each frame trailer contains a PayloadDigest field. Modification of the payload will cause verification of the PayloadDigest value to fail on that frame. Chain Each frame trailer contains PayloadDigest and ChainDigest fields allowing modifications to the payload data to be detected. Modification of the payload will cause verification of the PayloadDigest value to fail on that frame and verification of the ChainDigest value to fail on all subsequent frames. Merkle Tree Frame headers contain entries that specify the start position of previous frames at the apex of the immediately enclosing binary tree. Frame Trailers contain TreeDigestPartial and TreeDigestFinal entries forming a Merkle digest tree. 1.2. Write In normal circumstances, DARE Containers are written as an append only log. As with DARE Messages, integrity information (payload digest, signatures) is written to the message trailer. Thus, large Hallam-Baker Expires February 28, 2019 [Page 4] Internet-Draft DARE Data Container August 2018 payloads may be written without the need to buffer the payload data provided that the content length is known in advance. 1.3. Read Access The use of reverse length indicators allows DARE containers to support efficient sequential access in either the forward or reverse directions. Random access to any part of a file MAY be supported by means of a binary tree index and/or an index record providing direct access to any part of the file. 1.4. Encryption and Authentication Frame payloads and associated attributes MAY be encrypted and/or authenticated using the approach described in [draft-hallambaker-dare-message] . Incremental encryption is supported allowing encryption parameters from a single public key exchange operation to be applied to encrypt multiple frames. The public key exchange information is specified in the first encrypted frame and subsequent frames encrypted under those parameters specify the location at which the key exchange information is to be found by means of the ExchangePosition field. The only restriction on the use of incremental encryption is that the frame containing the key exchange information MUST precede the frames that reference the exchange parameters. To avoid cryptographic vulnerabilities resulting from key re-use, the DARE key exchange requires that each encrypted sequence use an encryption key and initialization vector derived from the master key established in the public key exchange by means of a unique salt. Each DARE Message and by extension, each DARE Container frame MUST specify a unique salt value of at least 128 bits. Since the encryption key is derived from the salt value by means of a Key Derivation Function, erasure of the salt MAY be used as a means of rendering the payload plaintext value inaccessible without changing the payload value. 1.5. Integrity and Signature Signatures MAY be applied to a payload digest, the final digest in a chain or tree. The chain and tree digest modes allow a single signature to be used to authenticate all frame payloads in a container. Hallam-Baker Expires February 28, 2019 [Page 5] Internet-Draft DARE Data Container August 2018 The tree signature mode is particularly suited to applications such as file archives as it allows files to be verified individually without requiring the signer to sign each individually. Furthermore, in applications such as code signing, it allows a single signature to be used to verify both the integrity of the code and its membership of the distribution. As with DARE Message, the signature mechanism does not specify the interpretation of the signature semantics. The presence of a signature demonstrates that the holder of the private key applied it to the specified digest value but not their motive for doing so. Describing such semantics is beyond the scope of this document and is deferred to future work. 1.6. Redaction The chief disadvantage of using an append-only format is that containers only increase in size. In many applications, much of the data in the container becomes redundant or obsolete and a process analogous to garbage collection is required. This process is called redaction. The simplest method of redaction is to create a new container and sequentially copy each entry from the old container to the new, discarding redundant frames and obsolete header information. For example, partial index records may be consolidated into a single index record placed in the last frame of the container. Unnecessary signature and integrity data may be discarded and so on. It is also possible but not necessarily advisable to perform such a redaction in-place provided that the redaction process does not increase the length of any individual frame and that appropriate provision is made for file locking to prevent conflicts and to provide for safe resumption should an interruption occur during the process. 1.7. Alternative approaches Many file proprietary formats are in use that support some or all of these capabilities but only a handful have public, let alone open, standards. DARE Container is designed to provide a superset of the capabilities of existing message and file syntaxes, including: o Cryptographic Message Syntax [RFC5652] defines a syntax used to digitally sign, digest, authenticate, or encrypt arbitrary message content. Hallam-Baker Expires February 28, 2019 [Page 6] Internet-Draft DARE Data Container August 2018 o The.ZIP File Format specification [ZIPFILE] developed by Phil Katz. o The BitCoin Block chain [BLOCKCHAIN] . o JSON Web Encryption and JSON Web Signature Attempting to make use of these specifications in a layered fashion would require at least three separate encoders and introduce unnecessary complexity. Furthermore, there is considerable overlap between the specifications providing multiple means of achieving the same ends, all of which must be supported if decoders are to work reliably. 1.8. Efficiency Every data format represents a compromise between different concerns, in particular: Compactness The space required to record data in the encoding. Memory Overhead The additional volatile storage (RAM) required to maintain indexes etc. to support efficient retrieval operations. Number of Operations The number of operations required to retrieve data from or append data to an existing encoded sequence. Number of Disk Seek Operations Optimizing the response time of magnetic storage media to random access read requests has traditionally been one of the central concerns of database design. The DARE Container format is designed to the assumption that this will cease to be a concern as solid state media replaces magnetic. While the cost of storage of all types has declined rapidly over the past decades, so has the amount of data to be stored. DARE Container represents a pragmatic balance of these considerations for current technology. In particular, since payload volumes are likely to be very large, memory and operational efficiency are considered higher priorities than compactness. 2. Definitions 2.1. Related Specifications DARE Container makes use of the following related standards and specifications. Hallam-Baker Expires February 28, 2019 [Page 7] Internet-Draft DARE Data Container August 2018 Encoding Content frame headers are encoded using JavaScript Object Notation (JSON) [RFC7159] , JSON-B or JSON-C [draft-hallambaker-jsonbcd] . Cryptography The encryption and signature schemes used are based on JSON Web Signature [RFC7515] and JSON Web Encryption [RFC7516] as applied in DARE Message [draft-hallambaker-dare-message] . 2.2. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] . 3. Container Navigation Three means of locating frames in a container are supported: Sequential Access frames sequentially starting from the start or the end of the container. Binary search Access any container frame by frame number in O(log2(n)) time by means of a binary tree constructed while the container is written. Index Access and container frame by frame number or by key by means of an index record. All DARE Containers support sequential access. Only tree and Merkle tree containers support binary search access. An index frame MAY be written appended to any container and provides O(1) access to any frame listed in the index. Two modes of compilation are considered: Monolithic Frames are added to the container in a single operation, e.g. file archives, Incremental Additional frames are written to the container at various intervals after it was originally created, e.g. server logs, message spools. In the monolithic mode, navigation requirements are best met by writing an index frame to the end of the container when it is complete. It is not necessary to construct a binary search tree unless a Merkle tree integrity check is required. Hallam-Baker Expires February 28, 2019 [Page 8] Internet-Draft DARE Data Container August 2018 In the incremental mode, Binary search provides an efficient means of locating frames by frame number but not by key. Writing a complete index to the container every m write operations provides O(m) search access but requires O(n2) storage. Use of partial indexes provides a better compromise between speed and efficiency. A partial index is written out every m frames where m is a power of two. A complete index is written every time a binary tree apex record is written. This approach provides for O(log2(n)) search with incremental compilation with approximately double the overhead of the monolithic case. 3.1. Tree Binary search is supported by means of the TreePosition parameter specified in the FrameHeader. This parameter specifies the value of the immediately preceding apex. Calculation of the immediately preceding apex is most easily described by representing the array index in binary with base of 1 (rather than 0). An array index that is a power of 2 (2, 4, 8, 16, etc.) will be the apex of a complete tree. Every other array index has the value of the sum of a set of powers of 2 and the immediately preceding apex will be the value of the next smallest power of 2 in the sum. For example, to find the immediately preceding apex for frame 5, we add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame is 4. The values of Tree Position are shown for the first 8 frames in figure xx below: [[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-dare- container.html [3].]] Merkle Tree Integrity check An algorithm for efficiently calculating the immediately preceding apex is provided in Appendix C. 3.2. Position Index Contains a table of frame number, position pairs pointing to prior locations in the file. Hallam-Baker Expires February 28, 2019 [Page 9] Internet-Draft DARE Data Container August 2018 3.3. Metadata Index Contains a list of IndexMeta entries. Each entry contains a metadata description and a list of frame indexes (not positions) of frames that match the description. 4. Integrity Mechanisms Frame sequences in a DARE container MAY be protected against a frame insertion attack by means of a digest chain, a binary Merkle tree or both. 4.1. Digest Chain calculation A digest chain is simple to implement but can only be verified if the full chain of values is known. Appending a frame to the chain has O(1) complexity but verification has O(n) complexity: [[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-dare- container.html [4].]] Hash chain integrity check The value of the chain digest for the first frame (frame 0) is H(H(null)+H(Payload0)), where null is a zero length octet sequence and payloadn is the sequence of payload data bytes for frame n The value of the chain digest for frame n is H(H(Payloadn-1 +H(Payloadn)), where A+B stands for concatenation of the byte sequences A and B. 4.2. Binary Merkle tree calculation The tree index mechanism describe earlier may be used to implement a binary Merkle tree. The value TreeDigest specifies the apex value of the tree for that node. Appending a frame to the chain has O(log2n) complexity provided that the container format supports at least the binary tree index. Verifying a chain has O(log2 n) complexity, provided that the set of necessary digest inputs is known. To calculate the value of the tree digest for a node, we first calculate the values of all the sub trees that have their apex at that node and then calculate the digest of that value and the immediately preceding local apex. Hallam-Baker Expires February 28, 2019 [Page 10] Internet-Draft DARE Data Container August 2018 5. Reference TBS stuff 5.1. Container Headers TBS stuff 5.1.1. Structure: ContainerHeaderFirst Inherits: ContainerHeader Inherits: ContainerHeader DataEncoding: String (Optional) Specifies the data encoding for the header section of for the following frames. This value is ONLY valid in Frame 0 which MUST have a header encoded in JSON. 5.1.2. Structure: ContainerHeader Inherits: DAREHeader Describes a container header. A container header MAY contain any DARE Message header. Index: Integer (Optional) The record index within the file. This MUST be unique and satisfy any additional requirements determined by the ContainerType. ContainerType: String (Optional) Specifies the container type for the following records. IsMeta: Boolean (Optional) If true, the current frame is a meta frame and does not contain a payload. Note: Meta frames MAY be present in any container. Applications MUST accept containers that contain meta frames at any position in the file. Applications MUST NOT interpret a meta frame as a data frame with an enpty payload. UniqueID: String (Optional) Unique object identifier ContentMeta: ContentMeta (Optional) Content meta data. TreePosition: Integer (Optional) Position of the frame containing the apex of the preceding sub-tree. Hallam-Baker Expires February 28, 2019 [Page 11] Internet-Draft DARE Data Container August 2018 IndexPosition: Integer (Optional) Specifies the position in the file at which the last index entry is to be found ExchangePosition: Integer (Optional) Specifies the position in the file at which the key exchange data is to be found ContainerIndex: ContainerIndex (Optional) An index of records in the current container up to but not including this one. PayloadDigest: Binary (Optional) If present, contains the digest of the Payload. ChainDigest: Binary (Optional) If present, contains the digest of the PayloadDigest values of this frame and the frame immediately preceding. TreeDigest: Binary (Optional) If present, contains the Binary Merkle Tree digest value. Event: String (Optional) Unique object identifier Labels: String [0..Many] List of labels that are applied to the payload of the frame. KeyValues: KeyValue [0..Many] List of key/value pairs describing the payload of the frame. First: Integer (Optional) Frame number of the first object instance value. Previous: Integer (Optional) Frame number of the immediately prior object instance value 5.2. Content Metadata Structure TBS stuff 5.2.1. Structure: ContentMeta Information describing the object instance ContentType: String (Optional) The content type field as specified in JWE Paths: String [0..Many] List of filename paths for the payload of the frame. UniqueID: String (Optional) Unique object identifier Hallam-Baker Expires February 28, 2019 [Page 12] Internet-Draft DARE Data Container August 2018 Created: DateTime (Optional) Initial creation date. Modified: DateTime (Optional) Date of last modification. 5.3. Index Structures TBS stuff 5.3.1. Structure: ContainerIndex A container index Full: Boolean (Optional) If true, the index is complete and contains position entries for all the frames in the file. If absent or false, the index is incremental and only contains position entries for records added since the last frame containing a ContainerIndex. Positions: IndexPosition [0..Many] List of container position entries Metas: IndexMeta [0..Many] List of container position entries 5.3.2. Structure: IndexPosition Specifies the position in a file at which a specified record index is found Index: Integer (Optional) The record index within the file. Position: Integer (Optional) The record position within the file relative to the index base. 5.3.3. Structure: KeyValue Specifies a key/value entry Key: String (Optional) The key Value: String (Optional) The value corresponding to the key 5.3.4. Structure: IndexMeta Specifies the list of index entries at which a record with the specified metadata occurrs. Index: Integer [0..Many] List of record indicies within the file where frames matching the specified criteria are found. Hallam-Baker Expires February 28, 2019 [Page 13] Internet-Draft DARE Data Container August 2018 ContentType: String (Optional) Content type parameter Paths: String [0..Many] List of filename paths for the current frame. Labels: String [0..Many] List of labels that are applied to the current frame. 5.4. Signature Payload data MAY be signed using a JWS [RFC7515] as applied in the DARE Message format [draft-hallambaker-dare-message] . Signatures are specified by the Signatures parameter in the content header. The data that the signature is calculated over is defined by the typ parameter of the Signature as follows. Payload The value of the PayloadDigest parameter Chain The value of the ChainDigest parameter Tree The value of the TreeDigestFinal parameter If the typ parameter is absent, the value Payload is implied. A frame MAY contain multiple signatures created with the same signing key and different typ values. The use of signatures over chain and tree digest values permit multiple frames to be validated using a single signature verification operation. 6. Security Considerations 7. IANA Considerations 8. Acknowledgements 9. Appendix A: Examples and Test Vectors The data payloads in all the following examples are identical, only the authentication and/or encryption is different. o Frame 1..n consists of 300 bytes being the byte sequence 00, 01, 02, etc. repeating after 256 bytes. Hallam-Baker Expires February 28, 2019 [Page 14] Internet-Draft DARE Data Container August 2018 For conciseness, the raw data format is omitted for examples after the first, except where the data payload has been transformed, (i.e. encrypted). 9.1. Simple container the following example shows a simple container with first frame and a single data frame: f4 5d f0 59 f0 00 5d f4 f5 01 40 f0 0f f1 01 2c 40 01 f5 Figure 1 Since there is no integrity check, there is no need for trailer entries. The header values are: Frame 0 { "Index":0, "ContainerType":"List", "ContentMeta":{}, "DataEncoding":"JSON"} [Empty trailer] Figure 2 Frame 1 { "Index":1} [Empty trailer] Figure 3 Hallam-Baker Expires February 28, 2019 [Page 15] Internet-Draft DARE Data Container August 2018 9.2. Payload and chain digests The following example shows a chain container with a first frame and three data frames. The headers of these frames is the same as before but the frames now have trailers specifying the PayloadDigest and ChainDigest values: Frame 0 { "Index":0, "ContainerType":"Chain", "ContentMeta":{}, "DataEncoding":"JSON"} [Empty trailer] Figure 4 Frame 1 { "Index":1} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9 tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 5 Frame 2 { "Index":2} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9 tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 6 Frame 3 Hallam-Baker Expires February 28, 2019 [Page 16] Internet-Draft DARE Data Container August 2018 { "Index":3} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9 tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 7 9.3. Merkle Tree The following example shows a chain container with a first frame and six data frames. The trailers now contain the TreePosition and TreeDigest values: Frame 0 { "Index":0, "ContainerType":"Merkle", "ContentMeta":{}, "DataEncoding":"JSON"} [Empty trailer] Figure 8 Frame 1 { "Index":1, "TreePosition":0} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 9 Frame 2 Hallam-Baker Expires February 28, 2019 [Page 17] Internet-Draft DARE Data Container August 2018 { "Index":2, "TreePosition":319} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokkFnZ QcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} Figure 10 Frame 3 { "Index":3, "TreePosition":319} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 11 Frame 4 { "Index":4, "TreePosition":1451} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvKSj8 gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"} Figure 12 Frame 5 Hallam-Baker Expires February 28, 2019 [Page 18] Internet-Draft DARE Data Container August 2018 { "Index":5, "TreePosition":1451} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 13 Frame 6 { "Index":6, "TreePosition":2586} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9SZ0m dr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"} Figure 14 9.4. Signed container The following example shows a tree container with a signature in the final record. The signing key parameters are: Hallam-Baker Expires February 28, 2019 [Page 19] Internet-Draft DARE Data Container August 2018 { "PrivateKeyRSA":{ "kid":"MBWNO-2J43U-ESWKW-XQWL6-6YGEW-UOPWU-A", "n":"1NzbmakMalVH1mRv7TEDEhwXDNojn5wZbq1tv1gp5PgZwzX-klYXuFhj0- MpO0zcwptsUaYJhwdvvgW_1udUpISQYluXOB3UMj2e_0yl64MvnqTL47SZQuAN3QQ 9cuCw_-_Eyj_jerspauqa6RpNzGcabZrtRl7J7DPVZ3SNlw-H_Wxd4HkrFVW_Yqup htNL1JciQJYm2DSu9dbetqPZ80x6IBargY850mBYOzvNNE5S-dRJQHoJY4SG-ESYF BuAHtBlOMgbI0XNiq96EegA-vPW9XRF-SHdX5mkafefDGK4rT_RoE4gRwhDM3jbZ8 1-F2ZA_GpNVEvB-25_vF96lQ", "e":"AQAB", "d":"k_v_h7Jo-TvUt44X6jSax-pTdBHrljk1zSYxGEe4yIBbmMVe-Gl2ECkTLe nNbnafO4RGJ_Vgxkk7PEZO-p7Uz5OBtX-rf83tCgihEyg8aaFIZ-h1_xY9Pqr5uGA MQGNJaoVMsLb99QNNZhE4JTquP56mVvDQaI3Zn6bhhA0ZqpxS_x6iRUV5KnHCRd47 DKGHcAsQR_caxGec7M_XNPqpl0tcoGQz46-I0SVVcqtjb_YysEVez4eJhb_ZTU4C7 pz4wXDj6B0ppFJVZEMaIhKo8FCnoQdXEJl4vSiBzUFrSlPc6gjQk2CBhxc_kb782w VwW5wtgOVxhV1k2piQ2NrlOQ", "p":"6osXfrJLiPfK0ky17vqzRs-M5mOxZU2LEFGyHTXxx6EYTWixEsx2Sdf6kx UD5K_QdYnqqd3yobbGdDpMwwEuwgogWVCz90nc9QdCUy4MCpz8lpOQd1i_tXMmtOg GBu9mapMTEOwc7HlLlSDRezv_TzTAH4izv-CUEZ_M5EcwyFM", "q":"6FYD6NV4rKU1ACtGQyIvwGrkrS_F6phB-whx0nSVFkbkwppJiPnC6XqjZ3 OYPCZylxaTHFnxCs0nntrraeNEcWfNPrpTN5XIZjbOiIaKA2iCQkWlDoi8oduEtTK oIcuy32oBz6MpUeCWzl0ZQ4EeTF3RyCc_jpf8oRvZ0e3ItHc", "dp":"hfjbe9BmWx-HqCaPSanEW-9UQYmym_X2OGUiA5N7vxci5ZymgOFvs_B9v iQj7C4NOgaEl3EjFgJsS5m9nSoAxm-4WKxDkD6NyxzRYugLkshnc69otvNn1kKnWn CqeK2o57mJC4KDZwRGCzIK1oTH6jtsfta8Lh8fFQ4doEuV7uc", "dq":"r6R_ViE0Foja1aLhflU09mmZMViBbkXm86nBqtHZ97pmrLvJRdVTxgCh0 c6w0yBZ1uEJHBDeykSoZE6qVCWtE3Le1kI0MTx6ANQENXBInCUA_Kr8Ck3TFSYIYJ fIRaxiMMZKUjfOQAji2WXGeKL_TcpLkt4hDWLXaNDOTgdOiSc", "qi":"DfHtLB1Ox1Kgp3E4jqy5Qxeb7-v7_uv8n_5-E1OQ3NLSRV2m_auojkR19 nY3gokHKNSXM41qKlJLU00lROjOO2KUq57s8GZkheVfbJLNCJ6KAw_aRT2IgyJm2b e2v5OCHSkm88tgJWbtKj-OPKTFV5gOMVdeCzGX286ErjDHGCM"}} Figure 15 The container headers and trailers are: Frame 0 { "Index":0, "ContainerType":"Merkle", "ContentMeta":{}, "DataEncoding":"JSON"} [Empty trailer] Figure 16 Hallam-Baker Expires February 28, 2019 [Page 20] Internet-Draft DARE Data Container August 2018 Frame 1 { "Index":1, "TreePosition":0} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} Figure 17 Frame 2 { "dig":"S512", "Index":2, "TreePosition":319} { "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", "TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokkFnZ QcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} Figure 18 9.5. Encrypted container The following example shows a container in which all the frame payloads are encrypted under the same master secret established in a key agreement specified in the first frame. Frame 0 Hallam-Baker Expires February 28, 2019 [Page 21] Internet-Draft DARE Data Container August 2018 { "enc":"A256CBC", "Salt":"Th7V_b2--KdV5rETQZbHsw", "recipients":[{ "kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A", "epk":{ "PublicKeyDH":{ "kid":"MANS7-VSJ2X-75NSK-4SEOM-FRITG-6Q7EO-A", "Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA", "Public":"MYw__h5Ii4honclOGFKY42KdvJed5T47CZI3QTRSBXUPiOD 0D0W49YIyf-m30K-BvMWVzNsh9qAQFpK53A5UJO_pQmfbmrZLHq3AjdYoRlc44YOn usUokZN43xtGly4m7Rcf_42M3cofa80UWejL3mz3zQKTojxwXz3cykLA2tsTjHdKE O0Q6jWef952YkPQNKJmR-42aX7Pfz5kZt1QtgfUV4QYk_b0NFfHSNk9CAs6Klav5K qGjxL4mfIxvmrksyIR5solCo2BMbh_AOdyZNuEa4nURvS9YtWrXZRjQtMZPKB8MXG _mSAxUco3U1em2hmdkgIkTveF7-da6otILw"}}, "wmk":"0HfYk15gAvxngunLoHymHSunrb0KlGjGuTERlpSxB5JLpFJj6KJZXA"} ], "Index":0, "ContainerType":"List", "ContentMeta":{}, "DataEncoding":"JSON"} [Empty trailer] Figure 19 Frame 1 { "enc":"A256CBC", "Salt":"BmY8ZYeD8Kg1snIrBIAUyA", "Index":1} [Empty trailer] Figure 20 Frame 2 { "enc":"A256CBC", "Salt":"pA-kigxfhhsGC5lM9xrAZQ", "Index":2} [Empty trailer] Figure 21 Hallam-Baker Expires February 28, 2019 [Page 22] Internet-Draft DARE Data Container August 2018 Here are the container bytes. Note that the content is now encrypted and has expanded by 25 bytes. These are the salt (16 bytes), the AES padding (4 bytes) and the JSON-B framing (5 bytes). f5 03 3c f1 03 27 f0 10 3c 03 f5 f5 01 7c f0 47 f1 01 30 7c 01 f5 f5 01 7c f0 47 f1 01 30 7c 01 f5 Figure 22 The following example shows a container in which all the frame payloads are encrypted under separate key agreements specified in the payload frames. Frame 0 { "Index":0, "ContainerType":"List", "ContentMeta":{}, "DataEncoding":"JSON"} [Empty trailer] Figure 23 Frame 1 Hallam-Baker Expires February 28, 2019 [Page 23] Internet-Draft DARE Data Container August 2018 { "enc":"A256CBC", "Salt":"LUAkC3V-ztXJfF7OS-RNwQ", "recipients":[{ "kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A", "epk":{ "PublicKeyDH":{ "kid":"MCYIZ-PH7YZ-XAY6Q-2EMQR-GMRXG-66YPK-A", "Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA", "Public":"4cDrUAlsZpyGZGhgVbLlunEyl7O9a8ku-6IGVZaSqyOmsb5 rCt_pjnN49fjy7hqsS364S4eUDRj7e3vfIBEav-zeynaiJB6lY6eJBMmlaEuomyyy 01Srq5eaX6i0A8CRWuuRK8IJAHQ31Q89Bah5XayMxc3bqPxciOL1gqU75w65fKmY9 4mRdAIw2Yk-PTcQVztVDk2ooGJuvLENuIk29g-leFbMmCAXND9hhDErVupBND3Ljh BXylTvfIx-mrw0SMPO_IsVLbhrXQ6lTMygucttT3GPf25lgWBaEdiHVuPqmTBtagB Vgq9-gHySYtuCNrK7RfB5rtz6CUmENEwx1QA"}}, "wmk":"25yqAeqIG1S6BrCSQxU_M8WKB4t8J_lHWI-68DNYORTUoCpaitosig"} ], "Index":1} [Empty trailer] Figure 24 Frame 2 { "enc":"A256CBC", "Salt":"EVrEz8ai_v9s9hZ0ZXMB1Q", "recipients":[{ "kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A", "epk":{ "PublicKeyDH":{ "kid":"MB43P-C7BVX-SROQ6-Z5LW5-5DLRO-PS675-A", "Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA", "Public":"mJRyK45fAf2ksxW7M8W961hCKI1g7dQqn9rA2K6vdSh1o4v gLqjHDzLYqr9yQKteGUfl0PLIuHAOFq3p_oJDhaioJKX6Akv-y-dcfDdi2WgOOr_k R9NPtl-nmsbJ8fx6v1QdXLzdoGZjsj_t6wyIo5w287p7DylFdYkhriqVv7tgBELZI 53Pur4F5QbK-qJYeSwY3fAksQYrviMZbFVwic8v5QQNoAGcoGguBOOz-aljFuo1g9 0o7pNT-Soj3DAPfmXS1hmPR-hpqyzELAat-q_O22P4d_NtUOmaLwZXSZVYUBLE8Fj 1okt4_8zGfCsGpsHl9rwqlyoVIZ4VadldWw"}}, "wmk":"P6UjESsvq5bATPHc5vPfEcs5u1pCkLVBCv5_aYdHdwQG3rpPihHj2g"} ], "Index":2} [Empty trailer] Figure 25 Hallam-Baker Expires February 28, 2019 [Page 24] Internet-Draft DARE Data Container August 2018 10. Appendix B public long PreviousFrame (long Frame) { long x2 = Frame + 1; long d = 1; while (x2 > 0) { if ((x2 & 1) == 1) { return x2 == 1 ? (d / 2) - 1 : Frame - d; } d = d * 2; x2 = x2 / 2; } return 0; } Figure 26 11. References 11.1. Normative References [draft-hallambaker-dare-message] Hallam-Baker, P., "Data At Rest Encryption Part 1: DARE Message", draft-hallambaker-dare-message-01 (work in progress), July 2018. [draft-hallambaker-jsonbcd] Hallam-Baker, P., "Binary Encodings for JavaScript Object Notation: JSON-B, JSON-C, JSON-D", draft-hallambaker- jsonbcd-13 (work in progress), July 2018. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. [RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2014. [RFC7515] Jones, M., Bradley, J., and N. Sakimura, "JSON Web Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May 2015. [RFC7516] Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)", RFC 7516, DOI 10.17487/RFC7516, May 2015. Hallam-Baker Expires February 28, 2019 [Page 25] Internet-Draft DARE Data Container August 2018 11.2. Informative References [BLOCKCHAIN] Chain.com, "Blockchain Specification". [draft-hallambaker-mesh-architecture] Hallam-Baker, P., "Mathematical Mesh: Architecture", draft-hallambaker-mesh-architecture-04 (work in progress), September 2017. [RFC5652] Housley, R., "Cryptographic Message Syntax (CMS)", STD 70, RFC 5652, DOI 10.17487/RFC5652, September 2009. [ZIPFILE] PKWARE Inc, "APPNOTE.TXT - .ZIP File Format Specification", October 2014. 11.3. URIs [1] http://mathmesh.com/Documents/draft-hallambaker-dare- container.html [2] http://mathmesh.com/Documents/draft-hallambaker-dare- container.html [3] http://mathmesh.com/Documents/draft-hallambaker-dare- container.html [4] http://mathmesh.com/Documents/draft-hallambaker-dare- container.html Author's Address Phillip Hallam-Baker Comodo Group Inc. Email: philliph@comodo.com Hallam-Baker Expires February 28, 2019 [Page 26]