JMAP N. Jenkins
Internet-Draft FastMail
Intended status: Standards Track C. Newman
Expires: May 30, 2019 Oracle
November 26, 2018

JSON Meta Application Protocol
draft-ietf-jmap-core-11

Abstract

This document specifies a protocol for clients to access JSON-based data objects efficiently, with support for push and out-of-band binary data upload/download.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on May 30, 2019.

Copyright Notice

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.


Table of Contents

1. Introduction

JMAP is a protocol for synchronising data, such as mail, calendars or contacts, between a client and a server. It is optimised for mobile and web environments, and aims to provide a consistent interface to different data types.

This specification is for the generic mechanism of data synchronisation. Further specifications define the data models for different data types that may be synchronised via JMAP.

JMAP is designed to make efficient use of limited network resources. Multiple API calls may be batched in a single request to the server, reducing round trips and improving battery life on mobile devices. Push connections remove the need for polling, and an efficient delta update mechanism ensures a minimum of data is transferred.

JMAP is designed to be horizontally scalable to a very large number of users. This is facilitated by separate end points for users after login, the separation of binary and structured data, and a data model for sharing that does not allow data dependencies between accounts.

1.1. Notational conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

The underlying format used for this specification is JSON. Consequently, the terms "object" and "array" as well as the four primitive types (strings, numbers, booleans, and null) are to be interpreted as described in Section 1 of [RFC7159]. Unless otherwise noted, all the property names and values are case sensitive.

Some examples in this document contain "partial" JSON documents used for illustrative purposes. In these examples, three periods "..." are used to indicate a portion of the document that has been removed for compactness.

Unless otherwise specified, examples of API exchanges only show the methodCalls array of the Request object or the methodResponses array of the Response object. For compactness, the rest of the Request/Response object is omitted.

Type signatures are given for all JSON values in this document. The following conventions are used:

Other types may also be given, with their representation defined elsewhere in this document.

Object properties may also have a set of attributes defined along with the type signature. These have the following meanings:

1.2. The Id data type

All record ids are assigned by the server, and are immutable.

Where Id is given as a datatype, it means a String of at least 1 and maximum 255 octets in size, and MUST only contain characters from the "URL and Filename safe" Base 64 Alphabet, as defined in section 5 of [RFC4648]. This is the ASCII alphanumeric characters (A-Za-z0-9), hyphen (-), and underscore (_).

These characters are safe to use in almost any context (e.g., filesystems, URIs, IMAP atoms). For maximum safety, servers should also follow defensive allocation strategies to avoid creating risks where glob completion or data type detection may be present (e.g., on filesystems or in spreadsheets). In particular, it is wise to avoid:

A good solution to these issues is to prefix every id with a single alphabetical character.

1.3. The Int and PositiveInt data types

Where Int is given as a data type, it means an integer in the range -2^53+1 <= value <= 2^53-1, the safe range for integers stored in a floating-point double, represented as a JSON Number.

Where PositiveInt is given as a data type, it means an Int where the value MUST be in the range 0 <= value <= 2^53-1.

1.4. The Date and UTCDate data types

Where Date is given as a type, it means a string in [RFC3339] date-time format. To ensure a normalised form, the time-secfrac MUST always be omitted if zero, and any letters in the string (e.g. "T" and "Z") MUST be upper-case. For example, "2014-10-30T14:12:00+08:00".

Where UTCDate is given as a type, it means a Date where the time-offset component MUST be Z (i.e. it must be in UTC time). For example, "2014-10-30T06:12:00Z".

1.5. JSON as the data encoding format

JSON is a text-based data interchange format as specified in [RFC7159]. The I-JSON format defined in [RFC7493] is a strict subset of this, adding restrictions to avoid potentially confusing scenarios (for example, it mandates that an object MUST NOT have two members with the same name).

All data sent from the client to the server or from the server to the client (except binary file upload/download) MUST be valid I-JSON according to the RFC, and is therefore case-sensitive and encoded in UTF-8 ([RFC3629]).

1.6. Terminology

1.6.1. User

A user represents a set of permissions relating to what data can be seen.

1.6.2. Accounts

An account is a collection of data. A single account may contain an arbitrary set of data types, for example a collection of mail, contacts and calendars.

All data belong to a single account. Most JMAP methods take a mandatory accountId argument that specifies on which account the operations are to take place.

An account is not the same as a user, although it is common for a primary account to directly belong to the user. For example, you may have an account that contains data for a group or business, to which multiple users have access.

A single set of credentials may provide access to multiple accounts, for example if another user is sharing their mail with the logged in user, or if there is a group account.

In the event of a severe internal error, a server may have to reallocate ids or do something else that violates standard JMAP data constraints for an account. In this situation, the data on the server is no longer compatible with cached data the client may have from before. The server MUST treat this as though the account has been deleted and then recreated with a new account id. Clients will then be forced to throw away any data with the old account id and refetch all data from scratch.

1.6.3. Data types and records

JMAP provides a uniform interface for creating, retrieving, updating and deleting various types of objects. A data type is a collection of named, typed properties, just like the schema for a database table. Each instance of a data type is called a record.

The id of a record is immutable, and normally assigned by the server. The id MUST be unique among all records of the same type within the same account. Ids may clash across accounts, or for two records of different types within the same account.

1.7. The JMAP API model

JMAP uses HTTP [RFC7230] to expose API, Push, Upload and Download resources. Implementations MUST support HTTP/1.1, and MAY support later versions. All HTTP requests MUST use [RFC5246] TLS (HTTPS) transport. Support for common HTTP mechanisms such as redirection and caching are assumed.

All HTTP requests MUST be authenticated. Servers MUST conform with the [RFC7235] HTTP Authentication framework to reject requests that fail authentication and inform the client of available authentication schemes.

Clients MUST understand and be able to handle standard HTTP status codes appropriately.

An authenticated client can fetch the JMAP Session object with details about the data and capabilities the server can provide as shown in section 2. The client may then exchange data with the server in the following ways:

  1. The client may make an API request to the server to get or set structured data. This request consists of an ordered series of method calls. These are processed by the server, which then returns an ordered series of responses. This is described in sections 3 to 5.
  2. The client may download or upload binary files from/to the server. This is detailed in section 6.
  3. The client may connect to a push channel on the server, to be notified when data has changed. This is explained in section 7.

1.8. Vendor-specific extensions

Individual services will have custom features they wish to expose over JMAP. This may take the form of extra data types and/or methods not in the spec, or extra arguments to JMAP methods, or extra properties on existing data types (which may also appear in arguments to methods that take property names).

The server can advertise custom extensions it supports by including the identifiers in the capabilities object. Identifiers for vendor extensions MUST be a URL belonging to a domain owned by the vendor, to avoid conflict. The URL SHOULD resolve to documentation for the changes the extension makes.

To ensure compatibility with clients that don't know about a specific custom extension, and for compatibility with future versions of JMAP, to use an extension the client MUST opt in by passing the appropriate capability identifier in the using array of the Request object, as described in section 3.2. The server MUST only follow the specifications that are opted-into and behave as though it does not implement anything else when processing a request.

2. The JMAP Session resource

You need two things to connect to a JMAP server:

  1. The URL for the JMAP Session resource. This may be requested directly from the user, or discovered automatically based on a username domain (see section 2.2 below).
  2. Credentials to authenticate with. How to obtain credentials is out of scope for this specification.

An authenticated GET request to the JMAP Session resource MUST return the details about the data and capabilities the server can provide to the client given those credentials.

The response to a successful request is a JSON-encoded JMAP Session object. It has the following properties:

To ensure future compatibility, other properties MAY be included on the JMAP Session object. Clients MUST ignore any properties they are not expecting.

Implementors must take care to avoid inappropriate caching of the session object at the HTTP layer. Since the client should only refetch when it detects there is a change (via the sessionState property of an API response), it is recommended to disable HTTP caching altogether, for example by setting Cache-Control: no-cache, no-store, must-revalidate on the response.

2.1. Example

In the following example JMAP Session object, the user has access to their own mail and contacts via JMAP, as well as read-only access to shared mail from another user. The server is advertising a custom https://example.com/apis/foobar capability.

{
  "username": "john@example.com",
  "accounts": {
    "13824": {
      "name": "john@example.com",
      "isPersonal": true,
      "isReadOnly": false,
      "hasDataFor": [
        "urn:ietf:params:jmap:mail",
        "urn:ietf:params:jmap:contacts"
      ]
    },
    "97813": {
      "name": "jane@example.com",
      "isPersonal": false,
      "isReadOnly": true,
      "hasDataFor": [ "urn:ietf:params:jmap:mail" ]
    }
  },
  "primaryAccounts": {
    "urn:ietf:params:jmap:mail": "13824",
    "urn:ietf:params:jmap:contacts": "13824"
  },
  "capabilities": {
    "urn:ietf:params:jmap:core": {
      "maxSizeUpload": 50000000,
      "maxConcurrentUpload": 8,
      "maxSizeRequest": 10000000,
      "maxConcurrentRequest": 8,
      "maxCallsInRequest": 32,
      "maxObjectsInGet": 256,
      "maxObjectsInSet": 128,
      "collationAlgorithms": [
        "i;ascii-numeric",
        "i;ascii-casemap",
        "i;unicode-casemap"
      ]
    },
    "https://example.com/apis/foobar": {
      "maxFoosFinangled": 42
    },
    ...
  },
  "apiUrl": "https://jmap.example.com/api/",
  "downloadUrl": "https://jmap.example.com/download/{accountId}/{blobId}/{name}?accept={type}",
  "uploadUrl": "https://jmap.example.com/upload/{accountId}/",
  "eventSourceUrl": "https://jmap.example.com/eventsource/",
  "state": "75128aab4b1b"
}

2.2. Service autodiscovery

There are two standardised autodiscovery methods in use for internet protocols:

A JMAP-supporting host for the domain example.com SHOULD publish a SRV record _jmap._tcp.example.com which gives a hostname and port (usually port 443). The JMAP Session resource is then https://${hostname}[:${port}]/.well-known/jmap (following any redirects).

If the client has a username in the form of an email address, it MAY use the domain portion of this to attempt autodiscovery of the JMAP server.

3. Structured data exchange

The client may make an API request to the server to get or set structured data. This request consists of an ordered series of method calls. These are processed by the server, which then returns an ordered series of responses.

3.1. Making an API request

To make an API request, the client makes an authenticated POST request to the API resource, which is defined by the apiUrl property in the JMAP Session object.

The request MUST be of type application/json and consist of a single JSON Request object. If successful, the response MUST also be of type application/json and consist of a single Response object.

3.2. The Request object

A Request object has the following properties:

Future specifications MAY add further properties to the Request object to extend the semantics. To ensure forwards compatibility, a server MUST ignore any other properties it does not understand on the JMAP request object.

3.2.1. Example request

{
  "using": [ "urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail" ],
  "methodCalls": [
    [ "method1", {
      "arg1": "arg1data",
      "arg2": "arg2data"
    }, "c1" ],
    [ "method2", {
      "arg1": "arg1data"
    }, "c2" ],
    [ "method3", {}, "c3" ]
  ]
}

3.3. The Response object

A Response object has the following properties:

Unless otherwise specified, if the method call completed successfully its response name is the same as the method name in the request.

3.3.1. Example response:

{
  "methodResponses": [
    [ "method1", {
      "arg1": 3,
      "arg2": "foo"
    }, "c1" ],
    [ "method2", {
      "isBlah": true
    }, "c2" ],
    [ "anotherResponseFromMethod2", {
      "data": 10,
      "yetmoredata": "Hello"
    }, "c2"],
    [ "error", {
      "type":"unknownMethod"
    }, "c3" ]
  ],
  "sessionState": "75128aab4b1b"
}

3.4. Omitting arguments

An argument to a method may be specified to have a default value. If omitted by the client, the server MUST treat the method call the same as if the default value had been specified. Similarly, the server MAY omit any argument in a response which has the default value.

Unless otherwise specified in a method description, null is the default value for any argument in a request or response where this is allowed by the type signature. Other arguments may only be omitted if an explicit default value is defined in the method description.

3.5. Errors

There are three different levels of granularity at which an error may be returned in JMAP.

When an API request is made, the request as a whole may be rejected due to rate limiting, malformed JSON, request for an unknown capability etc. In this case the entire request is rejected with an appropriate HTTP error response code, and an additional JSON body with more detail for the client.

Provided the request itself is syntactically valid, the methods within it are executed sequentially by the server. Each method may individually fail, for example if invalid arguments are given, or an unknown method name is called.

Finally, methods that make changes to the server state often act upon a number of different records within a single call. Each record change may be separately rejected with a SetError, as described in section 5.3.

3.5.1. Request-level errors

When an HTTP error response is returned to the client, the server SHOULD return a JSON "problem details" object as the response body, as per [RFC7807].

The following problem types are defined:

3.5.1.1. Example

{
  "type": "urn:ietf:params:jmap:error:unknownCapability",
  "status": 400,
  "detail": "The request object used capability 'https://example.com/apis/foobar', which is not supported by this server."
}

3.5.2. Method-level errors

If a method encounters an error, the appropriate error response MUST be inserted at the current point in the methodResponses array and, unless otherwise specified, further processing MUST NOT happen within that method call.

Any further method calls in the request MUST then be processed as normal. Errors at the method level MUST NOT generate an HTTP-level error.

An error response looks like this:

[ "error", {
  "type": "unknownMethod"
}, "client-id" ]

The response name is error, and it MUST have a type property. Other properties may be present with further information; these are detailed in the error type descriptions where appropriate.

With the exception of serverPartialFail, the externally-visible state of the server MUST NOT have changed if an error is returned at the method level.

The following error types are defined which may be returned for any method call where appropriate:

serverUnavailable: Some internal server resource was temporarily unavailable. Attempting the same operation later (perhaps after a backoff with a random factor) may succeed.

serverFail: An unexpected or unknown error occurred during the processing of the call. A description property should provide more details about the error. The method call made no changes to the server's state. Attempting the same operation again is expected to fail again. Contacting the service administrator is likely necessary to resolve this problem if it is persistent.

serverPartialFail: Some, but not all expected changes described by the method occurred. The client MUST re-synchronise impacted data to determine server state. Use of this error is strongly discouraged.

unknownMethod: The server does not recognise this method name.

invalidArguments: One of the arguments is of the wrong type or otherwise invalid, or a required argument is missing. A description property MAY be present to help debug with an explanation of what the problem was. This is a non-localised string, and is not intended to be shown directly to end users.

invalidResultReference: The method used a back reference for one of its arguments (see section 3.6), but this failed to resolve.

forbidden: The method and arguments are valid, but executing the method would violate an ACL or other permissions policy.

accountNotFound: The accountId does not correspond to a valid account.

accountNotSupportedByMethod: The accountId given corresponds to a valid account, but the account does not support this data type.

accountReadOnly: This method call would modify state in an account that is read-only (as returned on the corresponding Account object in the JMAP Session resource).

Further possible errors for a particular method are specified in the method descriptions.

Further general errors MAY be defined in future RFCs. Should a client receive an error type it does not understand, it MUST treat it the same as the serverFail type.

3.6. References to previous method results

To allow clients to make more efficient use of the network and avoid round trips, an argument to one method can be taken from the result of a previous method call in the same request.

To do this, the client prefixes the argument name with #. The value is a ResultReference object as described below. When processing a method call, the server MUST first check the arguments object for any names beginning with #. If found, the back reference should be resolved and the value used as the "real" argument. The method is then processed as normal. If any back reference fails to resolve, the whole method MUST be rejected with an invalidResultReference error. If an argument object contains the same argument name in normal and referenced form (e.g. foo and #foo), the method MUST return an invalidArguments error.

A ResultReference object has the following properties:

To resolve:

  1. Find the first response with a client id identical to the resultOf property of the ResultReference in the methodResponses array from previously processed method calls in the same request. If none, evaluation fails.
  2. If the response name is not identical to the name property of the ResultReference, evaluation fails.
  3. Apply the path to the arguments object of the response (the second item in the response array) following the [RFC6901] JSON Pointer algorithm, except with the following addition in Section 4 (Evaluation):

If the currently referenced value is a JSON array, the reference token may be exactly the single character *, making the new referenced value the result of applying the rest of the JSON pointer tokens to every item in the array and returning the results in the same order in a new array. If the result of applying the rest of the pointer tokens to a value was itself an array, its items should be included individually in the output rather than including the array itself (i.e. the result is flattened from an array of arrays to a single array).

As a simple example, suppose we have the following API request methodCalls:

[[ "Foo/changes", {
    "accountId": "1",
    "sinceState": "abcdef"
}, "t0" ],
[ "Foo/get", {
    "accountId": "1",
    "#ids": {
        "resultOf": "t0",
        "name": "Foo/changes",
        "path": "/created"
    }
}, "t1" ]]

After executing the first method call the methodResponses array is:

[[ "Foo/changes", {
    "accountId": "1",
    "oldState": "abcdef",
    "newState": "123456",
    "hasMoreChanges": false,
    "created": [ "f1", "f4" ],
    "updated": [],
    "destroyed": []
}, "t0" ]]

To execute the Foo/get call, we look through the arguments and find there is one with a # prefix. To resolve this, we apply the algorithm above:

  1. Find the first response with client id "t0". The Foo/changes response fulfils this criterion.
  2. Check the response name is the same as in the result reference. It is, so this is fine.
  3. Apply the path as a JSON pointer to the arguments object. This simply selects the "created" property, so the result of evaluating is: [ "f1", "f4" ]

The JMAP server now continues to process the Foo/get call as though the arguments were:

{
    "accountId": "1",
    "ids": [ "f1", "f4" ]
}

Now a more complicated example using the JMAP Mail data model: fetch the "from"/"date"/"subject" for every email in the first 10 threads in the Inbox (sorted newest first):

[[ "Email/query", {
  "accountId": "1",
  "filter": { "inMailbox": "id_of_inbox" },
  "sort": [{ "property": "receivedAt", "isAscending": false }],
  "collapseThreads": true,
  "position": 0,
  "limit": 10,
  "calculateTotal": true
}, "t0" ],
[ "Email/get", {
  "accountId": "1",
  "#ids": {
    "resultOf": "t0",
    "name": "Email/query",
    "path": "/ids"
  },
  "properties": [ "threadId" ]
}, "t1" ],
[ "Thread/get", {
  "accountId": "1",
  "#ids": {
    "resultOf": "t1",
    "name": "Email/get",
    "path": "/list/*/threadId"
  }
}, "t2" ],
[ "Email/get", {
  "accountId": "1",
  "#ids": {
    "resultOf": "t2",
    "name": "Thread/get",
    "path": "/list/*/emailIds"
  },
  "properties": [ "from", "receivedAt", "subject" ]
}, "t3" ]]

After executing the first 3 method calls the methodResponses array might be:

[[ "Email/query", {
    "accountId": "1",
    "queryState": "abcdefg",
    "canCalculateChanges": true,
    "position": 0,
    "total": 101,
    "ids": [ "msg1023", "msg223", "msg110", "msg93", "msg91", "msg38", "msg36", "msg33", "msg11", "msg1" ]
}, "t0" ],
[ "Email/get", {
    "accountId": "1",
    "state": "123456",
    "list": [{
        "id": "msg1023",
        "threadId": "trd194"
    }, {
        "id": "msg223",
        "threadId": "trd114"
    },
    ...
    ],
    "notFound": []
}, "t1" ],
[ "Thread/get", {
    "accountId": "1",
    "state": "123456",
    "list": [{
        "id": "trd194",
        "emailIds": [ "msg1020", "msg1021", "msg1023" ]
    }, {
        "id": "trd114",
        "emailIds": [ "msg201", "msg223" ]
    },
    ...
    ],
    "notFound": []
}, "t2" ]]

So to execute the final Email/get call, we look through the arguments and find there is one with a # prefix. To resolve this, we apply the algorithm:

  1. Find the first response with client id "t2". The "Thread/get" response fulfils this criterion.
  2. "Thread/get" is the name specified in the result reference, so this is fine.
  3. Apply the path as a JSON pointer to the arguments object. Token-by-token:
    1. list: get the array of thread objects
    2. *: for each of the items in the array:
      1. emailIds: get the array of email ids
      2. Concatenate these into a single array of all the ids in the result.

The JMAP server now continues to process the Email/get call as though the arguments were:

{
    "accountId": "1",
    "ids": [ "msg1020", "msg1021", "msg1023", "msg201", "msg223", ... ],
    "properties": [ "from", "receivedAt", "subject" ]
}

3.7. Security

As always, the server must be strict about data received from the client. Arguments need to be checked for validity; a malicious user could attempt to find an exploit through the API. In case of invalid arguments (unknown/insufficient/wrong type for data etc.) the method MUST return an invalidArguments error and terminate.

3.8. Concurrency

Method calls within a single request MUST be executed in order. However, method calls from different concurrent API requests may be interleaved. This means that the data on the server may change between two method calls within a single API request.

4. The Core/echo method

The Core/echo method returns exactly the same arguments as it is given. It is useful for testing you have a valid authenticated connection to a JMAP API endpoint.

4.1. Example

Request:

[[ "Core/echo", {
  "hello": true,
  "high": 5
}, "b3ff" ]]

Response:

[[ "Core/echo", {
  "hello": true,
  "high": 5
}, "b3ff" ]]

5. Standard methods and naming convention

JMAP provides a uniform interface for creating, retrieving, updating and deleting objects of a particular type. For a Foo data type, records of that type would be fetched via a Foo/get call and modified via a Foo/set call. Delta updates may be fetched via a Foo/changes call. These methods all follow a standard format as described below.

Not all types may have all methods. Specifications defining types MUST specify which methods are available for the type.

5.1. /get

Objects of type Foo are fetched via a call to Foo/get.

It takes the following arguments:

The response has the following arguments:

The following additional error may be returned instead of the Foo/get response:

requestTooLarge: The number of ids requested by the client exceeds the maximum number the server is willing to process in a single method call.

5.2. /changes

When the state of the set of Foo records in an account changes on the server (whether due to creation, updates or deletion), the state property of the Foo/get response will change. The Foo/changes method allows a client to efficiently update the state of its Foo cache to match the new state on the server. It takes the following arguments:

The response has the following arguments:

If a record has been created AND updated since the old state, the server SHOULD just return the id in the created list, but MAY return it in the updated list as well.

If a record has been updated AND destroyed since the old state, the server SHOULD just return the id in the destroyed list, but MAY return it in the updated list as well.

If a record has been created AND destroyed since the old state, the server SHOULD remove the id from the response entirely, but MAY include it in the destroyed list, and if so MAY also include it in the created list.

If a maxChanges is supplied, or set automatically by the server, the server MUST ensure the number of ids returned across created, updated and destroyed does not exceed this limit. If there are more changes than this between the client's state and the current server state, the server SHOULD generate an update to take the client to an intermediate state, from which the client can continue to call Foo/changes until it is fully up to date. If it is unable to calculate an intermediate state, it MUST return a cannotCalculateChanges error response instead.

When generating intermediate states, the server may choose how to divide up the changes. For many types it will provide a better user experience to return the more recent changes first, as this is more likely to be what the user is most interested in. The client can then continue to page in the older changes while the user is viewing the newer data. For example, suppose a server went through the following states:

A -> B -> C -> D -> E

And a client asks for changes from state B. The server might first get the ids of records created, updated or destroyed between states D and E, returning them with:

state: "B-D-E"
hasMoreChanges: true

The client will then ask for the change from state B-D-E, and the server can return the changes between states C and D, returning:

state: "B-C-E"
hasMoreChanges: true

Finally the client will request the changes from B-C-E and the server can return the changes between states B and C, returning:

state: "E"
hasMoreChanges: false

Should the state on the server be modified in the middle of all this (to F), the server still does the same but now when the update to state E is returned, it would indicate that it still has more changes for the client to fetch.

Where multiple changes to a record are split across different intermediate states, the server MUST NOT return a record as created in a later response than one which gives it as updated or destroyed, and MUST NOT return a record as destroyed before a response that gives it as created or updated. The server may have to coalesce multiple changes to a record to satisfy this requirement.

The following additional errors may be returned instead of the Foo/changes response:

cannotCalculateChanges: The server cannot calculate the changes from the state string given by the client. Usually due to the client's state being too old, or the server being unable to produce an update to an intermediate state when there are too many updates. The client MUST invalidate its Foo cache.

Maintaining state to allow calculation of Foo/changes can be expensive for the server, but always returning cannotCalculateChanges severely increases network traffic and resource usage for the client. To allow efficient sync, servers SHOULD be able to calculate changes from any state string that was given to a client within the last 30 days (but of course may support calculating updates from states older than this).

5.3. /set

Modifying the state of Foo objects on the server is done via the Foo/set method. This encompasses creating, updating and destroying Foo records. This allows the server to sort out ordering and dependencies that may exist if doing multiple operations at once (for example to ensure there is always a minimum number of a certain record type).

The Foo/set method takes the following arguments:

Each creation, modification or destruction of an object is considered an atomic unit. It is permissible for the server to commit changes to some objects but not others, however it MUST NOT only commit part of an update to a single record (e.g. update a name property but not a count property, if both are supplied in the update object).

The final state MUST be valid after the Foo/set is finished, however the server may have to transition through invalid intermediate states (not exposed to the client) while processing the individual create/update/destroy requests. For example, suppose there is a "name" property that must be unique. A single method call could rename an object A => B, and simultaneously rename another object B => A. If the final state is valid, this is allowed. Otherwise, each creation, modification or destruction of an object should be processed sequentially and accepted/rejected based on the current server state.

If a create, update or destroy is rejected, the appropriate error MUST be added to the notCreated/notUpdated/notDestroyed property of the response and the server MUST continue to the next create/update/destroy. It does not terminate the method.

If an id given cannot be found, the update or destroy MUST be rejected with a notFound set error.

The server MAY skip an update (rejecting it with a willDestroy SetError) if that object is destroyed in the same /set request.

Some records may hold references to other records (foreign keys). That reference may be set (via create or update) in the same request as the referenced record is created. To do this, the client refers to the new record using its creation id prefixed with a #. The order of the method calls in the request by the client MUST be such that the record being referenced is created in the same or an earlier call. The server thus never has to look ahead. Instead, while processing a request the server MUST keep a simple map for the duration of the request of creation id to record id for each newly created record, so it can substitute in the correct value if necessary in later method calls.

Creation ids are not scoped by type but are a single map for all types. A client SHOULD NOT reuse a creation id anywhere in the same API request. If a creation id is reused, the server MUST map the creation id to the most recently created item with that id. To allow easy proxying of API requests, an initial set of creation id to real id values may be passed with a request (see The Request object in section 3.2) and the final state of the map passed out with the response (see section 3.3).

The response has the following arguments:

A SetError object has the following properties:

The following SetError types are defined and may be returned for set operations on any record type where appropriate:

Other possible SetError types MAY be given in specific method descriptions. Other properties MAY also be present on the SetError object, as described in the relevant methods.

The following additional errors may be returned instead of the Foo/set response:

requestTooLarge: The total number of objects to create, update or destroy exceeds the maximum number the server is willing to process in a single method call.

stateMismatch: An ifInState argument was supplied and it does not match the current state.

5.4. /copy

The only way to move Foo records between two different accounts is to copy them using the Foo/copy method, then once the copy has succeeded, delete the original. The onSuccessDestroyOriginal argument allows you to try to do this in one method call, however note that the two different actions are not atomic, and so it is possible for the copy to succeed but the original not to be destroyed for some reason.

The copy is conceptually in three phases:

  1. Reading the current values from the "from" account.
  2. Writing the new copies to the other account.
  3. Destroying the originals in the "from" account, if requested.

Data may change in between phases due to concurrent requests.

The Foo/copy method takes the following arguments:

Each record copy is considered an atomic unit which may succeed or fail individually.

The response has the following arguments:

The SetError may be any of the standard set errors that may be returned for a create or update. In addition, the following SetError is defined:

alreadyExists: The server forbids duplicates and the record already exists in the target account. An existingId property of type Id MUST be included on the error object with the id of the existing record.

The following additional errors may be returned instead of the Foo/copy response:

fromAccountNotFound: The fromAccountId does not correspond to a valid account.

fromAccountNotSupportedByMethod: The fromAccountId given corresponds to a valid account, but the account does not support this data type.

stateMismatch: An ifInState argument was supplied and it does not match the current state, or an ifFromInState argument was supplied and it does not match the current state in the from account.

5.5. /query

For data sets where the total amount of data is expected to be very small, clients can just fetch the complete set of data and then do any sorting/filtering locally. However, for large data sets (e.g. multi-gigabyte mailboxes), the client needs to be able to search/sort/window the data type on the server.

A query on the set of Foos in an account is made by calling Foo/query. This takes a number of arguments to determine which records to include, how they should be sorted, and which part of the result should be returned (the full list may be very long). The result is returned as a list of Foo ids.

A call to Foo/query takes the following arguments:

If an anchor argument is given, then after filtering and sorting the anchor is looked for in the results. If found, the anchor offset is then added to its index. If the resulting index is now negative, it is clamped to 0. This index is now used exactly as though it were supplied as the position argument. If the anchor is not found, the call is rejected with an anchorNotFound error.

If an anchor is specified, any position argument supplied by the client MUST be ignored. If no anchor is supplied, any anchor offset argument MUST be ignored.

A client can use anchor instead of position to find the index of an id within a large set of results.

The response has the following arguments:

The following additional errors may be returned instead of the Foo/query response:

anchorNotFound: An anchor argument was supplied, but it cannot be found in the results of the query.

unsupportedSort: The sort is syntactically valid, but includes a property the server does not support sorting on, or a collation method it does not recognise.

unsupportedFilter: The filter is syntactically valid, but the server cannot process it. If the filter was the result of a user's search input, the client SHOULD suggest the user simplify their search.

5.6. /queryChanges

The Foo/queryChanges method allows a client to efficiently update the state of a cached query to match the new state on the server. It takes the following arguments:

The response has the following arguments:

The result of this is that if the client has a cached sparse array of foo ids corresponding to the results in the old state:

fooIds = [ "id1", "id2", null, null, "id3", "id4", null, null, null ]

then if it splices out all foos in the removed array:

removed = [ "id2", ... ];
fooIds => [ "id1", null, null, "id3", "id4", null, null, null ]

and splices in (in order) all of the foos in the added array:

added = [{ id: "id5", index: 0, ... }];
fooIds => [ "id5", "id1", null, null, "id3", "id4", null, null, null ]

and truncates or extends to the new total length, then the results will now be in the new state.

The following additional errors may be returned instead of the Foo/queryChanges response:

tooManyChanges: There are more changes than the client's maxChanges argument. Each item in the removed or added array is considered as one change. The client may retry with a higher max changes or invalidate its cache of the query results.

cannotCalculateChanges: The server cannot calculate the changes from the queryState string given by the client. Usually due to the client's state being too old. The client MUST invalidate its cache of the query results.

5.7. Examples

Suppose we have a type Todo with the following properties:

Suppose also that all the standard methods are defined for this type, and the FilterCondition object supports a hasKeyword property to match todos with the given keyword.

A client might want to display the list of todos with either a "music" keyword or a "video" keyword, so it makes the following method call:

[[ "Todo/query", {
  "accountId": "x",
  "filter": {
    "operator": "OR",
    "conditions": [
      { "hasKeyword": "music" },
      { "hasKeyword": "video" }
    ]
  },
  "sort": [{ "property": "title" }],
  "position": 0,
  "limit": 10
}, "0" ],
[ "Todo/get", {
  "accountId": "x",
  "#ids": {
    "resultOf": "0",
    "name": "Todo/query",
    "path": "/ids"
  }
}, "1" ]]

This would query the server for the set of todos with a keyword of "music", sorted by title, and limited to the first 10 results. It fetches the full object for each of these Todos using back-references to reference the result of the query. The response might look something like:

[[ "Todo/query", {
  "accountId": "x",
  "queryState": "y13213",
  "canCalculateChanges": true,
  "position": 0,
  "ids": [ "a", "b", "c", "d", "e", "f", "g", "h", "i", "j" ]
}, "0" ],
[ "Todo/get", {
  "accountId": "x",
  "state": "10324",
  "list": [{
    "id": "a",
    "title": "Practise Piano",
    "keywords": {
      "music": true,
      "beethoven": true,
      "mozart": true,
      "liszt": true,
      "rachmaninov": true
    },
    "neuralNetworkTimeEstimation": 3600
  }, {
    "id": "b",
    "title": "Listen to Daft Punk",
    "keywords": {
      "music": true,
      "trance": true
    },
    "neuralNetworkTimeEstimation": 18000
  },
  ...
  ]
}, "1" ]]

Now suppose the user adds a keyword "chopin" and removes the keyword "mozart" from the "Practise Piano" task. The client may send the whole object to the server, as this is a valid PatchObject:

[[ "Todo/set", {
  "accountId": "x",
  "ifInState": "10324",
  "update": {
    "a": {
      "id": "a",
      "title": "Practise Piano",
      "keywords": {
        "music": true,
        "beethoven": true,
        "chopin": true,
        "liszt": true,
        "rachmaninov": true
      },
      "neuralNetworkTimeEstimation": 360
    }
  }
}, "0" ]]

or it may send a minimal patch:

[[ "Todo/set", {
  "accountId": "x",
  "ifInState": "10324",
  "update": {
    "a": {
      "keywords/chopin": true,
      "keywords/mozart": null
    }
  }
}, "0" ]]

The effect is exactly the same on the server in either case, and presuming the server is still in state "10324" it will probably return success:

[[ "Todo/set", {
  "accountId": "x",
  "oldState": "10324",
  "newState": "10329",
  "updated": {
    "a": {
      "neuralNetworkTimeEstimation": 5400
    }
  }
}, "0" ]]

The server changed the "neuralNetworkTimeEstimation" property on the object as part of this change; as this changed in a way not explicitly requested by the PatchObject sent to the server, it is returned with the "updated" confirmation.

Let us now add a subtodo to our new "Practice Piano" todo. In this example we can see the use of a reference to a creation id to allow us to set a foreign key reference to a record created in the same request:

[[ "Todo/set", {
  "accountId": "x",
  "create": {
    "k15": {
      "title": "Warm up with scales"
    }
  },
  "update": {
    "a": {
      "subTodoIds": [ "#k15" ]
    }
  }
}, "0" ]]

Now, suppose another user deleted the "Listen to Daft Punk" todo. The first user will receive a push notification (see section 7) with the changed state string for the "Todo" type. Since the new string does not match its current state, it knows it needs to check for updates. It may make a request like:

[[ "Todo/changes", {
  "accountId": "x",
  "sinceState": "10324",
  "maxChanges": 50
}, "0" ],
[ "Todo/queryChanges", {
  "accountId": "x",
  "filter": {
    "operator": "OR",
    "conditions": [
      { "hasKeyword": "music" },
      { "hasKeyword": "video" }
    ]
  },
  "sort": [{ "property": "title" }],
  "sinceQueryState": "y13213",
  "maxChanges": 50
}, "1" ]]

and receive in response:

[[ "Todo/changes", {
  "accountId": "x",
  "oldState": "10324",
  "newState": "871903",
  "hasMoreChanges": false,
  "created": [],
  "updated": [],
  "destroyed": ["b"]
}, "0" ],
[ "Todo/queryChanges", {
  "accountId": "x",
  "oldQueryState": "y13213",
  "newQueryState": "y13218",
  "removed": ["b"],
  "added": null
}, "1" ]]

Suppose the user has access to another account y, for example a team account shared between multiple users. To move an existing Todo from account x, the client would call:

[[ "Todo/copy", {
  "fromAccountId": "x",
  "accountId": "y",
  "create": {
    "k5122": {
      "id": "a"
    }
  },
  "onSuccessDestroyOriginal": true
}, "0" ]]

The server successfully copies the Todo to a new account (where it receives a new id) and deletes the original. Due to the implicit call to "Todo/set", there are two responses to the single method call, both with the same client id:

[[ "Todo/copy", {
  "fromAccountId": "x",
  "accountId": "y",
  "created": {
    "k5122": {
      "id": "97"
    }
  },
  "oldState": "c1d64ecb038c",
  "newState": "33844835152b"
}, "0" ],
[ "Todo/set", {
  "accountId": "x",
  "oldState": "871903",
  "newState": "871909",
  "destroyed": [ "a" ],
  ...
}, "0" ]]

5.8. Proxy considerations

JMAP has been designed to allow an API endpoint to easily proxy through to one or more JMAP servers. This may be useful for load balancing, augmenting capabilities, or presenting a single endpoint to accounts hosted on different JMAP servers (splitting the request based on each method's "accountId" argument). The proxy need only understand the general structure of a JMAP Request object, it does not need to know anything specifically about the methods and arguments it will pass through to other servers.

If splitting up the methods in a request to call them on different backend servers, the proxy must do two things to ensure back-references and creation id references resolve the same as if the entire request were processed on a single server:

  1. It must pass a createdIds property with each subrequest. If this is not given by the client, an empty object should be used for the first subrequest. The createdIds property of each subresponse should be passed on in the next subrequest.
  2. It must resolve back-references to previous method results that were processed on a different server. This is a relatively simple syntactic substitution, described in section 3.6.

When splitting a request based on accountId, proxy implementors do need to be aware of /copy methods, that copy between accounts. If the accounts are on different servers, the proxy will have to implement this functionality directly.

6. Binary data

Binary data is referenced by a blobId in JMAP, and uploaded/downloaded separately to the core API. The blobId solely represents the raw bytes of data, not any associated metadata such as a file name or content type. Such metadata is stored alongside the blobId in the object referencing it. The data represented by a blobId is immutable.

Any blobId that exists within an account may be used when creating/updating another object in that account. For example, an Email type may have a blobId that represents the [RFC5322] representation of the message. A client could create a new Email object with an attachment and use this blobId, in effect attaching the old message to the new one. Similarly it could attach any existing attachment of an old message without having to download and upload it again.

When the client uses a blobId in a create/update, the server MAY assign a new blobId to refer to the same binary data within the new/updated object. If it does so, it MUST return any properties that contain a changed blobId in the created/updated response so the client gets the new ids.

A blob that is not referenced by a JMAP object (e.g. as a message attachment) MAY be deleted by the server to free up resources. Uploads (see below) are initially unreferenced blobs. To ensure interoperability:

6.1. Uploading binary data

There is a single endpoint which handles all file uploads for an account, regardless of what they are to be used for. The JMAP Session object has an uploadUrl property in [RFC6570] URI Template (level 1) format, which MUST contain a variable called accountId. The client may use this template in combination with an accountId to get the URL of the file upload resource.

To upload a file, the client submits an authenticated POST request to the file upload resource.

A successful request MUST return a single JSON object with the following properties as the response:

If identical binary content to an existing blob in the account is uploaded, the existing blobId MAY be returned.

Clients should use the blobId returned in a timely manner. Under rare circumstances the server may have deleted the blob before the client uses it; the client should keep a reference to the local file so it can upload it again in such a situation.

When an HTTP error response is returned to the client, the server SHOULD return a JSON "problem details" object as the response body, as per [RFC7807].

As access controls are often determined by the object holding the reference to a blob, unreferenced blobs MUST only be accessible to the uploader, even in shared accounts.

6.2. Downloading binary data

The JMAP Session object has a downloadUrl property, which is in [RFC6570] URI Template (level 1) format. The URL MUST contain variables called accountId, blobId, type and name.

To download a file, the client makes an authenticated GET request to the download URL with the appropriate variables substituted in:

As the data for a particular blobId is immutable, and thus the response in the generated download URL is too, implementors are recommended to set long cache times for successful responses, for example Cache-Control: private, max-age=31536000.

When an HTTP error response is returned to the client, the server SHOULD return a JSON "problem details" object as the response body, as per [RFC7807].

6.3. Blob/copy

Binary data may be copied between two different accounts using the Blob/copy method, rather than having to download then reupload on the client.

The Blob/copy method takes the following arguments:

The response has the following arguments:

The SetError may be any of the standard set errors that may be returned for a create. In addition, the notFound SetError error may be returned if the blobId to be copied cannot be found.

The following additional errors may be returned instead of the Blob/copy response:

fromAccountNotFound: The fromAccountId included with the request does not correspond to a valid account.

7. Push

Push notifications allow clients to efficiently update (almost) instantly to stay in sync with data changes on the server. In JMAP, push notifications occur out-of-band (i.e. not over the same connection as API exchanges), so that they can make use of efficient native push mechanisms on different platforms.

The general model for push is simple and sends minimal data over the push channel. The format allows multiple changes to be coalesced into a single push update, and the frequency of pushes to be rate limited by the server. It doesn't matter if some push events are dropped before they reach the client; it will still get all changes next time it syncs.

7.1. The StateChange object

When something changes on the server, the server pushes a StateChange object to the client. A StateChange object has the following properties:

7.1.1. Example

In this example, the server has amalgamated a few changes together across two different accounts the user has access to, before pushing the following StateChange object to the client:

{
  "changed": {
    "a3123": {
      "Email": "d35ecb040aab",
      "EmailDelivery": "428d565f2440",
      "CalendarEvent": "87accfac587a"
    },
    "a43461d": {
      "Mailbox": "0af7a512ce70",
      "CalendarEvent": "7a4297cecd76"
    }
  }
}

The client can compare the state strings with its current state for the Email, CalendarEvent etc. object types in the appropriate accounts to see if it needs to fetch changes. If the client is itself making changes, it may receive a StateChange object while the /set API call is in flight. It can wait until the call completes and then compare if the new state string after the /set is the same as was pushed in the StateChange object; if so, it does not need to waste a request asking for changes it already knows.

7.2. PushSubscription

A push subscription is a message delivery context established between the client and a push service. A PushSubscription object has the following properties:

Clients may create a push subscription on the JMAP server, which will then make a POST request to the associated push endpoint whenever an event occurs.

The POST request MUST have a content type of application/json and contain the UTF-8 JSON encoded StateChange object as the body. The request MUST have a TTL header, and MAY have Urgency and/or Topic headers, as specified in section 5 of [RFC8030]. The server is expected to understand and handle HTTP status responses in a reasonable manner. See the security considerations in section 8.6 for a discussion of the risks in connecting to unknown servers.

The use of this push endpoint conforms with the use of a push endpoint by an Application Server as defined in [RFC8030]. A client MAY use the rest of [RFC8030] in combination with its own Push Server to form a complete end-to-end solution, or MAY rely on alternative mechanisms to ensure the delivery of the pushed data after it leaves the JMAP server.

The push subscription is tied to the credentials used to authenticate the API request that created it. Should these credentials expire or be revoked, the push subscription MUST be destroyed by the JMAP server.

When these credentials have their own expiry (i.e. it is a session with a timeout), the server SHOULD NOT set or bound the expiry time for the push subscription given by the client, but MUST expire it when the session expires.

When these credentials are not time bounded (e.g. [RFC7617] Basic Authentication), the server SHOULD set an expiry time for the push subscription if none given, and limit the expiry time if set too far in the future. This maximum expiry time MUST be at least 48 hours in the future and SHOULD be at least 7 days in the future. An app running on a mobile device may only be able to refresh the push subscription lifetime when it is in the foreground, and so this gives a reasonable timeframe to allow this to happen.

In the case of separate access and refresh credentials, as in [RFC6749] Oauth 2.0, the server SHOULD tie the push subscription to the validity of the refresh token rather than the access token, and behave according to whether this is time-limited or not.

7.2.1. PushSubscription/get

Standard /get method, except it does not take or return an accountId argument, as push subscriptions are not tied to specific accounts. It also does not return a state argument. The ids argument may be null to fetch all at once.

As the url and keys properties may contain data that is private to a particular device, the values for these properties MUST NOT be returned. If the properties argument is null or omitted, the server MUST default to all properties excluding these two. If one of them is explicitly requested, the method call MUST be rejected with a forbidden error.

7.2.2. PushSubscription/set

Standard /set method except it does not take or return an accountId argument, as push subscriptions are not tied to specific accounts. It also does not take an ifInState argument or return oldState or newState arguments.

The url and keys properties are immutable; if the client wishes to change these, it must destroy the current push subscription and create a new one.

The client may update the expires property to extend (or, less commonly, shorten) the lifetime of a push subscription. The server MAY modify the proposed new expiry time to enforce server-defined limits.

Clients SHOULD NOT update or destroy a push subscription that they did not create (i.e. has a deviceClientId that they do not recognise).

7.2.3. Example

A client with deviceClientId a889-ffea-910 fetches the set of push subscriptions currently on the server, making an API request with:

[[ "PushSubscription/get", {
  "ids": null
}, "0" ]]

Which returns:

[[ "PushSubscription/get", {
  "list": [{
      "id": "e50b2c1d-9553-41a3-b0a7-a7d26b599ee1",
      "deviceClientId": "b37ff8001ca0",
      "expires": "2018-01-31T00:13:21Z",
      "types": [ "Todo" ]
  }, {
      "id": "f2d0aab5-e976-4e8b-ad4b-b380a5b987e4",
      "deviceClientId": "8980f37f6c71",
      "expires": "2018-02-12T05:55:00Z",
      "types": [ "Mailbox", "Email", "EmailDelivery" ]
  }],
  "notFound": []
}, "0" ]]

Since neither of the returned push subscription objects have the client's deviceClientId, it knows it does not have a current push subscription active on the server. So it creates one, sending this request:

[[ "PushSubscription/set", {
  "create": {
    "4f29": {
      "deviceClientId": "a889-ffea-910",
      "url": "https://example.com/push/?device=8980f37f6c&client=12c6d086",
      "types": null
    }
  }
}, "0" ]]

The server creates the push subscription but limits the expiry time to 7 days in the future, returning this response:

[[ "PushSubscription/set", {
  "created": {
    "4f29": {
      "id": "043dcfa4-1dd4-41ef-9156-2c89b3b19c60",
      "keys": null,
      "expires": "2018-07-13T02:14:29Z"
    }
  }
}, "0" ]]

Two days later, the client updates the subscription to extend its lifetime, sending this request:

[[ "PushSubscription/set", {
  "update": {
    "043dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
      "expires": "2018-08-13T00:00:00Z"
    }
  }
}, "0" ]]

The server extends the expiry time, but only again to its maximum limit of 7 days in the future, returning this response:

[[ "PushSubscription/set", {
  "updated": {
    "043dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
      "expires": "2018-07-16T02:22:50Z"
    }
  }
}, "0" ]]

7.3. Event Source

Clients that can hold open TCP connections can connect directly to the JMAP server to receive push notifications via a text/event-stream resource, as described in <http://www.w3.org/TR/eventsource/>. This is a long running HTTP request down which the server can push data.

When a change occurs in the data on the server, it pushes an event called state to any connected clients, with the StateChange object as the data.

The server SHOULD also send a new event id that encodes the entire server state visible to the user immediately after sending a state event. When a new connection is made to the event-source endpoint, a client following the server-sent events specification will send a Last-Event-ID HTTP header with the last id it saw, which the server can use to work out whether the client has missed some changes. If so, it SHOULD send these changes immediately on connection.

The client may add a query parameter called types, with the value being a comma-separated list of type names. If present, the server MUST only push changes for the types in this list. If omitted, changes to all types are pushed.

The client may add a query parameter called closeafter with value state to the event-source resource URL when requesting the event-source resource. If set, the server MUST end the HTTP response after pushing a state event. This can be used by clients in environments where buffering proxies prevent the pushed data from arriving immediately, or indeed at all, when operating in the usual mode.

The client may add a query parameter called ping, with a positive integer value representing a length of time in seconds, e.g. ping=300. If set, the server MUST send an event called ping whenever this time elapses since the previous event was sent. This MUST NOT set a new event id.

The server MAY modify the ping interval given as a query parameter to be subject to a minimum and/or maximum value. For interoperability, servers MUST NOT have a minimum allowed value higher than 30 or a maximum allowed value less than 300.

The data for the ping event MUST be a JSON object containing an interval property, the value (type PositiveInt) being the interval in seconds the server is using to send pings (this may be different to the requested value if the server clamped it to be within a min/max value).

Clients can monitor for the ping event to help determine when the closeafter mode may be required.

Refer to the JMAP Session resource in section 2 for details on how to get the URL for the event-source resource. Requests to the resource MUST be authenticated.

A client MAY hold open multiple connections to the event-source resource, although it SHOULD try to use a single connection for efficiency.

8. Security considerations

8.1. Transport confidentiality

All HTTP requests MUST use [RFC5246] TLS (https) transport to ensure the confidentiality of data sent and received via JMAP. Clients MUST validate TLS certificate chains to protect against man-in-the-middle attacks.

8.2. Authentication scheme

A number of HTTP authentication schemes have been standardised (<https://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml>). Servers should take care to assess the security characteristics of different schemes in relation to their needs when deciding what to implement.

If offering the Basic authentication scheme, services are strongly recommended to not allow a user's regular password but require generation of a unique "app password" via some external mechanism for each client they wish to connect. This allows connections from different devices to be differentiated by the server, and access to be individually revoked.

8.3. Service autodiscovery

Unless secured by something like DNSSEC, autodiscovery of server details is vulnerable to a DNS poisoning attack leading to the client talking to an attacker's server instead of the real JMAP server. The attacker may then man-in-the-middle requests and depending on the authentication scheme, steal credentials to generate its own requests.

Clients that do not support SRV lookups are likely to try just using the /.well-known/jmap path directly against the domain of the username over HTTPS. Servers SHOULD ensure this path resolves or redirects to the correct JMAP Session resource to allow this to work. If this is not feasible, servers MUST ensure this path cannot be controlled by an attacker, as again it may be used to steal credentials.

8.4. JSON parsing

The security considerations of [RFC7159] apply to the use of JSON as the data interchange format.

8.5. Denial of service

A small request may result in a very large response, and require considerable work on the server if resource limits are not enforced. JMAP provides mechanisms for advertising and enforcing a wide variety of limits for mitigating this threat, including limits on number of objects fetched in a single method call, number of methods in a single request, number of concurrent requests, etc.

JMAP servers MUST implement sensible limits to mitigate against resource exhaustion attacks.

8.6. Connection to unknown push server

When a push subscription is registered, the application server will make POST requests to the given URL. There are a number of security considerations that MUST be considered when implementing this. The server MUST ensure the URL is externally resolvable to avoid server-side request forgery, where the server makes a request to a resource on its internal network.

A malicious client may use the push subscription to attempt to flood a 3rd party server with requests, creating a denial of service attack and masking the attacker's true identity. There is no guarantee the URL that was given to the JMAP server is actually a valid push server. The JMAP server MUST therefore be defensive in its handling of error responses when it tries to make a request to the given URL. Any permanent HTTP error MUST destroy the push subscription and prevent further attempts to send requests to that server. A 429 (Too Many Requests) response MUST cause the JMAP server to reduce the frequency of pushes; the JMAP push structure allows multiple to be coalesced into a single minimal object that will still ensure the client comes fully up to date when it resyncs.

8.7. Push encryption

When data changes, a small object is pushed with the new state strings for the types that have changed. While the data here is minimal, a passive man-in-the-middle attacker may be able to gain useful information. To ensure confidentiality, if the push is sent via a third party outside of the control of the client and JMAP server the client MUST specify encryption keys when establishing the PushSubscription.

The privacy and security considerations of [RFC8030] and [RFC8291] also all apply to the use of the PushSubscription mechanism.

9. IANA considerations

9.1. Assignment of jmap service name

IANA will assign the 'jmap' service name in the 'Service Name and Transport Protocol Port Number Registry' [RFC6335].

Service Name: jmap

Transport Protocol(s): tcp

Assignee: IESG

Contact: IETF Chair

Description: JSON Meta Application Protocol

Reference: this document

Assignment Notes: this service name was previously assigned under the name JSON Mail Access Protocol. This will be de-assigned and re-assigned with the approval of the previous assignee.

9.2. Registration of well-known URI suffix for JMAP

IANA will register the following well-known URI suffix for JMAP as described in [RFC5785]:

URI Suffix: jmap

Change Controller: IETF

Specification Document: this document, section 2.2.

9.3. Registration of the jmap URN sub-namespace

IANA will register the following URN sub-namespace in the "IETF URN Sub-namespace for Registered Protocol Parameter Identifiers" registry as described in [RFC3553].

Registered Parameter Identifier: jmap

Reference: this document, next section

IANA Registry Reference: {insert IANA registry URL for registry in next section, upon approval}

9.4. Creation of "JMAP Capabilities" registry

IANA will create a registry for JMAP capabilities as described in section 2. JMAP capabilities are advertised in the capabilities property of the JMAP Session resource. They are used to extend the functionality of a JMAP server. A capability is referenced by a URI. The JMAP capability URI can be a URN starting with urn:ietf:params:jmap: plus a unique suffix which is the index value in the jmap URN sub-namespace. Registration of a JMAP capability with another form of URI has no impact on the jmap URN sub-namespace.

This registry follows the expert review process unless the "intended use" field is common or placeholder in which case registration follows the specification required process.

A JMAP capability registration can have an intended use of common, placeholder, limited, or obsolete. IANA will list common use registrations prominently and separately from those with other intended use values.

The JMAP capability registration procedure is not a formal standards process, but rather an administrative procedure intended to allow community comment and sanity checking without excessive time delay.

A placeholder registration reserves part of the jmap urn namespace for another purpose but is typically not included in the capabilities property of the JMAP Session resource.

9.4.1. Preliminary community review

Notice of a potential JMAP common use registration SHOULD be sent to the jmap@ietf.org mailing list for review. This mailing list is appropriate to solicit community feedback on a proposed JMAP capability. Registrations that are not intended for common use MAY be sent to the list for review as well; doing so is entirely OPTIONAL, but is encouraged.

The intent of the public posting to this list is to solicit comments and feedback on the choice of capability name, the unambiguity of the specification document, and a review of any interoperability or security considerations. The submitter may submit a revised registration proposal or abandon the registration completely and at any time.

9.4.2. Submit request to IANA

Registration requests can be sent to iana@iana.org.

9.4.3. Designated expert review

For a limited use registration, the designated expert's (DE) primary concern is preventing name collisions and encouraging the submitter to document security and privacy considerations; a published specification is not required. For a common use registration, the DE is expected to confirm that suitable documentation as described in [RFC8126], Section 4.6, is available. The DE should also verify the capability does not conflict with work that is active or already published within the IETF.

Before a period of 30 days has passed, the DE will either approve or deny the registration request and publish a notice of the decision to the JMAP WG mailing list or its successor, as well as informing IANA. A denial notice must be justified by an explanation, and in the cases where it is possible, concrete suggestions on how the request can be modified so as to become acceptable should be provided.

9.4.4. Change procedures

Once a JMAP capability has been published by the IANA, the change controller may request a change to its definition. The same procedure that would be appropriate for the original registration request is used to process a change request.

JMAP capability registrations may not be deleted; capabilities that are no longer believed appropriate for use can be declared obsolete by a change to their "intended use" field; such capabilities will be clearly marked in the lists published by the IANA.

Significant changes to a capability's definition should be requested only when there are serious omissions or errors in the published specification. When review is required, a change request may be denied if it renders entities that were valid under the previous definition invalid under the new definition.

The owner of a JMAP capability may pass responsibility to another person or agency by informing the IANA; this can be done without discussion or review.

The IESG may reassign responsibility for a JMAP capability. The most common case of this will be to enable changes to be made to capabilities where the author of the registration has died, moved out of contact, or is otherwise unable to make changes that are important to the community.

9.4.5. JMAP Capabilities registry template:

Capability name: (see capability property in section 2)

Specification document:

Intended use: (one of common, limited, or obsolete)

Change controller: (IETF for standards-track/BCP RFCs)

Security and privacy considerations:

9.4.6. Initial registration for JMAP core

Capability Name: urn:ietf:params:jmap:core

Specification document: this document, section 2

Intended use: common

Change Controller: IETF

Security and privacy considerations: this document, section 8.

9.4.7. Registration for JMAP error placeholder in JMAP capabilities registry

Capability Name: urn:ietf:params:jmap:error:

Specification document: this document, next section.

Intended use: placeholder

Change Controller: IETF

Security and privacy considerations: this document, section 8.

9.5. Creation of "JMAP Error Codes" registry

IANA will create a registry for JMAP error codes. JMAP error codes appear in the "type" member of a JSON problem details object (as described in section 3.5.1), in the "type" member in a JMAP error object (as described in section 3.5.2), or the "type" member of a JMAP method-specific error object (such as SetError in section 5.3). When used in a problem details object, the prefix 'urn:ietf:params:jmap:error:' is always included, and when used in JMAP objects, the prefix is always omitted.

This registry follows the expert review process. Preliminary community review for this registry follows the same procedures as the JMAP capabilities registry but is optional. The change procedures for this registry are the same as the change procedures for the JMAP capabilities registry.

9.5.1. Designated expert review

The designated expert should review the following aspects of the registration:

  1. Verify the error code does not conflict with existing names.
  2. Verify the error code follows the syntax limitations (does not require URI encoding).
  3. Encourage the error code to follow the naming convention of previously registered errors.
  4. Encourage description of client behaviors that are recommended in response to the error code. These may distinguish the error code from other error codes.
  5. Encourage description of when the server should issue the error as opposed to some other error code.
  6. Encourage the submitter to note any security considerations associated with the error, if any. For example, an error code that might disclose existence of data the authenticated user does not have permission to know about.

Steps 3-6 are meant to promote a higher-quality registry. However, the expert is encouraged to approve any registration that would not actively harm JMAP interoperability to make this a relatively lightweight process.

9.5.2. JMAP Error Codes registry template:

JMAP Error Code:

Intended use: (one of common, limited, obsolete)

Change Controller: (IETF for standards-track/BCP RFCs)

Description or Reference:

9.5.3. Initial JMAP Error Codes registry

JMAP Error Code Intended Use Change Controller Description or Reference
accountNotFound common IETF RFC XXXX section 3.5.2
accountNotSupportedByMethod common IETF RFC XXXX section 3.5.2
accountReadOnly common IETF RFC XXXX section 3.5.2
anchorNotFound common IETF RFC XXXX section 5.5
alreadyExists common IETF RFC XXXX section 5.4
cannotCalculateChanges common IETF RFC XXXX sections 5.2 and 5.6
forbidden common IETF RFC XXXX sections 3.5.2, 5.3, and 7.2.1
fromAccountNotFound common IETF RFC XXXX sections 5.4 and 6.3
fromAccountNotSupportedByMethod common IETF RFC XXXX section 5.4
invalidArguments common IETF RFC XXXX section 3.5.2
invalidPatch common IETF RFC XXXX section 5.3
invalidProperties common IETF RFC XXXX section 5.3
notFound common IETF RFC XXXX section 5.3
notJSON common IETF RFC XXXX section 3.5.1
notRequest common IETF RFC XXXX section 3.5.1
overQuota common IETF RFC XXXX section 5.3
rateLimit common IETF RFC XXXX section 5.3
requestTooLarge common IETF RFC XXXX sections 5.1 and 5.3
invalidResultReference common IETF RFC XXXX section 3.5.2
serverFail common IETF RFC XXXX section 3.5.2
serverPartialFail limited IETF RFC XXXX section 3.5.2
serverUnavailable common IETF RFC XXXX section 3.5.2
singleton common IETF RFC XXXX section 5.3
stateMismatch common IETF RFC XXXX section 5.3
tooLarge common IETF RFC XXXX section 5.3
tooManyChanges common IETF RFC XXXX section 5.6
unknownCapability common IETF RFC XXXX section 3.5.1
unknownMethod common IETF RFC XXXX section 3.5.2
unsupportedFilter common IETF RFC XXXX section 5.5
unsupportedSort common IETF RFC XXXX section 5.5
willDestroy common IETF RFC XXXX section 5.3

10. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.
[RFC2782] Gulbrandsen, A., Vixie, P. and L. Esibov, "A DNS RR for specifying the location of services (DNS SRV)", RFC 2782, DOI 10.17487/RFC2782, February 2000.
[RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002.
[RFC3553] Mealling, M., Masinter, L., Hardie, T. and G. Klyne, "An IETF URN Sub-namespace for Registered Protocol Parameters", BCP 73, RFC 3553, DOI 10.17487/RFC3553, June 2003.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 2003.
[RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006.
[RFC4790] Newman, C., Duerst, M. and A. Gulbrandsen, "Internet Application Protocol Collation Registry", RFC 4790, DOI 10.17487/RFC4790, March 2007.
[RFC5051] Crispin, M., "i;unicode-casemap - Simple Unicode Collation Algorithm", RFC 5051, DOI 10.17487/RFC5051, October 2007.
[RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.2", RFC 5246, DOI 10.17487/RFC5246, August 2008.
[RFC5322] Resnick, P., "Internet Message Format", RFC 5322, DOI 10.17487/RFC5322, October 2008.
[RFC5785] Nottingham, M. and E. Hammer-Lahav, "Defining Well-Known Uniform Resource Identifiers (URIs)", RFC 5785, DOI 10.17487/RFC5785, April 2010.
[RFC6186] Daboo, C., "Use of SRV Records for Locating Email Submission/Access Services", RFC 6186, DOI 10.17487/RFC6186, March 2011.
[RFC6335] Cotton, M., Eggert, L., Touch, J., Westerlund, M. and S. Cheshire, "Internet Assigned Numbers Authority (IANA) Procedures for the Management of the Service Name and Transport Protocol Port Number Registry", BCP 165, RFC 6335, DOI 10.17487/RFC6335, August 2011.
[RFC6570] Gregorio, J., Fielding, R., Hadley, M., Nottingham, M. and D. Orchard, "URI Template", RFC 6570, DOI 10.17487/RFC6570, March 2012.
[RFC6749] Hardt, D., "The OAuth 2.0 Authorization Framework", RFC 6749, DOI 10.17487/RFC6749, October 2012.
[RFC6764] Daboo, C., "Locating Services for Calendaring Extensions to WebDAV (CalDAV) and vCard Extensions to WebDAV (CardDAV)", RFC 6764, DOI 10.17487/RFC6764, February 2013.
[RFC6838] Freed, N., Klensin, J. and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013.
[RFC6901] Bryan, P., Zyp, K. and M. Nottingham, "JavaScript Object Notation (JSON) Pointer", RFC 6901, DOI 10.17487/RFC6901, April 2013.
[RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2014.
[RFC7230] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing", RFC 7230, DOI 10.17487/RFC7230, June 2014.
[RFC7235] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol (HTTP/1.1): Authentication", RFC 7235, DOI 10.17487/RFC7235, June 2014.
[RFC7493] Bray, T., "The I-JSON Message Format", RFC 7493, DOI 10.17487/RFC7493, March 2015.
[RFC7617] Reschke, J., "The 'Basic' HTTP Authentication Scheme", RFC 7617, DOI 10.17487/RFC7617, September 2015.
[RFC7807] Nottingham, M. and E. Wilde, "Problem Details for HTTP APIs", RFC 7807, DOI 10.17487/RFC7807, March 2016.
[RFC8030] Thomson, M., Damaggio, E. and B. Raymor, "Generic Event Delivery Using HTTP Push", RFC 8030, DOI 10.17487/RFC8030, December 2016.
[RFC8126] Cotton, M., Leiba, B. and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017.
[RFC8291] Thomson, M., "Message Encryption for Web Push", RFC 8291, DOI 10.17487/RFC8291, November 2017.

Authors' Addresses

Neil Jenkins FastMail PO Box 234, Collins St West Melbourne, VIC 8007 Australia EMail: neilj@fastmailteam.com URI: https://www.fastmail.com
Chris Newman Oracle 440 E. Huntington Dr., Suite 400 Arcadia, CA 91006 United States of America EMail: chris.newman@oracle.com