More Instant Messaging Interoperability T. Ralston Internet-Draft M. Hodgson Intended status: Standards Track The Matrix.org Foundation C.I.C. Expires: 25 September 2023 24 March 2023 Linearized Matrix API draft-ralston-mimi-linearized-matrix-00 Abstract Matrix is an existing openly specified decentralized secure communications protocol able to provide a framework for instant messaging interoperability. Matrix rooms (aka conversations) currently use a Directed Acyclic Graph (DAG) for persisting events/ messages. Servers broadcast their changes to the DAG to every other server in order to create new events. This model provides excellent decentralization characteristics and conversation history replication, but proves complex when aiming to use Matrix strictly for interoperability between today's existing messaging service providers, which often do not persist chat history serverside, and do not seek to replicate it between servers. This document explores an API surface for Matrix which optimizes for ease of interoperability at the expense of decentralized conversation history at a per-room level. We call this API surface "Linearized Matrix". About This Document This note is to be removed before publishing as an RFC. The latest revision of this draft can be found at https://turt2live.github.io/ietf-mimi-linearized-matrix/draft- ralston-mimi-linearized-matrix.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft- ralston-mimi-linearized-matrix/. Discussion of this document takes place on the More Instant Messaging Interoperability Working Group mailing list (mailto:mimi@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/mimi/. Subscribe at https://www.ietf.org/mailman/listinfo/mimi/. Source for this draft and an issue tracker can be found at https://github.com/turt2live/ietf-mimi-linearized-matrix. Ralston & Hodgson Expires 25 September 2023 [Page 1] Internet-Draft Linearized Matrix API March 2023 Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 25 September 2023. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3 3. Identifiers . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Room Architecture . . . . . . . . . . . . . . . . . . . . . . 4 5. Event Signing . . . . . . . . . . . . . . . . . . . . . . . . 5 6. Membership . . . . . . . . . . . . . . . . . . . . . . . . . 5 7. State Events API . . . . . . . . . . . . . . . . . . . . . . 6 8. Room Events API . . . . . . . . . . . . . . . . . . . . . . . 6 9. Room Transfers . . . . . . . . . . . . . . . . . . . . . . . 7 10. Other APIs . . . . . . . . . . . . . . . . . . . . . . . . . 7 11. Matrix Room Version . . . . . . . . . . . . . . . . . . . . . 7 12. Anti-Abuse and Anti-Spam . . . . . . . . . . . . . . . . . . 8 13. DAG-Compatible Event Structure . . . . . . . . . . . . . . . 8 14. Security Considerations . . . . . . . . . . . . . . . . . . . 10 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 Ralston & Hodgson Expires 25 September 2023 [Page 2] Internet-Draft Linearized Matrix API March 2023 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 16.1. Normative References . . . . . . . . . . . . . . . . . . 10 16.2. Informative References . . . . . . . . . . . . . . . . . 11 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 12 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 1. Introduction At a high level, rooms using Linearized Matrix have a single server which owns that room. The owner can change, but will typically be the server which created the room. All other servers are known as participating servers and call the owner server to send events. The owner server is then responsible for informing all the other servers of any changes/messages in the room. Many aspects for how Matrix works as an interoperable messaging framework is described by [I-D.ralston-mimi-matrix-framework]. This document replaces the eventual consistency model, federation API, and DAG-related features of the framework document by presenting rooms as a single, flat, array of events, without being incompatible with those same replaced components. This document does not currently define a transport layer for the Linearized Matrix API, instead focusing its efforts on the operational aspects of a room. 2. Conventions and Definitions This document additionally uses the following definitions: * Owner Server: The server responsible for holding the room history, accepting new events, etc. * Participant Server: Every other server. Note that a server may inherit this role even if not (currently) participating in the room. *TODO*: Merge/add definitions from framework to here, such as "homeserver", "user", etc. 3. Identifiers *TODO*: Expand upon this section. A room ID has the format !localpart:domain, where the localpart is an opaque string and the domain provides global uniqueness. The domain does not indicate that the room exists on that server, just that it was originally created there. Ralston & Hodgson Expires 25 September 2023 [Page 3] Internet-Draft Linearized Matrix API March 2023 A user ID has the format @localpart:domain, where the localpart is again an opaque string and the domain is where the user was created (the server owns the user account). 4. Room Architecture As mentioned, rooms over Linearized Matrix have a concept of an "owning server". Typically the room owner will be the server which created the room, however ownership can shift as needed. The room owner is responsible for applying the room version semantics/access controls and distributing the changes to other applicable servers (called participant servers). At an implementation level, it should be possible for an owning server to use a DAG if it so wishes, however for the protocol considerations a room has a single flat array to store state changes and room events. Room state is the same as non-Linearized Matrix: represented by an event type and state key tuple which maps to a state event. "Current state" is simply the most recent instance of each event type and state key pair in the array. To send an event into the room, each "participant server" (non-owner) asks the owner to send it to the room. The owner applies access controls to the event, following Matrix's existing access controls (power levels, bans, server_acls etc.) and then adds the event to the room's array, and sends it out to all participating servers (including the original sender, for simplicity of implementation). If the owner would like to send an event, it simply adds the event to the array (assuming such an action is valid) and broadcasts it. The owner server MUST follow the access control semantics defined by the room's current state - it MUST NOT make up its own rules. For instance, the owning server must only let Alice invite Bob to a room if Alice has permission to invite, and if Alice's server sent the invite event. Each room additionally records which Matrix room version it is using for access control behaviours, such as Authorization Rules [MxV10AuthRules]. This is required for when rooms gain a DAG- compatible server in them. Note that this document introduces new semantics requiring a new room version. Ralston & Hodgson Expires 25 September 2023 [Page 4] Internet-Draft Linearized Matrix API March 2023 5. Event Signing Events are signed by the participant/original server to ensure the owning server is not spoofing events on behalf of another server. The exact details for how a server's signing keys are shared to other servers is left as a transport consideration, however signing keys are currently expected to be Ed25519 keys. In the existing Matrix Federation APIs, a PDU [MxV10PDUFormat] contains an event and has several DAG-specific fields to it. When using the Linearized Matrix API, we introduce a concept of a _Linear PDU_ which looks similar to a regular room event, but has all non- essential fields removed. { // the room ID the event is sent within "room_id": "!room:example.org", // the implied (or explicit) event type "type": "org.example.event_type", // for state events, even if an empty string "state_key": "", // the user ID of the sender "sender": "@user:example.org", // milliseconds since epoch "origin_server_ts": 123456789, // the domain of the room owner "authorized_sending_server": "owner.example.org", "content": { // the normal event content }, "hashes": { "sha256": "" } } [MxContentHashCalculation] The Linear PDU is then redacted [MxRedaction], canonicalized [MxCanonicalJSON], and signed [MxSigning]. The signature is supplied to the owner server alongside the event itself for sending to the room. 6. Membership After a room is created (by an imagined /createRoom API, for example), it will exist on a single server: the owner's. This is not particularly helpful if the goal is to talk to other people, so a way to involve others in the conversation is needed. Ralston & Hodgson Expires 25 September 2023 [Page 5] Internet-Draft Linearized Matrix API March 2023 Matrix currently has membership states for join, leave, invite, kick, ban, and knock (request invite). These states have their own set of rules governed by the room version to prevent cases of, for example, ban evasion. *TODO*: Describe those membership transitions. Currently specified in the Client-Server API https://spec.matrix.org/v1.6/client-server- api/#room-membership (we should move that). The owner server broadcasts successful membership changes as m.room.member events to all participant servers in the room, including the sending server. A server is considered to be "in the room" if it has at least one user with join membership state. 7. State Events API Matrix, and therefore Linearized Matrix, tracks changes to the room as _state events_. State events have both an event type and state key to differentiate them from room (or non-state) events. While history for state changes is stored in the room, only the most recent change for an event type and state key pair is considered "current state". For example, the current room name is the most recent m.room.name state event. As mentioned above, a transport layer would be responsible for the request/response structure for this API, however a need would be present to send (arbitrary) state events, read those state events back, and read the whole of current state (including membership). 8. Room Events API Room events include messages and redactions, as well as messaging features like reactions, edits, etc. These may be encrypted in supported rooms (ones which specify an encryption algorithm in their room state), and in their unencrypted form will use the Matrix concept of Extensible Events. *TODO*: Update the message format I-D for MSC1767 extensible events and link it here. https://github.com/matrix-org/matrix-spec- proposals/blob/main/proposals/1767-extensible-events.md A transport layer would specify request/response structures for sending, receiving, reading, and discovering nearby events (for scrollback purposes). Ralston & Hodgson Expires 25 September 2023 [Page 6] Internet-Draft Linearized Matrix API March 2023 9. Room Transfers The current room owner would be stored as a state event within the room, defaulting to the room creator. To transfer ownership, the current owner chooses a participant server and requests that it accept the ownership role. If the participant server agrees to take ownership, it would create and sign a new room ownership state event. The current owner then signs the ownership state event itself and sends it to all participating servers (including the new owner), just as it would for any other event. All requests from that point forward now go to the new owner, and the old owner becomes a regular participating server. *TODO*: What do you do if the owner server dies or partitions before transferring to a successor? 10. Other APIs *TODO*: Expand upon this section. A transport layer would specify request/response structures for: * Media/attachment distribution * APIs to support end-to-end encryption * Ephemeral data such as receipts, typing notifications, and presence * Resolving an identifer to a room ID * User profiles (display names and avatars) * Other APIs as required to support interoperable messaging 11. Matrix Room Version The first room version which supports Linearized Matrix will base its requirements on Matrix's existing Room Version 10 definition [MxRoomVersion10]. Changes will be made to support the following features: * A description of the authorization rules when not using a DAG. * The DAG-compatible signing structure for events. * The use of Matrix's Extensible Events content format. Ralston & Hodgson Expires 25 September 2023 [Page 7] Internet-Draft Linearized Matrix API March 2023 *TODO*: Expand upon this section with formal details of what the above looks like for a room version. 12. Anti-Abuse and Anti-Spam *TODO*: Expand upon this section. In a Matrix room, state events get appended to the DAG/array to show intent. If a server wishes to decline the request, such as in the case where the recipient server believes an invite is spammy, it can do so by sending another event to the room. For example, an antispam system might issue redactions for messages which look spammy on behalf of a room admin. 13. DAG-Compatible Event Structure Linearized Matrix is essentially an alternative API for accessing normal Matrix rooms over federation, which means servers which support a full-blown DAG can still join and participate in the room. This is critical in order to avoid breaking compatibility with today's fully-decentralized Matrix, and provides a way to decentralize ownership of rooms even if large messaging providers are themselves not able to implement full decentralization yet. [I-D.nottingham-avoiding-internet-centralization] With DAG-compatible servers in the room, the DAG-compatible servers talk to each other directly as they do with the current Matrix APIs. Any DAG-compatible server which can also speak Linearized Matrix can connect to the owner server - effectively trunking Linearized Matrix into normal Matrix and tracking its events into the DAG. As long as servers speaking Linearized Matrix uphold the room's access controls, then they appear as a single logical DAG-compatible server to normal Matrix, and will maintain consistency with the rest of normal Matrix. Normally, events are checked for signatures from the "origin" server implied by the sender on an event. Events sent with the Linearized Matrix API are already signed by the participant server to ensure the owner server isn't spoofing them, however an owner server might not always be DAG-compatible itself. To remedy this, owner servers can delegate their DAG involvement to a DAG-compatible server in the room. Delegating to a DAG-compatible server means creating a _Delegated Linear PDU_ from a _Linear PDU_. The owner moves the authorized_sending_server value (which should be itself) to original_authorized_sending_server then populates authorized_sending_server with the domain name for the DAG-compatible server it is using. The owner server then signs the Delegated Linear Ralston & Hodgson Expires 25 September 2023 [Page 8] Internet-Draft Linearized Matrix API March 2023 PDU and sends it to the DAG-compatible server, which then appends all the DAG-specific fields and signs the resulting PDU itself before sending it to all the other DAG-compatible servers in the room. Note that while a Delegated Linear PDU modifies the structure that was signed as a Linear PDU, it is easily possible to reconstruct a Linear PDU from a Delegated Linear PDU. Similarly, a DAG-ready PDU can be redacted down to a Delegated Linear PDU with ease. A complete DAG-ready PDU would look like: { // the room ID the event is sent within "room_id": "!room:example.org", // the implied (or explicit) event type "type": "org.example.event_type", // for state events, even if an empty string "state_key": "", // the user ID of the sender "sender": "@user:example.org", // milliseconds since epoch "origin_server_ts": 123456789, // the domain of the room owner "original_authorized_sending_server": "owner.example.org", // DAG-capable server "authorized_sending_server": "dag.example.org", "content": { // the normal event content }, "hashes": { "sha256": "" }, // other event format stuff: "auth_events": ["$event1", "$event2", "$etc"], "depth": 42, "prev_events": ["$event3", "$event4", "$etc2"], "signatures": { "dag.example.org": { "ed25519:abc": "" }, "owner.example.org": { "ed25519:def": "" }, "example.org": { "ed25519:ghi": "" } } } Ralston & Hodgson Expires 25 September 2023 [Page 9] Internet-Draft Linearized Matrix API March 2023 When validating the signatures [MxSignatureValidation] on this PDU, DAG-capable servers would apply the following algorithm. If at any point the check fails, the algorithm bails. 1. If an original_authorized_sending_server is present, construct the implied Linear PDU from the PDU and validate the signature for the server implied by the sender. Additionally, redact the PDU down to a Delegated Linear PDU and validate the signature for the authorized_sending_server value. 2. If an original_authorized_sending_server is NOT present, redact the PDU down to a Linear PDU and validate the signature for the authorized_sending_server value. 3. Without considering the signatures from the domains in previous steps, verify the signatures per normal. Note that the step where the signature for the "origin server" (defined as the one implied by the PDU's sender) is implicitly checked as part of step 1 and 2, and not actually possible to verify in the traditional sense. That particular step in event validation is therefore skipped when step 1 or 2 is performed. *TODO*: How does an owner server pick a DAG server to communicate with, and how does the owner receive events from the DAG to send to participant servers? One option might be to have DAG-capable servers identify themselves during joins with the owner server, then the current owner can transfer ownership to the DAG-capable server. The DAG-capable owner would simply shuffle events around internally to feed both API surfaces, though this means all DAG-capable servers need to implement both API surfaces. 14. Security Considerations *TODO*: Expand upon this section. As discussed in the Event Signing section, we ensure servers are not able to spoof events. 15. IANA Considerations This document has no IANA actions. 16. References 16.1. Normative References Ralston & Hodgson Expires 25 September 2023 [Page 10] Internet-Draft Linearized Matrix API March 2023 [I-D.ralston-mimi-matrix-framework] Ralston, T., "Matrix as a Messaging Framework", Work in Progress, Internet-Draft, draft-ralston-mimi-matrix- framework-01, 13 March 2023, . 16.2. Informative References [I-D.nottingham-avoiding-internet-centralization] Nottingham, M., "Internet Centralization: What Can Standards Do?", Work in Progress, Internet-Draft, draft- nottingham-avoiding-internet-centralization-09, 17 February 2023, . [MxCanonicalJSON] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Appendices | Canonical JSON", 2023, . [MxContentHashCalculation] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Federation API | Calculating the Content Hash", 2023, . [MxRedaction] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Client-Server API | Redaction Algorithm", 2023, . [MxRoomVersion10] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Room Version 10", 2023, . [MxSignatureValidation] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Federation API | Validating Hashes and Signatures", 2023, . Ralston & Hodgson Expires 25 September 2023 [Page 11] Internet-Draft Linearized Matrix API March 2023 [MxSigning] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Appendices | Signing", 2023, . [MxV10AuthRules] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Room Version 10 | Authorization Rules", 2023, . [MxV10PDUFormat] The Matrix.org Foundation C.I.C., "Matrix Specification | v1.6 | Room Version 10 | Event Format", 2023, . Acknowledgments Thank you to the Matrix Spec Core Team (SCT), and in particular Richard van der Hoff, for exploring how Matrix rooms could be represented as a linear structure, leading to this document. Authors' Addresses Travis Ralston The Matrix.org Foundation C.I.C. Email: travisr@matrix.org Matthew Hodgson The Matrix.org Foundation C.I.C. Email: matthew@matrix.org Ralston & Hodgson Expires 25 September 2023 [Page 12]