Constrained RESTful Environments A. Castellani Internet-Draft University of Padova Intended status: Informational February 21, 2011 Expires: August 25, 2011 CoAP overhead: protocol analysis and reduction proposals draft-castellani-core-coap-overhead-00.txt Abstract This draft aims at providing an analysis of the current overhead present in CoAP and at proposing alternative formats leading to an even more compact procotol. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on August 25, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Castellani Expires August 25, 2011 [Page 1] Internet-Draft CoAP overhead February 2011 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Overhead analysis . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Message ID and Token . . . . . . . . . . . . . . . . . . . 3 2.2. Uri-* Options . . . . . . . . . . . . . . . . . . . . . . 4 2.3. Overhead in CoAP examples . . . . . . . . . . . . . . . . 4 3. Transport-related base header optimization . . . . . . . . . . 5 3.1. Simpler session matching . . . . . . . . . . . . . . . . . 5 3.1.1. Session matching rule . . . . . . . . . . . . . . . . 7 3.2. Smaller base header . . . . . . . . . . . . . . . . . . . 7 3.2.1. Action and Action Code . . . . . . . . . . . . . . . . 8 3.2.2. Multi-datagram transfers . . . . . . . . . . . . . . . 9 3.3. Overhead analysis with new base header . . . . . . . . . . 12 4. URI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 14 6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14 6.1. Normative References . . . . . . . . . . . . . . . . . . . 14 6.2. Informative References . . . . . . . . . . . . . . . . . . 14 Editorial Comments . . . . . . . . . . . . . . . . . . . . . . . . Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 15 Castellani Expires August 25, 2011 [Page 2] Internet-Draft CoAP overhead February 2011 1. Introduction Constrained Application Protocol (CoAP) [I-D.ietf-core-coap] design is focused on defining a REST protocol for constrained devices. CoAP's strength is mainly to be a light binary REST protocol mappable to HTTP and suitable for M2M communication. The design of such a protocol requires a lot of effort to effectively reduce the required overhead to a minimum, still leaving the resulting protocol simple enought for severely limited devices to handle it. This document aims at highlighting possible aspects of the protocol design not fully optimized. To the best of the author's knowledge, discussion about the outlined topics has not yet happened in the WG. The document is organized as follows: o Section 2 briefly analyzes the protocol overhead by providing some examples; o Section 3 discusses alternative base header formats and transport- related optimizations; o Section 4 describes possible URI encodings to reduce overhead (TBD). 2. Overhead analysis A synthetic description of the fields with higher impact on overhead and an example follow: o Section 2.1 briefly discusses the Message ID field and Token Option; o Section 2.2 outlines the overhead related Uri-* options (TBD); o Finally, Section 2.3 analyzes the overhead allocation of a sample CoAP exchange. 2.1. Message ID and Token In [I-D.ietf-core-coap] two different fields are available to perform session matching when concurrent sessions are present between the endpoints: o Message ID: defined in Sec. 3.1 of [I-D.ietf-core-coap] o Token: defined in Sec. 5.10.1 of [I-D.ietf-core-coap] The Message ID field is also intended to handle datagram duplication. The use of the aforementioned fields introduces a total overhead ranging between 2 and 11 bytes per message; moreover using two Castellani Expires August 25, 2011 [Page 3] Internet-Draft CoAP overhead February 2011 different fields to perform session matching seems an added complexity for constrained devices. As a side note, dynamic memory allocation is usually avoided on contrained devices. The Token Option as currently defined requires implementors on statically allocating server devices to fully reserve 8 bytes per request for Token independently on the size chosen by the client. 2.2. Uri-* Options TBD 2.3. Overhead in CoAP examples An overhead analysis of the example present in Section 2.2 of [I-D.ietf-core-coap] and for convenience reported hereafter in Figure 1 follows. Client Server | | | CON [MID=0xbc90], GET, /temperature, Token=0x71 ----->| | | |<--- ACK [MID=0xbc90], 2.00 OK, Token=0x71, Content=6B | Figure 1: CoAP example Table 1 and Table 2 analyze how the request/response bytes are allocated to the different fields. Req1/Res1 refer to the exact example shown in Figure 1. Req2/Res2 are referred to the same message exchange however with smaller Uri "/t" and 2B Content; the size of Uri/Content are under the control of the system developers and are thus expected to be reduced when needed. +---------------+--------------+--------------+ | Field(s) | Req1 (Ratio) | Req2 (Ratio) | +---------------+--------------+--------------+ | Base | 2B (0.11) | 2B (0.25) | | MID and Token | 4B (0.22) | 4B (0.50) | | URI | 12B (0.67) | 2B (0.25) | | Total | 18B (1.00) | 8B (1.00) | +---------------+--------------+--------------+ Table 1: Request size analysis Castellani Expires August 25, 2011 [Page 4] Internet-Draft CoAP overhead February 2011 +---------------+--------------+--------------+ | Field(s) | Res1 (Ratio) | Res2 (Ratio) | +---------------+--------------+--------------+ | Base | 2B (0.17) | 2B (0.25) | | MID and Token | 4B (0.33) | 4B (0.50) | | Content | 6B (0.50) | 2B (0.25) | | Total | 12B (1.00) | 8B (1.00) | +---------------+--------------+--------------+ Table 2: Response size analysis In Req1/Res1 results, it can be seen that Uri/Content have high impact on the total message size, however, when optimizing their size by using a shorter Uri and a more efficient content representation, their impact can be substantially reduced by system developers using CoAP (Res2/Req2). When Uri/Content are reduced in size as in Req2/ Res2, overhead related to MID and Token grows significantly, 50% of total message size in both Req2 and Res2. 3. Transport-related base header optimization The following section discusses some proposals to reduce the impact of overhead, related to the Message ID field and Token Option, by proposing an alternative and more compact design. This section is organized as follows: o Section 3.1 describes a more compact session matching technique; o Section 3.2 proposes an alternative smaller base header for CoAP and an extension useful for multi-datagram messages; o Section 3.3 shows the overhead reduction obtained with respect to the examples discussed in Section 2.3. 3.1. Simpler session matching [I-D.ietf-core-coap] uses UDP to transport protocol messages, mainly for two reasons: o reduced complexity in the constrained devices when compared to TCP o well-known performance issues are present when using TCP in multi- hop WPANs However, UDP lacks important characteristics needed by a REST application protocol, in particular CoAP needs a bidirectional reliable channel of arbitrary duration. To reproduce a suitable service on top of UDP, some transport features should be added: Castellani Expires August 25, 2011 [Page 5] Internet-Draft CoAP overhead February 2011 1. protocol session identification (e.g. request/response matching); 2. lost datagram retransmission; 3. duplicate datagram recognition; 4. congestion control; 5. ordered data transfer. If we assume that messages are transmitted a single datagram at a time, as long as we can identify to which session every message belongs, the overall protocol design is significantly simplified. For an extensive discussion on congestion control refer to [I-D.eggert-core-congestion-control] Assuming that between two hosts (IPs) there can exist at most one session between two specific UDP ports, protocol session matching can be performed without any further addition on the upper layer protocol. In a client/server protocol the server is usually spawned on a static port. However, even using a static server port, session matching is still feasible as long as concurrently active clients on a host can use different source ports. E.g. TCP successfully performs session matching using source/ destination IP and port. Avoiding more complex session matching techniques leads to a more interoperable protocol. In this way, session matching can be operated even by traditional network appliances (e.g., NATs/PATs). Example: NATs/PATs could incorrectly translate infrequent messages belonging to the same session to different external IPs/ports (e.g., observe notifications). The Token option is extensively used in [I-D.ietf-core-observe] and may be required for matching very long infrequent sessions. Some alternative methods for subscription functionality are proposed in Appendix A of [I-D.castellani-core-http-coap-mapping], relaxing or even fully removing the need for Token to keep matching with long- term subscriptions. The use of the proposed simpler session matching technique may not be suitable for some very specific CoAP applications. Regarding these special applications, the Token option can be used as an alternative parameter to distinguish concurrent sessions beyond the IPs and ports. Castellani Expires August 25, 2011 [Page 6] Internet-Draft CoAP overhead February 2011 3.1.1. Session matching rule To successfully perform session matching, as described in this section, it is sufficient that the following synthetic condition is verified: At any time, there MUST NOT exist two or more concurrent sessions having all the listed parameters equal between themselves: ( source IP, source UDP port, destination IP, destination UDP port ) If the Token option is present, it is used as an alternative parameter non-overlapping with source port. In this case the following list of parameters replaces the above list: ( source IP, 65536+Token, destination IP, destination UDP port ) 3.2. Smaller base header In this section two variants for a smaller base header format for CoAP are proposed. The proposed variants support a multi-datagram transfer with a simple extension, which is briefly discussed in Section 3.2.2. 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Ver| T | OC |R| A | AC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: Base header format The definition of the fields already introduced in Figure 6 of [I-D.ietf-core-coap] is still valid. New fields introduced in Figure 2 are defined as follows: Reserved field (R): 1-bit unsigned integer. This field is reserved for future use and MUST always be set to 0. If R field is set to 1, a RST message SHOULD be sent and the message MUST be dropped. Action (A): 3-bit unsigned integer. Indicates whether the message is empty (0) or is carrying a response (1-6) or a request (7). More details about the value of this field can be found in Table 3 Action Code (AC): 4-bit unsigned integer. Indicates more details about the kind of request (method), response (specific status) Castellani Expires August 25, 2011 [Page 7] Internet-Draft CoAP overhead February 2011 carried in the message. If the Action is an empty message, AC field usage is reserved for future use. Messages not containing any random field are known to introduce confusion when duplication has to be handled: when a duplicate is received, the duplication can be caused by an automatic retransmission (network duplication) or by the endpoint generating a new identical message (real duplication). Messages easily predicatable leave open the path to attacks similar to the well-known "TCP sequence number prediction attack" The aforementioned considerations leads to introducing the variant shown in Figure 3; discussion on the ML is encouraged to understand whether this issue is relevant in CoRE scenarios. If not relevant the following variant can be ignored. 0 1 2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Ver| T | OC |R| A | AC | Seq | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: Header format w/ sequence number The definition of the fields already introduced in Figure 2 is still valid. The Seq field introduced in Figure 3 is defined as follows: Sequence number (Seq): 8-bit unsigned integer. [Q1] Indicates the sequence number of the message. In the first message this value SHOULD be initialized to a random value. Each new non-empty message emitted from client or server endpoints MUST increase this value by 1. E.g. the response carries the Seq value contained in the request increased by 1. 3.2.1. Action and Action Code The Action field contains synthetic information about message semantics. By leveraging the information contained in the Action field (see Table 3), CoAP implementors can scalably choose the level of support to implement in each Action class. Example: On extremely constrained implementations, the Action Code can be read only for requests or for 2xx, 4xx, 5xx responses. For other actions the effects of the message do not depend upon the specific code received. Castellani Expires August 25, 2011 [Page 8] Internet-Draft CoAP overhead February 2011 +-------+----------------------------+ | Value | Action | +-------+----------------------------+ | 0 | Empty message | | 1 | 1xx Informational response | | 2 | 2xx Successful response | | 3 | 3xx Redirection response | | 4 | 4xx Client Error response | | 5 | 5xx Server Error response | | 6 | CoAP-specific response | | 7 | Request | +-------+----------------------------+ Table 3: CoAP Action values When the action is a request, the code field will contain the details about the method used to access the resource. The actual definition of the Action Code values to GET, POST, PUT and DELETE methods is out-of-scope of this document, however it can be easily obtained if the described compressed header is believed to be suitable to the CoAP protocol specification. As before, the actual mapping between Action Code and specific response codes is out-of-scope of this document, but an option for a simple assignment could be to use "direct YY mapping", that is deriving the value directly from YY of every response code (X.YY) as defined in Table 6 of [I-D.ietf-core-coap]. "Direct YY mapping" example: 5.02 Bad Gateway can be mapped to A=5 and AC=2. The response codes currently defined in CoAP are all mappable using "direct YY mapping". Even looking in [RFC2616] or in the more recent [I-D.ietf-httpbis-p2-semantics], all HTTP status codes except two 4xx status codes (416, 417) are mappable using "direct YY mapping". Moreover 416 and 417 status codes are intended for responses to complex request statements, currently out-of-scope from the CoRE charter and otherwise manageable using different AC mappings. 3.2.2. Multi-datagram transfers Limiting the CoAP message size to the datagram MTU can lead to some issues and limit its range of application, as discussed in [I-D.ietf-core-block]. [I-D.ietf-core-block] defines how to transfer large resource Castellani Expires August 25, 2011 [Page 9] Internet-Draft CoAP overhead February 2011 representations, by introducing the Block Option and defining a request format to get a specific part of a large resource. The Block option allows to transfer response content fragmented in parts of agreed size. Using this option, the client is enabled to request a specific part of the resource representation, and the resulting exchange is similar to subsequent HTTP Range requests (see Sec. 14.35 in [RFC2616]). Using this approach when dealing with large resources with non-static representations lead to an increased complexity in the upper layers, due to the necessary controls and buffering required to avoid transferring multiple parts belonging to different representations. A more traditional ordered data transfer technique handles more easily this situation, and can be built inside the CoAP "messages"- layer depicted in Figure 1 of [I-D.ietf-core-coap]. In the following part of this section, an alternative method for transferring segmented messages is described. Endpoints willing to support multi-datagram transfers MAY support the following variant. The reserved field R shown in Figure 2 can be redefined as follows: More datagrams pending (M): 1-bit boolean value. Indicates that the current message is formed by other datagrams too, the endpoint receiving this message MUST wait to receive all the datagrams before handling it. If the message does not fit into a single datagram, the M bit MUST be set. After receiving an ACK message confirming the reception of each data segment, the subsequent part fitting in a datagram should be sent. If the sender times out waiting for an ACK, it retransmits the same data segment. Multi-datagram messages SHOULD be sent only using CON messages in order to perform congestion control of any kind. The Seq field present in the variant shown in Figure 3 is suitable for multi-datagram tranfers too. Seq field MUST be increased at every subsequent data segment sent, ACK messages MUST have Seq value matching the Seq of the acknowledged message. The presence of this field is useful to avoid confusing a new ACK with a network duplicated one. When a message is received containing the field M set to 0, the last content segment is attached to the message, which is then passed to the CoAP "request/response"-layer and handled. Castellani Expires August 25, 2011 [Page 10] Internet-Draft CoAP overhead February 2011 Thanks to the half-duplex nature of a REST conversation, multi- datagram requests and responses can be tranferred using the same method: after sending all the request datagrams, the server handles the request and can send all the datagrams forming the response using this technique. The same Seq value is initially chosen and increased by the client to send the request, and is then increased by the server when sending back the response. Action, Action Code and Options related to the message SHOULD be sent only in the first datagram. To reduce overhead, the Options SHOULD be removed from subsequent datagrams; to avoid confusion and redundancy, Action and Action Code fields MAY be set to zero, the presence of content in the message allows to distinguish subsequent segments from an empty message. 3.2.2.1. Link-MTU Option As noted in [I-D.ietf-core-block], the fragmentation/reassembly process loads the lower layers. To avoid this problem multi-datagram transfer SHOULD transfer data segments fitting in a single link layer frame. +------+-----+----------+-----------+--------+---------+ | Type | C/E | Name | Data type | Length | Default | +------+-----+----------+-----------+--------+---------+ | TBD | E | Link-MTU | uint | 1-2 B | TBD | +------+-----+----------+-----------+--------+---------+ To address situations where the sending endpoint does not know the frame limits of the receiver, its size can be proactively shared attaching a Link-MTU option to requests expecting big responses. To react even more promptly, this option can be piggybacked over an empty ACK aknowledging a fragmented message. 3.2.2.2. Example Figure 4 shows an interaction between a CoAP Client and Server, where both are transferring content using multi-datagram transfers. Right after the client has completed the multi-datagram request, the server starts replying with a multi-datagram response. Castellani Expires August 25, 2011 [Page 11] Internet-Draft CoAP overhead February 2011 Client Server | | | CON [M=1,Seq=100], POST, /configuration ----------->| | | |<--------------------------------- ACK [M=0,Seq=100] | | | | CON [M=1,Seq=101] --------------------------------->| | | |<--------------------------------- ACK [M=0,Seq=101] | | | | CON [M=0,Seq=102] --------------------------------->| | | |<----------- (not useful to send?) ACK [M=0,Seq=102] | | | |<------------------------ CON [M=1,Seq=103], 2.00 OK | | | | ACK [M=0,Seq=103] --------------------------------->| | | |<--------------------------------- CON [M=1,Seq=104] | | | | ACK [M=0,Seq=104] --------------------------------->| | | |<--------------------------------- CON [M=0,Seq=105] | | | | ACK [M=0,Seq=105] --------------------------------->| | | Figure 4: Multi-datagram messaging example As an interesting side note in the message diagram depicted above, the ACK [M=0,Seq=102] can be suppressed, because the server promptly sends a CON [M=1,Seq=103] message implictly acknowledging CON [M=0,Seq=102] message. 3.3. Overhead analysis with new base header Table 4 and Table 5 analyze the request/response messages bytes allocation using the new base header proposed. Castellani Expires August 25, 2011 [Page 12] Internet-Draft CoAP overhead February 2011 +-----------------+--------------+-----------------+----------------+ | Field(s) | Req2 (Ratio) | ShortReq2 | SeqReq2 | | | | (Ratio) | (Ratio) | +-----------------+--------------+-----------------+----------------+ | Base | 2B (0.25) | 2B (0.50) | 2B (0.40) | | MID/Token or | 4B (0.50) | 0B (0.00) | 1B (0.20) | | Seq | | | | | URI | 2B (0.25) | 2B (0.50) | 2B (0.40) | | Total | 8B (1.00) | 4B (1.00) | 5B (1.00) | +-----------------+--------------+-----------------+----------------+ Table 4: Request size comparison +-----------------+--------------+-----------------+----------------+ | Field(s) | Res2 (Ratio) | ShortRes2 | SeqRes2 | | | | (Ratio) | (Ratio) | +-----------------+--------------+-----------------+----------------+ | Base | 2B (0.25) | 2B (0.50) | 2B (0.40) | | MID/Token or | 4B (0.50) | 0B (0.00) | 1B (0.20) | | Seq | | | | | Content | 2B (0.25) | 2B (0.50) | 2B (0.40) | | Total | 8B (1.00) | 4B (1.00) | 5B (1.00) | +-----------------+--------------+-----------------+----------------+ Table 5: Response size comparison The above tables compare the current CoAP format (Req2/Res2) with the smaller base header format defined in Section 3.2 (SeqReq2,SeqRes2). As shown in the proposed results, the number of bytes required for session matching and transport issues is lowered from 4B (Req2/Res2) to 1B (SeqReq2/SeqRes2), leading to a message reduction of more than 1/3 in this specific example. The use of the format shown in Figure 2 (ShortReq2/ShortRes2) halves the message size when compared to current CoAP (Req2/Res2), however at the cost of some possible confusion due to the easy accidental duplication; before selecting this solution, the WG is encouraged to discuss the relevance of accidental/intentional duplication in CoRE. 4. URI Further discussion about URI encoding using two optional compressed URI options will follow in a planned revision of the document. Refer to the following message to get an idea of the encodings that will be discussed: http://www.ietf.org/mail-archive/web/core/current/msg00315.html Castellani Expires August 25, 2011 [Page 13] Internet-Draft CoAP overhead February 2011 Discussion about the proposed encodings was partly done on the ML, however a more complete proposal may be useful to better evaluate the subject. 5. Acknowledgements Special thanks to Nicola Bui and Michele Zorzi for their support and contributions. 6. References 6.1. Normative References [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [I-D.ietf-core-coap] Shelby, Z., Hartke, K., Bormann, C., and B. Frank, "Constrained Application Protocol (CoAP)", draft-ietf-core-coap-04 (work in progress), January 2011. [I-D.ietf-httpbis-p2-semantics] Fielding, R., Gettys, J., Mogul, J., Nielsen, H., Masinter, L., Leach, P., Berners-Lee, T., and J. Reschke, "HTTP/1.1, part 2: Message Semantics", draft-ietf-httpbis-p2-semantics-12 (work in progress), October 2010. [I-D.ietf-core-block] Shelby, Z. and C. Bormann, "Blockwise transfers in CoAP", draft-ietf-core-block-01 (work in progress), January 2011. [I-D.ietf-core-observe] Hartke, K. and Z. Shelby, "Observing Resources in CoAP", draft-ietf-core-observe-01 (work in progress), February 2011. 6.2. Informative References [I-D.castellani-core-http-coap-mapping] Castellani, A. and S. Loreto, "Best Practice to map HTTP to COAP and viceversa", draft-castellani-core-http-coap-mapping-00 (work in progress), January 2011. Castellani Expires August 25, 2011 [Page 14] Internet-Draft CoAP overhead February 2011 [I-D.eggert-core-congestion-control] Eggert, L., "Congestion Control for the Constrained Application Protocol (CoAP)", draft-eggert-core-congestion-control-01 (work in progress), January 2011. Editorial Comments [Q1] AC: 8-bit sequence number seems suitable to me. Is it sufficient or is a 16-bit unsigned integer required? Author's Address Angelo P. Castellani University of Padova Via Gradenigo 6/B Padova 35131 Italy Email: angelo@castellani.net Castellani Expires August 25, 2011 [Page 15]