Constrained Application
Protocol (CoAP) Block-Wise Transfer Options for Faster
TransmissionOrangeRennes35000Francemohamed.boucadair@orange.comUnited Kingdomsupjps-ietf@jpshallow.comCOREsecuritymitigationservice deliveryconnectivityanti-DDoSautomationcooperationResilienceFilteringSecurity CenterMitigatorScrubbingdynamic service protectiondynamic mitigationThis document specifies alternate Constrained Application Protocol
(CoAP) Block-Wise transfer options: Quick-Block1 and Quick-Block2
Options.These options are similar to the CoAP Block1 and Block2 Options, not
a replacement for them, but do enable faster transmission rates for
large amounts of data with less packet interchanges as well as
supporting faster recovery should any of the blocks get lost in
transmission.The Constrained Application Protocol (CoAP) , although inspired by HTTP, was designed to
use UDP instead of TCP. The message layer of CoAP over UDP includes
support for reliable delivery, simple congestion control, and flow
control. introduced the CoAP Block1 and
Block2 Options to handle data records that cannot fit in a single IP
packet, so not having to rely on IP fragmentation and further updated
by for use over TCP, TLS, and
Websockets.The CoAP Block1 and Block2 Options work well in environments where
there are no or minimal packet losses. These options operate
synchronously where each block has to be requested and can only ask
for (or send) the next block when the request for the previous block
has completed. Packet, and hence block transmission rate, is
controlled by Round Trip Times (RTTs).There is a requirement for these blocks of data to be transmitted
at higher rates under network conditions where there may be
asymmetrical transient packet loss. An example is when a network is
subject to a Distributed Denial of Service (DDoS) attack and there is
a need for DDoS mitigation agents relying upon CoAP to communicate
with each other (e.g., ). As a reminder, recommends use of Confirmable (CON) responses
to handle potential packet loss; which does not work with a flooded
pipe DDoS situation.This document introduces the CoAP Quick-Block1 and Quick-Block2
Options. These options are similar in operation to the CoAP Block1 and
Block2 Options respectively, they are not a replacement for them, but
have the following benefits:They can operate in environments where packet loss is highly
asymmetrical.They enable faster transmissions of sets of blocks of data with
less packet interchanges.They support faster recovery should any of the Blocks get lost
in transmission.There are the following disadvantages over using CoAP Block 1 and
Block2 Options:Loss of lock-stepping so payloads are not always received in
the correct (block ascending) order.Additional congestion control measures need to be put in
place.Using Non-confirmable (NON) messages, the faster transmissions
occur as all the Blocks can be transmitted serially (as are IP
fragmented packets) without having to wait for an acknowledgement or
next request from the remote CoAP peer. Recovery of missing Blocks is
faster in that multiple missing Blocks can be requested in a single
CoAP packet. Even if there is asymmetrical packet loss, a body can
still be sent and received by the peer whether the body compromises of
a single or multiple payloads assuming no recovery is required.Note that the same performance benefits can be applied to
Confirmable messages if the value of NSTART is increased from 1
(Section 4.7 of ). However, the
asymmetrical packet loss is not a benefit here. Some sample examples
with Confirmable messages are provided in .There is little, if any, benefit of using these options with CoAP
running over a reliable connection . In
this case, there is no differentiation between Confirmable and NON as
they are not used.A CoAP endpoint can acknowledge all or a subset of the blocks.
Concretely, the receiving CoAP endpoint informs the CoAP endpoint
sender either successful receipt or reports on all blocks in the body
that have been not yet been received. The CoAP endpoint sender will
then retransmit only the blocks that have been lost in
transmission.Quick-Block1 and Quick-Block2 Options can be used instead of Block1
and Block2 Options respectively when the different transmission
semantics are required. If the option is not supported by a peer, then
transmissions can fall back to using Block1 and Block2
respectively.The deviations from Block1 and Block2 Options are specified in
. Pointers to appropriate sections are provided.The specification refers to the base CoAP methods defined in
Section 5.8 of and the new CoAP
methods, FETCH, PATCH, and iPATCH introduced in .This document updates the 4.08 (Request Entity Incomplete) by
defining an additional message format for reporting on payloads using
the Quick-Block1 Option that are not received by the server.See for more details.The block-wise transfer specified in
covers the general case, but falls short in situations where packet
loss is highly asymmetrical. The mechanism specified in the document
provides roughly similar features to the Block1/Block2 Options. It
provides additional properties that are tailored towards the intended
use case. Concretely, this mechanism primarily targets applications
such as DDoS Open Threat Signaling (DOTS) that can't use Confirmable
(CON) responses to handle potential packet loss and that support
application-specific mechanisms to assess whether the remote peer is
able to handle the messages sent by a CoAP endpoint (e.g., DOTS
heartbeats in Section 4.7 of ).The mechanism includes guards to prevent a CoAP agent from
overloading the network by adopting an aggressive sending rate. These
guards MUST be followed in addition to the existing CoAP congestion
control as specified in Section 4.7 of .
See for more details.This mechanism is not intended for general CoAP usage, and any use
outside the intended use case should be carefully weighed against the
loss of interoperability with generic CoAP applications. It is hoped
that the experience gained with this mechanism can feed future
extensions of the block-wise mechanism that will both generally
applicable and serve this particular use case.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP 14
when, and
only when, they appear in all capitals, as shown here.Readers should be familiar with the terms and concepts defined in
.The terms "payload" and "body" are defined in . The term "payload" is thus used for the
content of a single CoAP message (i.e., a single block being
transferred), while the term "body" is used for the entire resource
representation that is being transferred in a block-wise fashion.The properties of Quick-Block1 and Quick-Block2 Options are shown
in Table 1. The formatting of this table follows the one used in Table
4 of (Section 5.10). The C, U, N, and R
columns indicate the properties Critical, Unsafe, NoCacheKey, and
Repeatable defined in Section 5.4 of .
Only C and U columns are marked for the Quick-Block1 Option. C, U, and
R columns are marked for the Quick-Block2 Option.The Quick-Block1 and Quick-Block2 Options can be present in both
the request and response messages. The Quick-Block1 Option pertains to
the request payload and the Quick-Block2 Option pertains to the
response payload. The Content-Format Option applies to the body, not
to the payload (i.e., it must be the same for all payloads of the same
body).Quick-Block1 Option is useful with the payload-bearing POST, PUT,
PATCH, and iPATCH requests and their responses (2.01 and 2.04).Quick-Block2 Option is useful with GET, POST, PUT, FETCH, PATCH,
and iPATCH requests and their payload-bearing responses (2.01, 2.03,
2.04, and 2.05) (Section 5.5 of ).To indicate support for Quick-Block2 responses, the CoAP client
MUST include the Quick-Block2 Option in a GET or similar request, or
the Quick-Block2 Option in a PUT or similar request, so that the
server knows that the client supports this Quick-Block2 functionality
should it needs to send back a body that spans multiple payloads.
Otherwise, the server would use the Block2 Option (if supported) to
send back a message body that is too large to fit into a single IP
packet .If Quick-Block1 Option is present in a request or Quick-Block2
Option in a response (i.e., in that message to the payload of which it
pertains), it indicates a block-wise transfer and describes how this
specific block-wise payload forms part of the entire body being
transferred. If it is present in the opposite direction, it provides
additional control on how that payload will be formed or was
processed.Implementation of the Quick-Block1 and Quick-Block2 Options is
intended to be optional. However, when it is present in a CoAP
message, it MUST be processed (or the message rejected). Therefore,
Quick-Block1 and Quick-Block2 Options are identified as Critical
options.The Quick-Block1 and Quick-Block2 Options are unsafe to forward.
That is, a CoAP proxy that does not understand the Quick-Block1 (or
Quick-Block2) Option MUST reject the request or response that uses
either option.The Quick-Block2 Option is repeatable when requesting
re-transmission of missing Blocks, but not otherwise. Except that
case, any request carrying multiple Quick-Block1 (or Quick-Block2)
Options MUST be handled following the procedure specified in Section
5.4.5 of .The Quick-Block1 and Quick-Block2 Options, like the Block1 and
Block2 Options, are both a class E and a class U in terms of OSCORE
processing (see Section 4.1 of ): The
Quick-Block1 (or Quick-Block2) Option MAY be an Inner or Outer option.
The Inner and Outer values are therefore independent of each other.
The Inner option is encrypted and integrity protected between clients
and servers, and provides message body identification in case of
end-to-end fragmentation of requests. The Outer option is visible to
proxies and labels message bodies in case of hop-by-hop fragmentation
of requests.The structure of Quick-Block1 and Quick-Block2 Options follows the
structure defined in Section 2.2 of .There is no default value for the Quick-Block1 and Quick-Block2
Options. Absence of one of these options is equivalent to an option
value of 0 with respect to the value of block number (NUM) and more
bit (M) that could be given in the option, i.e., it indicates that the
current block is the first and only block of the transfer (block
number is set to 0, M is unset). However, in contrast to the explicit
value 0, which would indicate a size of the block (SZX) of 0, and thus
a size value of 16 bytes, there is no specific explicit size implied
by the absence of the option -- the size is left unspecified. (As for
any uint, the explicit value 0 is efficiently indicated by a
zero-length option; this, therefore, is different in semantics from
the absence of the option).The Quick-Block1 Option is used when the client wants to send a
large amount of data to the server using the POST, PUT, PATCH, or
iPATCH methods where the data and headers do not fit into a single
packet.When Quick-Block1 Option is used, the client MUST include a single
Request-Tag Option . The Request-Tag value
MUST be the same for all of the blocks in the body of data that is
being transferred. It is also used to identify a particular block of a
body that needs to be re-transmitted. The Request-Tag is opaque in
nature, but it is RECOMMENDED that the client treats it as an unsigned
integer of 8 bytes in length. An implementation may want to consider
limiting this to 4 bytes to reduce packet overhead size. The server
still treats it as an opaque entity. The Request-Tag value MUST be
different for distinct bodies or sets of blocks of data and SHOULD be
incremented whenever a new body of data is being transmitted for a
CoAP session between peers. The initial Request-Tag value SHOULD be
randomly generated by the client.For Confirmable transmission, the server MUST continue to
acknowledge each packet. NSTART will also need to be increased from
the default (1) to get faster transmission rates.Each individual payload of the body is treated as a new
request.A 2.01 (Created) or 2.04 (Changed) Response Code indicates
successful receipt of the entire body. The 2.31 (Continue) Response
Code MUST NOT be used.The 2.31 (Continue) Response is not used.A 4.00 (Bad Request) Response Code MUST be returned if the request
does not include a Request-Tag Option but does include a Quick-Block1
option.A 4.02 (Bad Option) Response Code MUST be returned if the server
does not support the Quick-Block1 Option.A 4.13 (Request Entity Too Large) Response Code can be returned
under similar conditions to those discussed in Section 2.9.3 of .A 4.08 (Request Entity Incomplete) Response Code returned without
Content-Type "application/missing-blocks+cbor-seq" () is handled as in Section 2.9.2 .A 4.08 (Request Entity Incomplete) Response Code returned with
Content-Type "application/missing-blocks+cbor-seq" indicates that some
of the payloads are missing and need to be resent. The client then
re-transmits the missing payloads using the Request-Tag and
Quick-Block1 to specify the block number, SZX, and M bit as
appropriate. The Request-Tag value to use is determined from the
payload of the 4.08 (Request Entity Incomplete) Response Code. If the
client does not recognize the Request-Tag, the client can ignore this
response.If the server has not received all the payloads of a body, but one
or more payloads have been received, it SHOULD wait for up to
MAX_TRANSMIT_SPAN (Section 4.8.2 of )
before sending the 4.08 (Request Entity Incomplete) Response Code.
However, this time MAY be reduced to two times ACK_TIMEOUT before
sending a 4.08 (Request Entity Incomplete) Response Code to cover the
situation where MAX_PAYLOADS has been triggered by the client causing
a break in transmission.If the client transmits a new body of data with a new Request-Tag
to the same resource on a server, the server MUST remove any partially
received body held for a previous Request-Tag for that resource.If the server receives a duplicate block with the same Request-Tag,
it SHOULD silently ignore the packet.A server SHOULD only maintain a partial body (missing payloads) for
up to EXCHANGE_LIFETIME (Section 4.8.2 of ).In a request, for block number 0, the M bit unset indicates the
entire body is requested. If the M bit is set for block number 0, this
indicates that this is a repeat request. Otherwise for a request, the
Quick-Block2 Option MUST always have the M bit unset.The payloads sent back from the server as a response MUST all have
the same ETag (Section 5.10.6 of ) for
the same body. The server MUST NOT use the same ETag value for
different representations of a resource.The ETag is opaque in nature, but it is RECOMMENDED that the server
treats it as an unsigned integer of 8 bytes in length. An
implementation may want to consider limiting this to 4 bytes to reduce
packet overhead size. The client still treats it as an opaque entity.
The ETag value MUST be different for distinct bodies or sets of blocks
of data and SHOULD be incremented whenever a new body of data is being
transmitted for a CoAP session between peers. The initial ETag value
SHOULD be randomly generated by the server.If the client detects that some of the payloads are missing, the
missing payloads are requested by issuing a new GET, POST, PUT, FETCH,
PATCH, or iPATCH request that contains one or more Quick-Block2
Options that define the missing blocks.The ETag Option MUST NOT be used in the request as the server could
respond with a 2.03 (Valid Response) with no payload. If the server
responds with a different ETag Option value (as the resource
representation has changed), then the client SHOULD drop all the
payloads for the current body that are no longer valid.The client may elect to request the missing blocks or just ignore
the partial body. It SHOULD wait for up to MAX_TRANSMIT_SPAN (Section
4.8.2 of ) before issuing a GET, POST,
PUT, FETCH, PATCH, or iPATCH request for the missing blocks. However,
this time MAY be reduced to two times ACK_TIMEOUT before sending the
request to cover the situation where MAX_PAYLOADS has been triggered
by the server causing a break in transmission.With NON transmission, the client only needs to indicate that some
of the payloads are missing by issuing a GET, POST, PUT, FETCH, PATCH,
or iPATCH request for the missing blocks.For Confirmable transmission, the client SHOULD continue to
acknowledge each packet as well as issuing a separate GET, POST, PUT,
FETCH, PATCH, or iPATCH for the missing blocks.If the server transmits a new body of data (e.g., a triggered
Observe) with a new ETag to the same client as an additional response,
the client MUST remove any partially received body held for a previous
ETag.If the client receives a duplicate block with the same ETag, it
SHOULD silently ignore the packet.A client SHOULD only maintain a partial body (missing payloads) for
up to EXCHANGE_LIFETIME (Section 4.8.2 of ) or as defined by the Max-Age Option
whichever is the less.As the blocks of the body are sent without waiting for
acknowledgement of the individual blocks, the Observe value MUST be the same for all the blocks of the
same body.If the client requests missing blocks, this is treated as a new
request. The Observe value may change but MUST still be reported. If
the ETag value changes then the previously received partial body
should be destroyed and the whole body re-requested.Section 4 of defines two CoAP
options: Size1 for indicating the size of the representation
transferred in requests and Size2 for indicating the size of the
representation transferred in responses.The Size1 or Size2 option values MUST exactly represent the size of
the data on the body so that any missing data can easily be
determined.The Size1 Option MUST be used with the Quick-Block1 Option when
used in a request. The Size2 Option MUST be used with the Quick-Block2
Option when used in a response.If Size1 or Size2 Options are used, they MUST be used in all
payloads of the body and MUST have the same value.The behavior is similar to the one defined in Section 3.3 of with Quick-Block1 substituted for Block1 and
Quick-Block2 for Block2.4.08 (Request Entity Incomplete) Response Code has a new Content-Type
"application/missing-blocks+cbor-seq" used to indicate that the server
has not received all of the blocks of the request body that it needs to
proceed.Likely causes are the client has not sent all blocks, some blocks
were dropped during transmission, or the client has sent them
sufficiently long ago that the server has already discarded them.The data payload of the 4.08 (Request Entity Incomplete) Response
Code is encoded as a CBOR Sequence . First
is CBOR encoded Request-Tag followed by 1 or more missing CBOR encoded
missing block numbers. The missing block numbers MUST be unique in each
4.08 (Request Entity Incomplete) when created by the server; the client
SHOULD drop any duplicates in the same 4.08 (Request Entity Incomplete)
message.The Content-Format Option (Section 5.10.3 of ) MUST be used in the 4.08 (Request Entity
Incomplete) Response Code. It MUST be set to
"application/missing-blocks+cbor-seq" (see ).The Concise Data Definition Language
for the data describing these missing blocks is as follows:If the size of the 4.08 (Request Entity Incomplete) response packet
is larger than that defined by Section 4.6 , then the number of missing blocks MUST be
limited so that the response can fit into a single packet. If this is
the case, then the server can send subsequent 4.08 (Request Entity
Incomplete) responses containing the missing blocks on receipt of a new
request providing a missing payload with the same Request-Tag.Each new request MUST use a unique Token (Section 4 of ). Additional responses
may use the same Token.PROBING_RATE parameter in CoAP indicates the average data rate that
must not be exceeded by a CoAP endpoint in sending to a peer endpoint
that does not respond. The body of blocks will be subjected to
PROBING_RATE (Section 4.7 of ).Each NON 4.08 (Request Entity Incomplete) Response Codes is subjected
to PROBING_RATE.Each NON GET or similar request using Quick-Block2 Option is
subjected to PROBING_RATE.As the sending of many payloads of a single body may itself cause
congestion, it is RECOMMENDED that after transmission of every set of
MAX_PAYLOADS payloads of a single body, a delay is introduced of
ACK_TIMEOUT (Section 4.8.2 of ) before the
next set of payload transmissions to manage potential congestion issues.
MAX_PAYLOADS should be configurable with a default value of 10. Note: The default value is chosen for reasons similar to those
discussed in Section 5 of .For NON transmissions, it is permissible, but not required, to send
the ultimate payload of a MAX_PAYLOADS set as a Confirmable packet. If a
Confirmable packet is used, then the transmitting peer MUST wait for the
ACK to be returned before sending the next set of payloads, which can be
in time terms less than the ACK_TIMEOUT delay.Also, for NON transmissions, it is permissible, but not required, to
send a Confirmable packet for the final payload of a body (that is, M
bit unset). If a Confirmable packet is used, then the transmitting peer
MUST wait for the appropriate response to be returned for successful
transmission, or respond to requests for the missing blocks (if
any).The sending of the set of missing blocks is subject to
MAX_PAYLOADS.Caching block based information is not straight forward in a proxy.
For Quick-Block1 and Quick-Block2 Options, it is expected that the proxy
will reassemble the body (using any appropriate recovery options for
packet loss) before passing on the body to the appropriate CoAP
endpoint. The onward transmission of the body does not require the use
of the Quick-Block1 or Quick-Block2 Options. This means that the proxy
must fully support the Quick-Block1 and Quick-Block2 Options.How the body is cached in the initial CoAP client (Quick-Block1) or
ultimate CoAP server (Quick-Block2) is implementation specific.As the entire body is being cached in the proxy, the Quick-Block1 and
Quick-Block2 Options are not part of the cache key.For Quick-Block2 responses, the ETag Option value is associated with
the data (and onward transmitted to the CoAP client), but is not part of
the cache key.For requests with Quick-Block1 Option, the Request-Tag Option is
associated with the build up of the body from successive payloads, but
is not part of the cache key. For the onward transmission of the body
using CoAP, a new Request-Tag SHOULD be generated and used.It is possible that two or more CoAP clients are concurrently
updating the same resource through a common proxy to the same CoAP
server using Quick-Block1 (or Block1) Option. If this is the case, the
first client to complete building the body causes that body to start
transmitting to the CoAP server with an appropriate Request-Tag value.
When the next client completes building the body, any existing partial
body transmission to the CoAP server is terminated and the new body
representation transmission starts with a new Request-Tag value.A proxy that supports Quick-Block2 Option MUST be prepared to receive
a GET or similar message indicating one or more missing blocks. The
proxy will serve from its cache the missing blocks that are available in
its cache in the same way a server would send all the appropriate
Quick-Block2s. If the cache key matching body is not available in the
cache, the proxy MUST request the entire body from the CoAP server using
the information in the cache key.How long a CoAP endpoint (or proxy) keeps the body in its cache is
implementation specific (e.g., it may be based on Max-Age).As a reminder, the basic normative requirements on HTTP/CoAP mappings
are defined in Section 10 of . The
implementation guidelines for HTTP/CoAP mappings are elaborated in .The rules defined in Section 5 of are
to be followed.This section provides some sample flows to illustrate the use of
Quick-Block1 and Quick-Block2 Options.
lists the conventions that are used in the following subsections. depicts an example of a NON PUT
request conveying Quick-Block1 Option. All the blocks are received by
the server.Consider now a scenario where a new body of data is to be sent by
the client, but some blocks are dropped in transmission as illustrated
in .The server realizes that some blocks are missing and asks for the
missing ones in one go (). It does so by
indicating which blocks have been received in the data portion of the
response.Under high levels of traffic loss, the client can elect not to
retry sending missing blocks of data. This decision is implementation
specific. illustrates the example of
Quick-Block2 Option. The client sends a NON GET carrying an Observe
and a Quick-Block2 Options. The Quick-Block2 Option indicates a size
hint (1024 bytes). This request is replied by the server using four
(4) blocks that are transmitted to the client without any loss. Each
of these blocks carries a Quick-Block2 Option. The same process is
repeated when an Observe is triggered, but no loss is experienced by
any of the notification blocks. shows the example of an Observe that
is triggered but for which some notification blocks are lost. The
client detects the missing blocks and request their retransmission. It
does so by indicating the blocks that were successfully received.Under high levels of traffic loss, the client can elect not to
retry getting missing blocks of data. This decision is implementation
specific.IANA is requested to add the following entries to the "CoAP Option
Numbers" sub-registry :This document suggests 19 (TBA1) and 51 (TBA2) as a values to be
assigned for the new option numbers.This document requests IANA to register the CoAP Content-Format ID
for the "application/missing-blocks+cbor-seq" media type in the "CoAP
Content-Formats" registry :Security considerations discussed in Section 9 of should be taken into account.Security considerations related to the use of Request-Tag are
discussed in Section 5 of .Thanks to Achim Kraus and Jim Schaad for the comments on the mailing
list.Special thanks to Christian Amsüss and Carsten Bormann for their
suggestions and several reviews, which improved this specification
significantly.Some text from is reused for readers
convenience.These examples assume NSTART has been increased to at least 4.The notations provided in are used in
the following subsections.Let's now consider the use Quick-Block1 Option with a CON request
as shown in . All the blocks are
acknowledged (ACK).Now, suppose that a new body of data is to sent but with some
blocks dropped in transmission as illustrated in . The client will retry sending blocks for which
no ACK was received.It is implementation dependent as to whether a CoAP session
is terminated following acknowledge retry timeout, or whether the CoAP
session continues to be used under such adverse traffic
conditions.If there is likely to be the possibility of network transient
losses, then the use of Non-confirmable traffic should be
considered.An example of the use of Quick-Block2 Option with Confirmable
messages is shown in .It is implementation-dependent as to whether a CoAP session is
terminated following acknowledge retry timeout, or whether the CoAP
session continues to be used under such adverse traffic
conditions.If there is likely to be the possibility of network transient
losses, then the use of Non-confirmable traffic should be
considered.