Constrained Application
Protocol (CoAP) Block-Wise Transfer Options for Faster
TransmissionOrangeRennes35000Francemohamed.boucadair@orange.comUnited Kingdomsupjps-ietf@jpshallow.comCoRE Working GroupQuick-BlockRobust-BlockR-BlockTough-BlockResilient-BlockFast-BlockResilienceFilteringFaster transmissionLarge amounts of dataLess packet interchangeFast recoveryDOTSThis document specifies alternative Constrained Application Protocol
(CoAP) Block-Wise transfer options: Q-Block1 and Q-Block2 Options.These options are similar to the CoAP Block1 and Block2 Options
defined in RFC 7959, not a replacement for them, but do enable faster
transmission rates for large amounts of data with less packet
interchanges. Also, Q-Block1 and Q-Block2 Options support faster
recovery should any of the blocks get lost in transmission.The Constrained Application Protocol (CoAP) , although inspired by HTTP, was designed to use
UDP instead of TCP. The message layer of CoAP over UDP includes support
for reliable delivery, simple congestion control, and flow control.
introduced the CoAP Block1 and Block2
Options to handle data records that cannot fit in a single IP packet, so
not having to rely on IP fragmentation and was further updated by for use over TCP, TLS, and WebSockets.The CoAP Block1 and Block2 Options work well in environments where
there are no or minimal packet losses. These options operate
synchronously, i.e., each individual block has to be requested. A CoAP
endpoint can only ask for (or send) the next block when the transfer of
the previous block has completed. Packet transmission rate, and hence
block transmission rate, is controlled by Round Trip Times (RTTs).There is a requirement for these blocks of data to be transmitted at
higher rates under network conditions where there may be asymmetrical
transient packet loss (i.e., responses may get dropped). An example is
when a network is subject to a Distributed Denial of Service (DDoS)
attack and there is a need for DDoS mitigation agents relying upon CoAP
to communicate with each other (e.g., ).
As a reminder, recommends the use of
Confirmable (CON) responses to handle potential packet loss. However,
such a recommendation does not work with a flooded pipe DDoS
situation.This document introduces the CoAP Q-Block1 and Q-Block2 Options
().The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP 14
when, and
only when, they appear in all capitals, as shown here.Readers should be familiar with the terms and concepts defined in
, , and
.The terms "payload" and "body" are defined in . The term "payload" is thus used for the
content of a single CoAP message (i.e., a single block being
transferred), while the term "body" is used for the entire resource
representation that is being transferred in a block-wise fashion.This document introduces the CoAP Q-Block1 and Q-Block2 Options.
These options are similar in operation to the CoAP Block1 and Block2
Options, respectively. They are not a replacement for them, but have the
following benefits:They can operate in environments where packet loss is highly
asymmetrical.They enable faster transmissions of sets of blocks of data with
less packet interchanges.They support faster recovery should any of the blocks get lost in
transmission.They support sending an entire body using Non-confirmable (NON)
messages without requiring a response from the peer.There are the following disadvantages over using CoAP Block1 and
Block2 Options:Loss of lock-stepping so payloads are not always received in the
correct (block ascending) order.Additional congestion control measures need to be put in place
for NON messages ().To reduce the transmission times for CON transmission of large
bodies, NSTART needs to be increased from 1, but this affects
congestion control where other parameters need to be tuned (Section
4.7 of ). Such tuning is out of scope
of this document.Mixing of NON and CON during requests/responses using Q-Block is
not supported.The Q-Block Options do not support stateless operation/random
access.Proxying of Q-Block is limited to caching full
representations.There is no multicast support.Q-Block1 and Q-Block2 Options are designed to work in particular with
NON requests and responses.Using NON messages, the faster transmissions occur as all the blocks
can be transmitted serially (as are IP fragmented packets) without
having to wait for a response or next request from the remote CoAP peer.
Recovery of missing blocks is faster in that multiple missing blocks can
be requested in a single CoAP message. Even if there is asymmetrical
packet loss, a body can still be sent and received by the peer whether
the body comprises of a single or multiple payloads assuming no recovery
is required.A CoAP endpoint can acknowledge all or a subset of the blocks.
Concretely, the receiving CoAP endpoint either informs the CoAP sender
endpoint of successful reception or reports on all blocks in the body
that have not yet been received. The CoAP sender endpoint will then
retransmit only the blocks that have been lost in transmission.Note that similar performance benefits can be applied to Confirmable
messages if the value of NSTART is increased from 1 (Section 4.7 of
). However, the use of Confirmable
messages will not work if there is asymmetrical packet loss. Some
examples with Confirmable messages are provided in .There is little, if any, benefit of using these options with CoAP
running over a reliable connection . In
this case, there is no differentiation between CON and NON as they are
not used. Some examples using a reliable transport are provided in .Q-Block1 and Q-Block2 Options can be used instead of Block1 and
Block2 Options when the different transmission properties are required.
If the new options are not supported by a peer, then transmissions can
fall back to using Block1 and Block2 Options ().The deviations from Block1 and Block2 Options are specified in . Pointers to appropriate sections are provided.The specification refers to the base CoAP methods defined in Section
5.8 of and the new CoAP methods, FETCH,
PATCH, and iPATCH introduced in .The No-Response Option was considered
but was abandoned as it does not apply to Q-Block2 responses. A unified
solution is defined in the document.This document adds a media type for the 4.08 (Request Entity
Incomplete) response defining an additional message format for
reporting on payloads using the Q-Block1 Option that are not received
by the server.See for more details.The block-wise transfer specified in
covers the general case, but falls short in situations where packet
loss is highly asymmetrical. The mechanism specified in this document
provides roughly similar features to the Block1/Block2 Options. It
provides additional properties that are tailored towards the intended
use case of Non-Confirmable transmission. Concretely, this mechanism
primarily targets applications such as DDoS Open Threat Signaling
(DOTS) that cannot use CON responses to handle potential packet loss
and that support application-specific mechanisms to assess whether the
remote peer is able to handle the messages sent by a CoAP endpoint
(e.g., DOTS heartbeats in Section 4.7 of ).The mechanism includes guards to prevent a CoAP agent from
overloading the network by adopting an aggressive sending rate. These
guards MUST be followed in addition to the existing CoAP congestion
control as specified in Section 4.7 of .
See for more details.This mechanism is not intended for general CoAP usage, and any use
outside the intended use case should be carefully weighed against the
loss of interoperability with generic CoAP applications. It is hoped
that the experience gained with this mechanism can feed future
extensions of the block-wise mechanism that will both be generally
applicable and serve this particular use case.It is not recommended that these options are used in a NoSec
security mode (Section 9 of ) as the
source endpoint needs to be trusted. Using OSCORE does provide a security context and, hence, a
trust of the source endpoint. However, using a NoSec security mode may
still be inadequate for reasons discussed in .The properties of the Q-Block1 and Q-Block2 Options are shown in
Table 1. The formatting of this table follows the one used in Table 4
of (Section 5.10). The C, U, N, and R
columns indicate the properties Critical, UnSafe, NoCacheKey, and
Repeatable defined in Section 5.4 of .
Only Critical and UnSafe columns are marked for the Q-Block1 Option.
Critical, UnSafe, and Repeatable columns are marked for the Q-Block2
Option. As these options are UnSafe, NoCacheKey has no meaning and so
is marked with a dash.The Q-Block1 and Q-Block2 Options can be present in both the
request and response messages. The Q-Block1 Option pertains to the
request payload and the Q-Block2 Option pertains to the response
payload. When the Content-Format Option is present together with the
Q-Block1 or Q-Block2 Option, the option applies to the body not to the
payload (i.e., it must be the same for all payloads of the same
body).Q-Block1 Option is useful with the payload-bearing POST, PUT,
FETCH, PATCH, and iPATCH requests and their responses.Q-Block2 Option is useful with GET, POST, PUT, FETCH, PATCH, and
iPATCH requests and their payload-bearing responses (2.01, 2.02, 2.03,
2.04, and 2.05) (Section 5.5 of ).A CoAP endpoint (or proxy) MUST support either both or neither of
the Q-Block1 and Q-Block2 Options.If Q-Block1 Option is present in a request or Q-Block2 Option in a
response (i.e., in that message to the payload of which it pertains),
it indicates a block-wise transfer and describes how this specific
block-wise payload forms part of the entire body being transferred. If
it is present in the opposite direction, it provides additional
control on how that payload will be formed or was processed.To indicate support for Q-Block2 responses, the CoAP client MUST
include the Q-Block2 Option in a GET or similar request (FETCH, for
example), the Q-Block2 Option in a PUT or similar request, or the
Q-Block1 Option in a PUT or similar request so that the server knows
that the client supports this Q-Block2 functionality should it need to
send back a body that spans multiple payloads. Otherwise, the server
would use the Block2 Option (if supported) to send back a message body
that is too large to fit into a single IP packet . How a client decides whether it needs to include a Q-Block1 or
Q-Block2 Option can be driven by a local configuration knob, triggered
by an application (DOTS, for example), etc. Such considerations are
out of the scope of the document.Implementation of the Q-Block1 and Q-Block2 Options is intended to
be optional. However, when it is present in a CoAP message, it MUST be
processed (or the message rejected). Therefore, Q-Block1 and Q-Block2
Options are identified as Critical options.With CoAP over UDP, the way a request message is rejected for
critical options depends on the message type. A Confirmable message
with an unrecognized critical option is rejected with a 4.02 (Bad
Option) response (Section 5.4.1 of ). A
Non-confirmable message with an unrecognized critical option is either
rejected with a Reset message or just silently ignored (Sections 5.4.1
and 4.3 of ). To reliably get a
rejection message, it is therefore REQUIRED that clients use a
Confirmable message for determining support for Q-Block1 and Q-Block2
Options.The Q-Block1 and Q-Block2 Options are unsafe to forward. That is, a
CoAP proxy that does not understand the Q-Block1 (or Q-Block2) Option
MUST reject the request or response that uses either option.The Q-Block2 Option is repeatable when requesting retransmission of
missing blocks, but not otherwise. Except that case, any request
carrying multiple Q-Block1 (or Q-Block2) Options MUST be handled
following the procedure specified in Section 5.4.5 of .The Q-Block1 and Q-Block2 Options, like the Block1 and Block2
Options, both of class E and class U for OSCORE processing (Table 2).
The Q-Block1 (or Q-Block2) Option MAY be an Inner or Outer option
(Section 4.1 of ). The Inner and Outer
values are therefore independent of each other. The Inner option is
encrypted and integrity protected between clients and servers, and
provides message body identification in case of end-to-end
fragmentation of requests. The Outer option is visible to proxies and
labels message bodies in case of hop-by-hop fragmentation of
requests.Note that if Q-Block1 or Q-Block2 Options are included in a packet
as Inner options, Block1 or Block2 Options MUST NOT be included as
Inner options. Similarly there MUST NOT be a mix of Q-Block and Block
for the Outer options. Messages that do not adhere with this behavior
MUST be rejected with 4.02 (Bad Option). Q-Block and Block Options can
be mixed across Inner and Outer options as these are handled
independently of each other.The structure of Q-Block1 and Q-Block2 Options follows the
structure defined in Section 2.2 of .There is no default value for the Q-Block1 and Q-Block2 Options.
Absence of one of these options is equivalent to an option value of 0
with respect to the value of block number (NUM) and more bit (M) that
could be given in the option, i.e., it indicates that the current
block is the first and only block of the transfer (block number is set
to 0, M is unset). However, in contrast to the explicit value 0, which
would indicate a size of the block (SZX) of 0, and thus a size value
of 16 bytes, there is no specific explicit size implied by the absence
of the option -- the size is left unspecified. (As for any uint, the
explicit value 0 is efficiently indicated by a zero-length option;
this, therefore, is different in semantics from the absence of the
option).The Q-Block1 Option is used when the client wants to send a large
amount of data to the server using the POST, PUT, FETCH, PATCH, or
iPATCH methods where the data and headers do not fit into a single
packet.When Q-Block1 Option is used, the client MUST include a Request-Tag
Option . The
Request-Tag value MUST be the same for all of the requests for the
body of data that is being transferred. The Request-Tag is opaque, the
server still treats it as opaque but the client MUST ensure that it is
unique for every different body of transmitted data.Implementation Note: It is suggested that the client treats the
Request-Tag as an unsigned integer of 8 bytes in length. An
implementation may want to consider limiting this to 4 bytes to
reduce packet overhead size. The initial Request-Tag value should
be randomly generated and then subsequently incremented by the
client whenever a new body of data is being transmitted between
peers. discusses the use of Size1 Option.For Confirmable transmission, the server continues to acknowledge
each packet, but a response is not required (whether separate or
piggybacked) until successful receipt of the body by the server. For
Non-confirmable transmission, no response is required until the
successful receipt of the body by the server or some of the payloads
have not arrived after a timeout and a retransmit missing payloads
response is needed. For reliable transports (e.g., ), a response is not required until successful
receipt of the body by the server.Each individual message that carries a block of the body is treated
as a new request ().The client MUST send the payloads with the block numbers
increasing, starting from zero, until the body is complete (subject to
any congestion control ()). Any missing
payloads requested by the server must in addition be separately
transmitted with increasing block numbers.The following Response Codes are used:2.01 (Created)This Response Code indicates successful receipt of the entire
body and that the resource was created. The token used MUST be any
token that was received in a request using the same Request-Tag.
However, it is desirable to provide the one used in the last
received request, since that will aid any troubleshooting. The
client should then release all of the tokens used for this body.
Note that the last received payload may not be the one with the
highest block number.2.02 (Deleted)This Response Code indicates successful receipt of the entire
body and that the resource was deleted when using POST (Section
5.8.2 ). The token used MUST be any
token that was received in a request using the same Request-Tag.
However, it is desirable to provide the one used in the last
received request. The client should then release all of the tokens
used for this body.2.04 (Changed)This Response Code indicates successful receipt of the entire
body and that the resource was updated. The token used MUST be any
token that was received in a request using the same Request-Tag.
However, it is desirable to provide the one used in the last
received request. The client should then release all of the tokens
used for this body.2.05 (Content)This Response Code indicates successful receipt of the entire
FETCH request body (Section 2 of )
and that the appropriate representation of the resource is being
returned. The token used MUST be any token that was received in a
request using the same Request-Tag. However, it is desirable to
provide the one used in the last received request.If the FETCH request includes the Observe Option, then the
server MUST use the same token as used for the initial response
for returning any Observe triggered responses so that the client
can match them up.The client should then release all of the tokens used for this
body unless a resource is being observed.2.31 (Continue)This Response Code can be used to indicate that all of the
blocks up to and including the Q-Block1 Option block NUM (all
having the M bit set) have been successfully received. The token
used MUST be any token that was received in a request using the
same Request-Tag. However, it is desirable to provide the one used
in the last received request.A response using this Response Code SHOULD NOT be generated for
every received Q-Block1 Option request (). It SHOULD only be generated when all the
payload requests are Non-confirmable and a set of MAX_PAYLOADS
payloads have been received by the server. More details about the
motivations for this optimization are discussed in .It SHOULD NOT be generated for CON.4.00 (Bad Request)This Response Code MUST be returned if the request does not
include neither a Request-Tag Option nor a Size1 Option but does
include a Q-Block1 option.4.02 (Bad Option)Either this Response Code (in case of Confirmable request) or a
reset message (in case of Non-confirmable request) MUST be
returned if the server does not support the Q-Block Options.4.08 (Request Entity Incomplete)As a reminder, this Response Code returned without Content-Type
"application/missing-blocks+cbor-seq" () is handled as in Section 2.9.2 .This Response Code returned with Content-Type
"application/missing-blocks+cbor-seq" indicates that some of the
payloads are missing and need to be resent. The client then
retransmits the missing payloads using the same Request-Tag, Size1
and Q-Block1 to specify the block NUM, SZX, and M bit as
appropriate.The Request-Tag value to use is determined by taking the token
in the 4.08 (Request Entity Incomplete) response, locating the
matching client request, and then using its Request-Tag.The token used MUST be any token that was received in a request
using the same Request-Tag. However, it is desirable to provide
the one used in the last received request. See for further information.If the server has not received all the blocks of a body, but
one or more NON payloads have been received, it SHOULD wait for up
to NON_RECEIVE_TIMEOUT () before
sending a 4.08 (Request Entity Incomplete) response.4.13 (Request Entity Too Large)This Response Code can be returned under similar conditions to
those discussed in Section 2.9.3 of .This Response Code can be returned if there is insufficient
space to create a response PDU with a block size of 16 bytes (SZX
= 0) to send back all the response options as appropriate. In this
case, the Size1 Option is not included in the response.Further considerations related to the transmission timings of 4.08
(Request Entity Incomplete) and 2.31 (Continue) Response Codes are
discussed in .If a server receives payloads with different Request-Tags for the
same resource, it should continue to process all the bodies as it has
no way of determining which is the latest version, or which body, if
any, the client is terminating the transmission for.If the client elects to stop the transmission of a complete body,
and absent any local policy, the client MUST "forget" all tracked
tokens associated with the body's Request-Tag so that a reset message
is generated for the invalid token in the 4.08 (Request Entity
Incomplete) response. The server on receipt of the reset message
SHOULD delete the partial body.If the server receives a duplicate block with the same Request-Tag,
it MUST ignore the payload of the packet, but MUST still respond as if
the block was received for the first time.A server SHOULD maintain a partial body (missing payloads) for up
to NON_PARTIAL_TIMEOUT ().In a request for any block number, the M bit unset indicates the
request is just for that block. If the M bit is set, this has
different meanings based on the NUM value:This is a request for the entire
body.This
is used to confirm that the current set of MAX_PAYLOADS payloads
(the latest one having block number NUM-1) has been successfully
received and that, upon receipt of this request, the server can
continue to send the next set of payloads (the first one having
block number NUM). This is the 'Continue' Q-Block-2 and
conceptually has the same usage (i.e., continue sending the next
set of data) as the use of 2.31 (Continue) for Q-Block1.This is a request for that
block and for all of the remaining blocks in the current
MAX_PAYLOADS set.If the request includes multiple Q-Block2 Options and these options
overlap (e.g., combination of M being set (this and later blocks) and
being unset (this individual block)) resulting in an individual block
being requested multiple times, the server MUST only send back one
instance of that block. This behavior is meant to prevent
amplification attacks.The payloads sent back from the server as a response MUST all have
the same ETag (Section 5.10.6 of ) for
the same body. The server MUST NOT use the same ETag value for
different representations of a resource.The ETag is opaque, the client still treats it as opaque but the
server MUST ensure that it is unique for every different body of
transmitted data.Implementation Note: It is suggested that the server treats the
ETag as an unsigned integer of 8 bytes in length. An
implementation may want to consider limiting this to 4 bytes to
reduce packet overhead size. The initial ETag value should be
randomly generated and then subsequently incremented by the server
whenever a new body of data is being transmitted between
peers. discusses the use of Size2 Option.The client may elect to request any detected missing blocks or just
ignore the partial body. This decision is implementation specific.The client SHOULD wait for up to NON_RECEIVE_TIMEOUT () after the last received payload for NON
payloads before issuing a GET, POST, PUT, FETCH, PATCH, or iPATCH
request that contains one or more Q-Block2 Options that define the
missing blocks with the M bit unset. It is permissible to set the M
bit to request this and later blocks from this MAX_PAYLOADS set.
Further considerations related to the transmission timing for missing
requests are discussed in .The requested missing block numbers MUST have an increasing block
number in each additional Q-Block2 Option with no duplicates. The
server SHOULD respond with a 4.00 (Bad Request) to requests not
adhering to this behavior. Note that the ordering constraint is meant
to force the client to check for duplicates and remove them. This also
helps with troubleshooting.For Confirmable responses, the client continues to acknowledge each
packet. Typically, the server acknowledges the initial request using
an ACK with the payload, and then sends the subsequent payloads as CON
responses. The server will detect failure to send a packet, but the
client can issue, after a MAX_TRANSMIT_SPAN delay, a separate GET,
POST, PUT, FETCH, PATCH, or iPATCH for any missing blocks as
needed.If the client receives a duplicate block with the same ETag, it
MUST silently ignore the payload.A client SHOULD maintain a partial body (missing payloads) for up
to NON_PARTIAL_TIMEOUT () or as defined
by the Max-Age Option (or its default of 60 seconds (Section 5.6.1 of
)), whichever is the less. On release of
the partial body, the client should then release all of the tokens
used for this body unless a resource is being observed.The ETag Option should not be used in the request for missing
blocks as the server could respond with a 2.03 (Valid) response with
no payload. It can be used in the request if the client wants to check
the freshness of the locally cached body response.It is RECOMMENDED that the server maintains a cached copy of the
body when using the Q-Block2 Option to facilitate retransmission of
any missing payloads.If the server detects part way through a body transfer that the
resource data has changed and the server is not maintaining a cached
copy of the old data, then the transmission is terminated. Any
subsequent missing block requests MUST be responded to using the
latest ETag and Size2 Option values with the updated data.If the server responds during a body update with a different ETag
Option value (as the resource representation has changed), then the
client should treat the partial body with the old ETag as no longer
being fresh.If the server transmits a new body of data (e.g., a triggered
Observe notification) with a new ETag to the same client as an
additional response, the client should remove any partially received
body held for a previous ETag for that resource as it is unlikely the
missing blocks can be retrieved.If there is insufficient space to create a response PDU with a
block size of 16 bytes (SZX = 0) to send back all the response options
as appropriate, a 4.13 (Request Entity Too Large) is returned without
the Size1 Option.For a request that uses Q-Block1, the Observe value MUST be the same for all the payloads of the
same body. This includes any missing payloads that are
retransmitted.For a response that uses Q-Block2, the Observe value MUST be the
same for all the payloads of the same body. This includes payloads
transmitted following receipt of the 'Continue' Q-Block2 Option () by the server. If a missing payload is
requested, then both the request and response MUST NOT include the
Observe Option.Section 4 of defines two CoAP
options: Size1 for indicating the size of the representation
transferred in requests and Size2 for indicating the size of the
representation transferred in responses.The Size1 or Size2 option values MUST exactly represent the size of
the data on the body so that any missing data can easily be
determined.The Size1 Option MUST be used with the Q-Block1 Option when used in
a request and MUST be present in all payloads of the request
preserving the same value. The Size2 Option MUST be used with the
Q-Block2 Option when used in a response and MUST be present in all
payloads of the response preserving the same value.The behavior is similar to the one defined in Section 3.3 of with Q-Block1 substituted for Block1 and
Q-Block2 for Block2.Servers MUST ignore multicast requests that contain the Q-Block2
Option. As a reminder, Block2 Option can be used as stated in Section
2.8 of .4.08 (Request Entity Incomplete) Response Code has a new Content-Type
"application/missing-blocks+cbor-seq" used to indicate that the server
has not received all of the blocks of the request body that it needs to
proceed. Such messages must not be treated by the client as a fatal
error.Likely causes are the client has not sent all blocks, some blocks
were dropped during transmission, or the client has sent them
sufficiently long ago that the server has already discarded them.The data payload of the 4.08 (Request Entity Incomplete) response is
encoded as a CBOR Sequence . It comprises
of one or more missing block numbers encoded as CBOR unsigned integers
. The missing block numbers MUST be unique
in each 4.08 (Request Entity Incomplete) response when created by the
server; the client MUST ignore any duplicates in the same 4.08 (Request
Entity Incomplete) response.The Content-Format Option (Section 5.10.3 of ) MUST be used in the 4.08 (Request Entity
Incomplete) response. It MUST be set to
"application/missing-blocks+cbor-seq" ().The Concise Data Definition Language
(and see Section 4.1 ) for the data
describing these missing blocks is as follows:The token to use for the response SHOULD be the token that was used
in the last block number received so far with the same Request-Tag
value. Note that the use of any received token with the same Request-Tag
would be acceptable, but providing the one used in the last received
payload will aid any troubleshooting. The client will use the token to
determine what was the previously sent request to obtain the Request-Tag
value to be used.If the size of the 4.08 (Request Entity Incomplete) response packet
is larger than that defined by Section 4.6 , then the number of missing blocks MUST be
limited so that the response can fit into a single packet. If this is
the case, then the server can send subsequent 4.08 (Request Entity
Incomplete) responses containing the missing other blocks on receipt of
a new request providing a missing payload with the same Request-Tag.The missing blocks MUST be reported in ascending order without any
duplicates. The client SHOULD silently drop 4.08 (Request Entity
Incomplete) responses not adhering with this behavior.Consider limiting the number of
missing payloads to MAX_PAYLOADS to minimize congestion control
being needed. The CBOR sequence does not include any array
wrapper.The 4.08 (Request Entity Incomplete) with Content-Type
"application/missing-blocks+cbor-seq" SHOULD NOT be used when using
Confirmable requests or a reliable connection as the client will be able to determine that
there is a transmission failure of a particular payload and hence that
the server is missing that payload.Each new request generally uses a new Token (and sometimes must, see
Section 4 of ).
Additional responses to a request all use the token of the request they
respond to.By using 8-byte tokens, it is
possible to easily minimize the number of tokens that have to be
tracked by clients, by keeping the bottom 32 bits the same for the
same body and the upper 32 bits containing the current body's
request number (incrementing every request, including every
re-transmit). This allows the client to be alleviated from keeping
all the per-request-state, e.g., in Section 3 of .The transmission of the blocks of a body over an unreliable transport
MUST either all be Confirmable or all be Non-confirmable. This is meant
to simplify the congestion control procedure.As a reminder, there is no need for CoAP-specific congestion control
for reliable transports .Congestion control for CON requests and responses is specified in
Section 4.7 of . For faster transmission
rates, NSTART will need to be increased from 1. However, the other CON
congestion control parameters will need to be tuned to cover this
change. This tuning is out of scope of this document as it is expected
that all requests and responses using Q-Block1 and Q-Block2 will be
Non-confirmable ().It is implementation specific as to whether there should be any
further requests for missing data as there will have been significant
transmission failure as individual payloads will have failed after
MAX_TRANSMIT_SPAN.This document introduces new parameters MAX_PAYLOADS, NON_TIMEOUT,
NON_RECEIVE_TIMEOUT, NON_MAX_RETRANSMIT, NON_PROBING_WAIT, and
NON_PARTIAL_TIMEOUT primarily for use with NON (Table 3).MAX_PAYLOADS should be configurable with a default value of 10.
Both CoAP endpoints SHOULD have the same value (otherwise there will
be transmission delays in one direction) and the value MAY be
negotiated between the endpoints to a common value by using a higher
level protocol (out of scope of this document). This is the maximum
number of payloads that can be transmitted at any one time.Note: The default value of 10 is chosen for reasons similar to
those discussed in Section 5 of .NON_TIMEOUT is the maximum period of delay between sending sets of
MAX_PAYLOADS payloads for the same body. By default, NON_TIMEOUT has
the same value as ACK_TIMEOUT (Section 4.8 of ).NON_RECEIVE_TIMEOUT is the initial maximum time to wait for a
missing payload before requesting retransmission for the first time.
Every time the missing payload is re-requested, the time to wait value
doubles. The time to wait is calculated as:Time-to-Wait = NON_RECEIVE_TIMEOUT * (2 ** (Re-Request-Count -
1))NON_RECEIVE_TIMEOUT has a default value of twice NON_TIMEOUT.
NON_RECEIVE_TIMEOUT MUST always be greater than NON_TIMEOUT by at
least one second so that the sender of the payloads has the
opportunity to start sending the next set of payloads before the
receiver times out.NON_MAX_RETRANSMIT is the maximum number of times a request for the
retransmission of missing payloads can occur without a response from
the remote peer. After this occurs, the local endpoint SHOULD consider
the body stale, remove any body, and release Tokens and Request-Tag on
the client (or the ETag on the server). By default, NON_MAX_RETRANSMIT
has the same value as MAX_RETRANSMIT (Section 4.8 of ).NON_PROBING_WAIT is used to limit the potential wait needed
calculated when using PROBING_WAIT. NON_PROBING_WAIT has the same
value as computed for EXCHANGE_LIFETIME (Section 4.8.2 of ).NON_PARTIAL_TIMEOUT is used for expiring partially received bodies.
NON_PARTIAL_TIMEOUT has the same value as computed for
EXCHANGE_LIFETIME (Section 4.8.2 of ).PROBING_RATE parameter in CoAP indicates the average data rate that
must not be exceeded by a CoAP endpoint in sending to a peer endpoint
that does not respond. The single body of blocks will be subjected to
PROBING_RATE (Section 4.7 of ), not the
individual packets. If the wait time between sending bodies that are
not being responded to based on PROBING_RATE exceeds NON_PROBING_WAIT,
then the gap time is limited to NON_PROBING_WAIT.Note: For the particular DOTS application, PROBING_RATE and
other transmission parameters are negotiated between peers. Even
when not negotiated, the DOTS application uses customized defaults
as discussed in Section 4.5.2 of .
Note that MAX_PAYLOADS, NON_MAX_RETRANSMIT, and NON_TIMEOUT can be
negotiated between DOTS peers as per .Each NON 4.08 (Request Entity Incomplete) response is subject
to PROBING_RATE.Each NON GET or FETCH request using Q-Block2 Option is subject to
PROBING_RATE.As the sending of many payloads of a single body may itself cause
congestion, it is RECOMMENDED that after transmission of every set of
MAX_PAYLOADS payloads of a single body, a delay is introduced of
NON_TIMEOUT before sending the next set of payloads to manage
potential congestion issues.If the CoAP peer reports at least one payload has not arrived for
each body for at least a 24 hour period and it is known that there are
no other network issues over that period, then the value of
MAX_PAYLOADS can be reduced by 1 at a time (to a minimum of 1) and the
situation re-evaluated for another 24 hour period until there is no
report of missing payloads under normal operating conditions. The
newly derived value for MAX_PAYLOADS should be used for both ends of
this particular CoAP peer link. Note that the CoAP peer will not know
about the MAX_PAYLOADS change until it is reconfigured. As a
consequence of the two peers having different MAX_PAYLOADS values, a
peer may continue indicate that there are some missing payloads as all
of its MAX_PAYLOADS set may not have arrived. How the two peer values
for MAX_PAYLOADS are synchronized is out of the scope.The sending of a set of missing blocks of a body is subject to
MAX_PAYLOADS set of payloads.For Q-Block1 Option, if the server responds with a 2.31 (Continue)
Response Code for the latest payload sent, then the client can
continue to send the next set of payloads without any delay. If the
server responds with a 4.08 (Request Entity Incomplete) Response Code,
then the missing payloads SHOULD be retransmitted before going into
another NON_TIMEOUT delay prior to sending the next set of
payloads.For the server receiving NON Q-Block1 requests, it SHOULD send back
a 2.31 (Continue) Response Code on receipt of all of the MAX_PAYLOADS
payloads to prevent the client unnecessarily delaying. Otherwise the
server SHOULD delay for NON_RECEIVE_TIMEOUT (exponentially scaled
based on the repeat request count for a payload), before sending the
4.08 (Request Entity Incomplete) Response Code for the missing
payload(s). If this is a repeat for the 2.31 (Continue) response, the
server SHOULD send a 4.08 (Request Entity Incomplete) response
detailing the missing payloads after the block number that would have
been indicated in the 2.31 (Continue). If the repeat request count for
a missing payload exceeds NON_MAX_RETRANSMIT, the server SHOULD
discard the partial body and stop requesting the missing payloads.It is likely that the client will start transmitting the next set
of MAX_PAYLOADS payloads before the server times out on waiting for
the last of the previous MAX_PAYLOADS payloads. On receipt of the
first payload from the new set of MAX_PAYLOADS payloads, the server
SHOULD send a 4.08 (Request Entity Incomplete) Response Code
indicating any missing payloads from any previous MAX_PAYLOADS
payloads. Upon receipt of the 4.08 (Request Entity Incomplete)
Response Code, the client SHOULD send the missing payloads before
continuing to send the remainder of the MAX_PAYLOADS payloads and then
go into another NON_TIMEOUT delay prior to sending the next set of
payloads.For the client receiving NON Q-Block2 responses, it SHOULD send a
'Continue' Q-Block2 request () for the
next set of payloads on receipt of all of the MAX_PAYLOADS payloads to
prevent the server unnecessarily delaying. Otherwise the client SHOULD
delay for NON_RECEIVE_TIMEOUT (exponentially scaled based on the
repeat request count for a payload), before sending the request for
the missing payload(s). If the repeat request count for a missing
payload exceeds NON_MAX_RETRANSMIT, the client SHOULD discard the
partial body and stop requesting the missing payloads.The server SHOULD recognize the 'Continue' Q-Block2 request as a
continue request and just continue the transmission of the body
(including Observe Option, if appropriate for an unsolicited response)
rather than as a request for the remaining missing blocks.It is likely that the server will start transmitting the next set
of MAX_PAYLOADS payloads before the client times out on waiting for
the last of the previous MAX_PAYLOADS payloads. Upon receipt of the
first payload from the new set of MAX_PAYLOADS payloads, the client
SHOULD send a request indicating any missing payloads from any
previous set of MAX_PAYLOADS payloads. Upon receipt of such request,
the server SHOULD send the missing payloads before continuing to send
the remainder of the MAX_PAYLOADS payloads and then go into another
NON_TIMEOUT delay prior to sending the next set of payloads.The client does not need to acknowledge the receipt of the entire
body.Note: If there is asymmetric traffic loss causing responses to
never get received, a delay of NON_TIMEOUT after every
transmission of MAX_PAYLOADS blocks will be observed. The endpoint
receiving the body is still likely to receive the entire body.Caching block based information is not straight forward in a proxy.
For Q-Block1 and Q-Block2 Options, for simplicity it is expected that
the proxy will reassemble the body (using any appropriate recovery
options for packet loss) before passing on the body to the appropriate
CoAP endpoint. This does not preclude an implementation doing a more
complex per payload caching, but how to do this is out of the scope of
this document. The onward transmission of the body does not require the
use of the Q-Block1 or Q-Block2 Options as these options may not be
supported in that link. This means that the proxy must fully support the
Q-Block1 and Q-Block2 Options.How the body is cached in the CoAP client (for Q-Block1
transmissions) or the CoAP server (for Q-Block2 transmissions) is
implementation specific.As the entire body is being cached in the proxy, the Q-Block1 and
Q-Block2 Options are removed as part of the block assembly and thus do
not reach the cache.For Q-Block2 responses, the ETag Option value is associated with the
data (and onward transmitted to the CoAP client), but is not part of the
cache key.For requests with Q-Block1 Option, the Request-Tag Option is
associated with the build up of the body from successive payloads, but
is not part of the cache key. For the onward transmission of the body
using CoAP, a new Request-Tag SHOULD be generated and used. Ideally this
new Request-Tag should replace the client's request Request-Tag.It is possible that two or more CoAP clients are concurrently
updating the same resource through a common proxy to the same CoAP
server using Q-Block1 (or Block1) Option. If this is the case, the first
client to complete building the body causes that body to start
transmitting to the CoAP server with an appropriate Request-Tag value.
When the next client completes building the body, any existing partial
body transmission to the CoAP server is terminated and the new body
representation transmission starts with a new Request-Tag value. Note
that it cannot be assumed that the proxy will always receive a complete
body from a client.A proxy that supports Q-Block2 Option MUST be prepared to receive a
GET or similar request indicating one or more missing blocks. The proxy
will serve from its cache the missing blocks that are available in its
cache in the same way a server would send all the appropriate Q-Block2
responses. If the cache key matching body is not available in the cache,
the proxy MUST request the entire body from the CoAP server using the
information in the cache key.How long a CoAP endpoint (or proxy) keeps the body in its cache is
implementation specific (e.g., it may be based on Max-Age).As a reminder, the basic normative requirements on HTTP/CoAP mappings
are defined in Section 10 of . The
implementation guidelines for HTTP/CoAP mappings are elaborated in .The rules defined in Section 5 of are
to be followed.This section provides some sample flows to illustrate the use of
Q-Block1 and Q-Block2 Options with NON. Examples with CON are provided
in .The examples in the following subsections assume MAX_PAYLOADS is set
to 10 and NON_MAX_RETRANSMIT is set to 4. lists the conventions that are used in
the following subsections. depicts an example of a NON PUT
request conveying Q-Block1 Option. All the blocks are received by
the server. depicts an example of a NON PUT
request conveying Q-Block1 Option. The number of payloads exceeds
MAX_PAYLOADS. All the blocks are received by the server.Consider now a scenario where a new body of data is to be sent by
the client, but some blocks are dropped in transmission as
illustrated in .On seeing a payload from the next set of payloads, the server
realizes that some blocks are missing from the previous MAX_PAYLOADS
payloads and asks for the missing blocks in one go (). It does so by indicating which blocks from
the previous MAX_PAYLOADS payloads have not been received in the
data portion of the response. The token used in the response should
be the token that was used in the last received payload. The client
can then derive the Request-Tag by matching the token with the sent
request. depicts an example of a NON PUT
request conveying Q-Block1 Option where recovery takes place, but
eventually fails.These examples include the Observe Option to demonstrate how that
option is used. Note that the Observe Option is not required for
Q-Block2; the observe detail can thus be ignored. illustrates the example of Q-Block2
Option. The client sends a NON GET carrying Observe and Q-Block2
Options. The Q-Block2 Option indicates a block size hint (1024
bytes). This request is replied to by the server using four (4)
blocks that are transmitted to the client without any loss. Each of
these blocks carries a Q-Block2 Option. The same process is repeated
when an Observe is triggered, but no loss is experienced by any of
the notification blocks. illustrates the same as but this time has eleven (11) payloads which
exceeds MAX_PAYLOADS. There is no loss experienced. shows the example of an Observe
that is triggered but for which some notification blocks are lost.
The client detects the missing blocks and requests their
retransmission. It does so by indicating the blocks that are missing
as one or more Q-Block2 Options. shows the example of an Observe
that is triggered but only the first two notification blocks reach
the client. In order to retrieve the missing blocks, the client
sends a request with a single Q-Block2 Option with the M bit
set. illustrates the example of a FETCH
using both Q-Block1 and Q-Block2 Options along with an Observe
Option. No loss is experienced. illustrates the same as but this time has eleven (11) payloads in
both directions which exceeds MAX_PAYLOADS. There is no loss
experienced.Consider now a scenario where some blocks are lost in
transmission as illustrated in .The server realizes that some blocks are missing and asks for the
missing blocks in one go (). It does
so by indicating which blocks have not been received in the data
portion of the response. The token used in the response is the token
that was used in the last received payload. The client can then
derive the Request-Tag by matching the token with the sent
request.The client realizes that not all the payloads of the response
have been returned. The client then asks for the missing blocks in
one go (). Note that, following
Section 2.7 of , the FETCH request
does not include the Q-Block1 or any payload.Security considerations discussed in Section 7 of should be taken into account.Security considerations discussed in Sections 11.3 and 11.4 of should be taken into account.OSCORE provides end-to-end protection of all information that is not
required for proxy operations and requires that a security context is
set up (Section 3.1 of ). It can be
trusted that the source endpoint is legitimate even if NoSec security
mode is used. However, an intermediary node can modify the unprotected
outer Q-Block1 and/or Q-Block2 Options to cause a Q-Block transfer to
fail or keep requesting all the blocks by setting the M bit and, thus,
causing attack amplification. As discussed in Section 12.1 of , applications need to consider that certain
message fields and messages types are not protected end-to-end and may
be spoofed or manipulated. It is NOT RECOMMENDED that the NoSec security
mode is used if the Q-Block1 and Q-Block2 Options are to be used.Security considerations related to the use of Request-Tag are
discussed in Section 5 of .Please replace [RFCXXXX] with the RFC
number to be assigned to this document.IANA is requested to add the following entries to the "CoAP Option
Numbers" sub-registry defined in within the "Constrained RESTful Environments
(CoRE) Parameters" registry:This document suggests 19 (TBA1) and 31 (TBA2) as values to be
assigned for the new option numbers.This document requests IANA to register the
"application/missing-blocks+cbor-seq" media type in the "Media Types"
registry . This registration
follows the procedures specified in :
This document requests IANA to register the following CoAP
Content-Format for the "application/missing-blocks+cbor-seq" media
type in the "CoAP Content-Formats" registry , defined in ,
within the "Constrained RESTful Environments (CoRE) Parameters"
registry:This document suggests 272 (TBA3) as a value to be assigned for the
new content format number.CoAP Content-FormatsCoAP Option NumbersMedia TypesThe following examples assume NSTART has been increased to 3.The notations provided in are used in
the following subsections.Let's now consider the use of Q-Block1 Option with a CON request as
shown in . All the blocks are acknowledged
(ACK).Now, suppose that a new body of data is to be sent but with some
blocks dropped in transmission as illustrated in . The client will retry sending blocks for which
no ACK was received.It is up to the implementation as to whether the application
process stops trying to send this particular body of data on reaching
MAX_RETRANSMIT for any payload, or separately tries to initiate the
new transmission of the payloads that have not been acknowledged under
these adverse traffic conditions.If there is likely to be the possibility of network transient
losses, then the use of NON should be considered.An example of the use of Q-Block2 Option with Confirmable messages
is shown in .It is up to the implementation as to whether the application
process stops trying to send this particular body of data on reaching
MAX_RETRANSMIT for any payload, or separately tries to initiate the
new transmission of the payloads that have not been acknowledged under
these adverse traffic conditions.If there is likely to be the possibility of network transient
losses, then the use of NON should be considered.The notations provided in are used in
the following subsections.Let's now consider the use of Q-Block1 Option with a reliable
transport as shown in . There is no
acknowledgment of packets at the CoAP layer, just the final
result.If there is likely to be the possibility of network transient
losses, then the use of unreliable transport with NON should be
considered.An example of the use of Q-Block2 Option with a reliable transport
is shown in .If there is likely to be the possibility of network transient
losses, then the use of unreliable transport with NON should be
considered.Thanks to Achim Kraus, Jim Schaad, and Michael Richardson for their
comments.Special thanks to Christian Amsüss, Carsten Bormann, and Marco
Tiloca for their suggestions and several reviews, which improved this
specification significantly. Thanks to Francesca Palombini for the AD
review.Some text from is reused for readers
convenience.