NWCRG V. Roca (Ed.) Internet-Draft INRIA Intended status: Informational J. Detchart Expires: May 20, 2020 ISAE - Supaero C. Adjih INRIA M. Pedersen Steinwurf ApS November 17, 2019 Generic Application Programming Interface (API) for Sliding Window FEC Codes draft-roca-nwcrg-generic-fec-api-07 Abstract This document introduces a generic Application Programming Interface (API) for sliding window FEC codes. This API is meant to be compatible with any sliding window FEC code. It defines the core procedures and functions meant to control the codec (i.e., implementation of the FEC code). However, it leaves out all upper layer aspects that are the responsibility of the application or protocol making use of the codec. As a consequence, this is not an API for a FEC Scheme since certain mechanisms that must be defined by any FEC Scheme (e.g., signalling and FEC Payload IDs) are the responsibility of the caller instead of being addressed by the codec. A first goal of this document is to pave the way for a future open- source implementation of such codes, another goal is to simplify the development of content delivery protocols that rely on sliding window FEC codes for robust transmissions. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 20, 2020. Roca (Ed.), et al. Expires May 20, 2020 [Page 1] Internet-Draft Generic API November 2019 Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Definitions and Abbreviations . . . . . . . . . . . . . . . . 3 3. AL-FEC Codes and Mechanisms Considered by the Generic API . . 3 3.1. Mechanisms Considered or Ignored by the API . . . . . . . 5 4. Generic API for Sliding Window FEC Codes . . . . . . . . . . 6 4.1. General Definitions Common to the Encoder and Decoder . . 6 4.2. Encoder . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.3. Decoder . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.4. Coding Window Functions at an Encoder and Decoder . . . . 17 4.5. Coding Coefficients Functions at an Encoder and Decoder . 19 5. Security Considerations . . . . . . . . . . . . . . . . . . . 22 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 8.1. Normative References . . . . . . . . . . . . . . . . . . 23 8.2. Informative References . . . . . . . . . . . . . . . . . 23 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 1. Introduction Forward Erasure Correction (FEC) codes are a key element of communication systems, used to efficiently recover from packet losses during content delivery sessions. Among the FEC codes working at the network and higher layers, one can broadly distinguish block codes and sliding window codes. Block FEC codes require the data flow coming from the application to be segmented into blocks of a predefined maximum size, before generating a certain number of repair packets. With the second type of FEC codes, an encoding window continuously slides over the set of source data and repair packets are generated at any time by computing for instance a linear combination of data present in the encoding window. This fundamental Roca (Ed.), et al. Expires May 20, 2020 [Page 2] Internet-Draft Generic API November 2019 difference seriously impacts the way they can be used by a content delivery protocol or application. This document introduces a generic Application Programming Interface (API) for sliding window FEC codes. This API is meant to be usable by any sliding window FEC code and FEC Scheme independently of the protocol that may rely on it. This API defines the core procedures and functions meant to control the codec (i.e., implementation of the FEC code), but leaves out all upper layer aspects that are the responsibility of the application making use of the codec. This API is meant to be usable by any sliding window FEC code. independently of the FEC Scheme or network coding protocol that may rely on it This API defines the core procedures and functions meant to control the codec (i.e., implementation of the FEC code), but leaves out all upper layer aspects that are the responsibility of the application making use of the codec. For instance, those restricted to end-to-end use-cases as well as those compatible with in-network re-encoding use-cases. Additionally, this API is not impacted by the intra-flow versus inter-flow nature of the use-case, nor is it impacted by the single-path versus multi-paths nature of the use- case, since those are usage considerations under the responsibility of the caller. A goal of this document is to pave the way for a future open-source implementation of such codes. 2. Definitions and Abbreviations The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. This document uses the following definitions and abbreviations: XXX 3. AL-FEC Codes and Mechanisms Considered by the Generic API This generic FEC API is meant to be used with: o sliding window codes, that manage an encoding window (of fixed or variable size) that slides over the set of source symbols at the sender. On the opposite, block codes (e.g., Reed-Solomon, LDPC, Raptor(Q)) are out of scope; o codes that are restricted to use-cases that involve a single encoding point and a single decoding point (i.e., FEC operations are carried out either within the end-hosts or middle-boxes), as Roca (Ed.), et al. Expires May 20, 2020 [Page 3] Internet-Draft Generic API November 2019 well as codes that can be used with use-cases that involve in- network re-coding operations; o use-cases that are limited to an intra-flow coding (simple case), as well as use-cases that involve inter-flow coding. This second case is more complex to address (e.g., with questions such as how to identify a packet of a flow?) however this is the responsibility of the application or protocol using this codec and not the codec itself. This aspect is therefore transparent to the API; o use-cases that are limited to single-path communications and use- cases that consider multi-path communications. Here also this is a usage consideration that is transparent to the API; o use-cases that involve a dynamic adaptation of the codec parameters (e.g., its code rate because the communication path losses is known thanks to feedbacks and an appropriate strategy can be defined); o fixed code rate or not FEC codes, including rateless codes where the number of repair symbols that can be generated is huge (in theory unlimited); o ideal (MDS) or non-ideal (non-MDS) codes. However most of the time, sliding window codes are non-ideal codes, meaning that slightly more than l repair symbols may be required to recover all the l lost source symbols; A key question is to determine what mechanisms are included in the codec and what mechanisms are left to the responsibility of the caller (i.e., an application or a protocol making use of this codec) (Figure 1). More precisely, an FEC Scheme (such as the RLC FEC Scheme [RLC] in case of FECFRAME [fecframe-ext]) defines all the internal code details in order to enable interoperable implementations, but also signaling considerations that are essential to use them in a specific context. Roca (Ed.), et al. Expires May 20, 2020 [Page 4] Internet-Draft Generic API November 2019 +----------------------------------------------------------+ | Application / Protocol Mechanisms | | (out of scope) | | | | + - - - - - - - - - - - - - - - - - - - - - + | | FEC Scheme | | +--|-------------------------------------------|-+ | | | | | <----- FEC Codec API (this doc.) -----> | | | +---------------------------------------+ | | | | Low Level FEC Codec | | | | +---------------------------------------+ +---------+ + - - - - - - - - - - - - - - - - - - - - - + Figure 1: Position of the FEC Codec API with respect to the low level FEC Codec, the FEC Scheme, the protocol and other caller services. 3.1. Mechanisms Considered or Ignored by the API Applying FEC coding, through an FEC Scheme, in a given protocol to improve transmission robustness involves many mechanisms. However, these mechanisms are not all the responsibility of the codec and can be implemented within the application or within the protocol that uses this FEC codec. For instance, the following mechanisms are considered **out of scope of the API**, being implemented by the caller, without any impact on the codec: o memory management; o packet transmission and reception; o signaling header creation / parsing; o ADU to source symbol mapping; o code rate adjustment, for instance thanks to the knowledge of losses at a receiver via feedbacks; o selective ACK creation and parsing; o congestion control. The following mechanisms are **within scope of the API**: o session management (sender and receiver); o encoding window management (sender and receiver); o set/get/generate coding coefficients (sender and receiver); o build coded symbol (sender only); o decode with newly received source or repair symbol (receiver only); Roca (Ed.), et al. Expires May 20, 2020 [Page 5] Internet-Draft Generic API November 2019 4. Generic API for Sliding Window FEC Codes The following sections describe the generic API, following a C-language formalism. This API tries to adhere to C99 version of C, although it may not strictly be guaranteed. Everything is prefixed by "swif" (sliding window FEC). 4.1. General Definitions Common to the Encoder and Decoder This section gathers general definitions that are used by both an encoder and decoder. About FEC Codepoints: An application first needs to negotiate with its remote side the right FEC Scheme to use. This negotiation usually relies on the FEC Encoding ID associated to this FEC Scheme for this application. A difficulty is that the FEC Encoding ID space, associated to an IANA registry, is protocol specific and the same value are usually associated to different FEC Schemes depending on the protocol. For instance, the FEC Encoding ID value 2 may be used for two totally different FEC Schemes in protocol A and protocol B. Therefore, the FEC Encoding ID, from the Generic FEC API point of view, cannot be used to uniquely identify the target codec. The use of a codepoint to identify locally the right FEC codec requires that the application knows a mapping between the FEC Encoding ID it uses for a given protocol, and the local FEC Codepoints corresponding to available codecs. This will be done at development time, the FEC API header file giving access to the swif_codepoint_t enumeration with the list of all codecs available locally. /** * Return value of any function. * * SWIF_STATUS_OK = 0 Success * SWIF_STATUS_FAILURE Failure. The function called did not succeed to * perform its task, however this is not an error * (e.g., it happens when decoding fails). * SWIF_STATUS_ERROR Generic error type. The detailed error type is * stored in the errno variable of swif_encoder_t and * swif_decoder_t structures. */ typedef enum { SWIF_STATUS_OK = 0, SWIF_STATUS_FAILURE, Roca (Ed.), et al. Expires May 20, 2020 [Page 6] Internet-Draft Generic API November 2019 SWIF_STATUS_ERROR } swif_status_t; /** * Potential errors. */ typedef enum { SWIF_ERRNO_NULL = 0, /* everything is fine */ SWIF_ERRNO_UNSUPPORTED_CODEPOINT, /* and many more... */ } swif_errno_t; /** * FEC Codepoints. * These identifiers are opaque identifiers that fully identify an FEC * code locally, including certain parameters like its Galois Field. * These codepoints are codec specific and only have a local meaning. * They should not be transmitted as different implementations may use * them inconsistently. * Note that the same FEC code may be used by several FEC Encoding IDs * and therefore share the same codepoint. On the opposite multiple * implementations of a given FEC code may exist locally, for instance * with different optimizations, and then several codepoints, one per * codec, will exist for the same FEC code. The following names are * therefore only provided as examples. */ typedef enum { SWIF_CODEPOINT_NULL = 0, /* codepoint 0 is reserved */ /* codepoint for sliding window codec AAA. */ SWIF_CODEPOINT_AAA_CODEC, /* codepoint for sliding window codec BBB. */ SWIF_CODEPOINT_BBB_CODEC, /* list here other identifiers for any codec of interest... */ } swif_codepoint_t; /** * Encoding Symbol Identifier (ESI) generic type. * With Sliding Window FEC codes, an ESI is in fact a source symbol * identifier, unlike block FEC codes. */ typedef uint32_t esi_t; Roca (Ed.), et al. Expires May 20, 2020 [Page 7] Internet-Draft Generic API November 2019 /** * Throughout the API, a pointer to this structure is used as an * identifier of the encoder instance (or "enc"). * * This generic structure is meant to be extended by each codec with * new pieces of information that are specific to each codec. */ typedef struct swif_encoder { swif_codepoint_t codepoint; /* when a function returns with SWIF_STATUS_ERROR, the errno * variable contains a more detailed error type. This variable * is set by the codec and accessible to the application in * READ ONLY mode. Otherwise its value is undefined. */ swif_errno_t swif_errno; /* pointers to codec specific versions of API functions. */ swif_status_t (*set_callback_functions) ( struct swif_encoder*, void (*) (void*, esi_t), void*); swif_status_t (*set_parameters) ( struct swif_encoder*, uint32_t, uint32_t, void*); swif_status_t (*get_parameters) ( struct swif_encoder*, uint32_t, uint32_t, void*); swif_status_t (*build_repair_symbol) ( struct swif_encoder*, void*); swif_status_t (*reset_coding_window) (struct swif_encoder*); swif_status_t (*add_source_symbol_to_coding_window) ( struct swif_encoder*, void*, esi_t); swif_status_t (*remove_source_symbol_from_coding_window) ( struct swif_encoder*, esi_t); swif_status_t (*get_coding_window_information) ( struct swif_encoder*, esi_t*, esi_t*, uint32_t*); swif_status_t (*set_coding_coefs_tab) ( struct swif_encoder*, void*, uint32_t); swif_status_t (*generate_coding_coefs) ( struct swif_encoder*, uint32_t, uint32_t); swif_status_t (*get_coding_coefs_tab) ( struct swif_encoder*, void**, uint32_t*); } swif_encoder_t; /** * Decoder structure that contains whatever is needed for decoding. * The exact content of this structure is FEC code dependent, the * structure below being a non normative example. */ typedef struct swif_decoder { swif_codepoint_t codepoint; Roca (Ed.), et al. Expires May 20, 2020 [Page 8] Internet-Draft Generic API November 2019 /* when a function returns with SWIF_STATUS_ERROR, the errno * variable contains a more detailed error type. This variable * is set by the codec and accessible to the application in * READ ONLY mode. Otherwise its value is undefined. */ swif_errno_t swif_errno; /* pointers to codec specific versions of API functions. */ swif_status_t (*set_callback_functions) ( struct swif_decoder*, void (*) (void*, esi_t), void* (*) (void*, esi_t), void* (*) (void*, void*, esi_t), void*); swif_status_t (*set_parameters) ( struct swif_decoder*, uint32_t, uint32_t, void*); swif_status_t (*get_parameters) ( struct swif_decoder*, uint32_t, uint32_t, void*); swif_status_t (*decode_with_new_source_symbol) ( struct swif_decoder*, void* const, esi_t); swif_status_t (*decode_with_new_repair_symbol) ( struct swif_decoder*, void* const); swif_status_t (*reset_coding_window) (swif_encoder_t*); swif_status_t (*add_source_symbol_to_coding_window) ( struct swif_decoder*, esi_t); swif_status_t (*remove_source_symbol_from_coding_window) ( struct swif_decoder*, esi_t); swif_status_t (*set_coding_coefs_tab) ( struct swif_decoder*, void*, uint32_t); swif_status_t (*generate_coding_coefs) ( struct swif_decoder*, uint32_t, uint32_t); } swif_decoder_t; General definitions. 4.2. Encoder /** * Create and initialize an encoder, providing only key parameters. * * @param codepoint opaque identifier that fully identifies the FEC * code to use. * @param verbosity print information on the codec processing. * 0 is the minimum verbosity, the maximum verbosity * level being implementation specific. * @param symbol_size source and repair symbol size in bytes. Cannot * change during the codec instance lifetime. * @param max_encoding_window_size * @return pointer to a swif_encoder_t structure if okay, or Roca (Ed.), et al. Expires May 20, 2020 [Page 9] Internet-Draft Generic API November 2019 * NULL in case of error. **/ swif_encoder_t* swif_encoder_create ( swif_codepoint_t codepoint, uint32_t verbosity, uint32_t symbol_size, uint32_t max_coding_window_size); /** * Release an encoder and its associated ressources. **/ swif_status_t swif_encoder_release (swif_encoder_t* enc); /** * Set the various callback functions for this encoder. * All the callback functions require an opaque context parameter, that * must be initialized accordingly by the application, since it is * application specific. * * @param enc * @param source_symbol_removed_from_coding_window_callback * (IN) Pointer to the function, within the application, * that needs to be called each time a source symbol is * removed from the left side of the coding window. * This callback is called each time the encoding window * slides to the right and an old source symbol needs to * be removed on the left. The application therefore knows * this source symbol will no longer be used by the codec * and can free the associated buffer if need be. This * function does not return anything. * @param context_4_callback * (IN) Pointer to the application-specific context that * will be passed to the callback function (if any). This * context is not interpreted by this function. * @return */ swif_status_t swif_encoder_set_callback_functions ( swif_encoder_t* enc, void (*source_symbol_removed_from_coding_window_callback) ( void* context, esi_t old_symbol_esi), void* context_4_callback); /** * This function sets one or more FEC codec specific parameters, * using a type/length/value approach for maximum flexibility. Roca (Ed.), et al. Expires May 20, 2020 [Page 10] Internet-Draft Generic API November 2019 * * @param enc * @param type (IN) Type of parameter. * @param length (IN) length of the pointed value. * @param value (IN) Pointer to the value. The exact type of * the object pointed is FEC codec specific. * @return */ swif_status_t swif_encoder_set_parameters ( swif_encoder_t* enc, uint32_t type, uint32_t length, void* value); /** * This function gets one or more FEC codec specific parameters, * using a type/length/value approach for maximum flexibility. * * @param enc * @param type (IN) Type of parameter. * @param length (IN) length of the pointed value. * @param value (IN/OUT) Pointer to the value. The exact type of * the object pointed is FEC codec specific. * This function updates the value object * accordingly. The caller, who knows the FEC codec, * is responsible to allocate the appropriate * object buffer. * @return */ swif_status_t swif_encoder_get_parameters ( swif_encoder_t* enc, uint32_t type, uint32_t length, void* value); /** * List here the FEC codec specific control parameters. */ enum { swif_ENCODER_GET_PARAM_ENCODER_STATISTICS = 1, swif_ENCODER_SET_PARAM_RLC_DENSITY_THRESHOLD }; /** * Create a single repair symbol (i.e. perform an encoding). * Upon return of this function, the application has full control of the * buffer and is in charge of freeing it when appropriate. Roca (Ed.), et al. Expires May 20, 2020 [Page 11] Internet-Draft Generic API November 2019 * * @param new_buf (IN) The pointer to the buffer for the repair * symbol to build can either point to a buffer * allocated by the application and initialized to * zero, or let to NULL meaning that this function * will allocate memory. * @return */ swif_status_t swif_build_repair_symbol ( swif_encoder_t* enc, void* new_buf); /* FIX ME: must be void** to enable returning a pointer to buffer! */ Encoder API proposal /** * Encoder structure that contains whatever is needed for encoding. * The exact content of this structure is FEC code dependent, the * structure below being a non normative example. * However it MUST be aligned with swif_encoder_t (same first items) in * order to be able to cast a pointer to one of the two structures, * depending on the context. */ typedef struct swif_encoder_internal { /* generic part of any control block. MUST be first in structure */ swif_encoder_t gen; /* desired verbosity: 0 is the minimum verbosity, the maximum * level being implementation specific. */ uint32_t verbosity; /* maximum number of source symbols used for any repair symbol */ uint32_t max_coding_window_size; /* exact size (in bytes) of any source or repair symbol */ uint32_t symbol_size; /* add whatever may be needed hereafter... */ } swif_encoder_internal_t; Non normative example of internal structure used by an encoder. Roca (Ed.), et al. Expires May 20, 2020 [Page 12] Internet-Draft Generic API November 2019 4.3. Decoder /** * Create and initialize a decoder, providing only key parameters. * * @param codepoint opaque identifier that fully identifies the FEC * code to use. * @param verbosity print information on the codec processing. * 0 is the minimum verbosity, the maximum verbosity * level being implementation specific. * @param symbol_size source and repair symbol size in bytes. Cannot * change during the codec instance lifetime. * @param max_coding_window_size * @param max_linear_system_size * @return pointer to a swif_decoder_t structure if okay, or * NULL in case of error. **/ swif_decoder_t* swif_decoder_create ( swif_codepoint_t codepoint, uint32_t verbosity, uint32_t symbol_size, uint32_t max_coding_window_size, uint32_t max_linear_system_size); /** * Release a decoder and its associated ressources. * * @param dec context (i.e., pointer to decoder structure). **/ swif_status_t swif_decoder_release (swif_decoder_t* dec); /** * Set the various callback functions for this decoder. * All the callback functions require an opaque context parameter, that * must be initialized accordingly by the application, since it is * application specific. * * @param dec context (i.e., pointer to decoder structure). * @param source_symbol_removed_from_linear_system_callback * (IN) Pointer to the function, within the application, that * needs to be called each time a source symbol is removed from * the left side of the linear system. * This callback is called each time the linear system slides * to the right and an old source symbol needs to be removed * on the left. This function does not return anything. Roca (Ed.), et al. Expires May 20, 2020 [Page 13] Internet-Draft Generic API November 2019 * @param decodable_source_symbol_callback * (IN) Pointer to the function, within the application, that * needs to be called each time a source symbol is decodable. * What it does is application-dependent, but it MUST return * either a pointer to a data buffer, left uninitialized, of * the appropriate size, or NULL if the application prefers to * let the codec allocate the buffer. * In any case the codec is responsible for storing the actual * symbol value within the data buffer. Also, no matter * whether the data buffer is allocated by the application or * the codec, it is the responsibility of the application to * free this buffer when needed, once decoding is over (but * not before since the codec does not keep any internal copy). * @param decoded_source_symbol_callback * (IN) Pointer to the function, within the application, that * needs to be called each time a source symbol is decodable and * all computations performed (i.e., the buffer does contain the * symbol value). * This callback is called in a second time, when the newly * decodable source symbol is actually decoded and ready, * i.e., when all the computations (like XOR and GF(2**8) * operations) have been performed. In any case, it is the * responsibility of the application to free this buffer when * needed, once decoding is over (but not before since the * codec does not keep any internal copy). This function does * not return anything. * @param context_4_callback * (IN) Pointer to the application-specific context that will be * passed to the callback function (if any). This context is not * interpreted by this function. * @return */ swif_status_t swif_decoder_set_callback_functions ( swif_decoder_t* dec, void (*source_symbol_removed_from_linear_system_callback) ( void* context, esi_t old_symbol_esi), void* (*decodable_source_symbol_callback) ( void *context, esi_t esi), void (*decoded_source_symbol_callback) ( void *context, void *new_symbol_buf, esi_t esi), void* context_4_callback); /** Roca (Ed.), et al. Expires May 20, 2020 [Page 14] Internet-Draft Generic API November 2019 * This function sets one or more FEC codec specific parameters, * using a type/length/value approach for maximum flexibility. * * @param dec context (i.e., pointer to decoder structure). * @param type (IN) Type of parameter. * @param length (IN) length of the pointed value. * @param value (IN) Pointer to the value. The exact type of * the object pointed is FEC codec specific. * @return */ swif_status_t swif_decoder_set_parameters ( swif_decoder_t* dec, uint32_t type, uint32_t length, void* value); /** * This function gets one or more FEC codec specific parameters, * using a type/length/value approach for maximum flexibility. * * @param dec context (i.e., pointer to decoder structure). * @param type (IN) Type of parameter. * @param length (IN) length of the pointed value. * @param value (IN/OUT) Pointer to the value. The exact type of * the object pointed is FEC codec specific. * This function updates the value object * accordingly. The caller, who knows the FEC codec, * is responsible to allocate the appropriate * object buffer. * @return */ swif_status_t swif_decoder_get_parameters ( swif_decoder_t* dec, uint32_t type, uint32_t length, void* value); /** * List here the FEC codec specific control parameters. */ enum { swif_DECODER_GET_PARAM_DECODER_STATISTICS = 1, swif_DECODER_SET_PARAM_RLC_DENSITY_THRESHOLD }; /** * Submit a received source symbol and try to progress in the decoding. Roca (Ed.), et al. Expires May 20, 2020 [Page 15] Internet-Draft Generic API November 2019 * For each decoded source symbol (if any), the application is informed * through the dedicated callback functions. * * This function usually returns SWIF_STATUS_OK, regardless of whether * this new symbol enabled the decoding of one or several source symbols, * or SWIF_STATUS_ERROR. It cannot return SWIF_STATUS_FAILURE. * * @param dec context (i.e., pointer to decoder structure). * @param new_symbol_buf * (IN) Pointer to the new source symbol now available (i.e. * a new symbol received by the application, or a decoded * symbol in case of a recursive call if it makes sense). * @param new_symbol_esi * (IN) encoding symbol ID of the new source symbol. * @return */ swif_status_t swif_decoder_decode_with_new_source_symbol ( swif_decoder_t* dec, void* const new_symbol_buf, esi_t new_symbol_esi); /** * Submit a received repair symbol and try to progress in the decoding. * For each decoded source symbol (if any), the application is informed * through the dedicated callback functions. * * This function requires that the application has previously initialized * the coding window and coding coefficients appropriately. The application * keeps a full control of the repair symbol buffer, i.e., the application * is in charge of freeing this buffer as soon as it believes appropriate * (a copy is kept by the codec). This is motivated by the fact that a * repair symbol may be part of a larger buffer (e.g., if there are * several repair symbols per packet, or because of a packet header): only * the application knows when the buffer can be safely freed. * * This function usually returns SWIF_STATUS_OK, regardless of whether * this new symbol enabled the decoding of one or several source symbols, * or SWIF_STATUS_ERROR. It cannot return SWIF_STATUS_FAILURE. * * @param dec context (i.e., pointer to decoder structure). * @param new_symbol_buf * (IN) Pointer to the new repair symbol now available (i.e. * a new symbol received by the application or a decoded * symbol in case of a recursive call if it makes sense). * @return */ swif_status_t swif_decoder_decode_with_new_repair_symbol ( Roca (Ed.), et al. Expires May 20, 2020 [Page 16] Internet-Draft Generic API November 2019 swif_decoder_t* dec, void* const new_symbol_buf); Decoder API proposal /** * Decoder structure that contains whatever is needed for decoding. * The exact content of this structure is FEC code dependent, the * structure below being a non normative example. * However it MUST be aligned with swif_decoder_t (same first items) in * order to be able to cast a pointer to one of the two structures, * depending on the context. */ typedef struct swif_decoder_internal { /* generic part of any control block. MUST be first in structure */ swif_decoder_t gen; /* desired verbosity: 0 is the minimum verbosity, the maximum * level being implementation specific. */ uint32_t verbosity; /* maximum number of source symbols used for any repair symbol */ uint32_t max_coding_window_size; /* max. number of source symbols kepts in current linear system. * If the linear system grows above this limit, old source * symbols in excess are removed and the application callback * called. This value should be larger than the * max_coding_window_size. */ uint32_t max_linear_system_size; /* exact size (in bytes) of any source or repair symbol */ uint32_t symbol_size; /* add whatever may be needed hereafter... */ } swif_decoder_internal_t; Non normative example (RLC) of internal structure used by a decoder. 4.4. Coding Window Functions at an Encoder and Decoder This section gathers functions used to manage the coding window, both at an encoder and at a decoder. At an encoder a sliding (of fixed or elastic size) encoding window is managed. Whenever a repair symbol needs to be created, a linear combination (that is code specific) of source symbols currently in the encoding window is performed. This Roca (Ed.), et al. Expires May 20, 2020 [Page 17] Internet-Draft Generic API November 2019 encoding window is managed with the functions below plus, potentially, internal mechanisms that are code specific. At a decoder, before submitting a new repair symbol to the codec, the application must specify the associated encoding window used at the source. This is done by the reset/add a single or set of symbols/ remove a symbol functions. Once this coding window is ready, as well as the coding coefficient list if applicable, the application calls the decode_with_new_repair_symbol() function. A coding window may be reused for several repair symbols as long as they are all built from the same set of source symbols. In that case resetting the coding window and setting it from scratch would be a waste of time. The coding window must be viewed as a temporary list used solely by the decode_with_new_repair_symbol() function and kept independent from the linear system managed by the codec. /** * This function resets the current coding window. We assume here that * this window is maintained by the FEC codec instance. * Encoder: reset the encoding window for the encoding of future * repair symbols. * Decoder: reset the coding window under preparation associated to * a repair symbol just received. * * @return */ swif_status_t swif_encoder_reset_coding_window (swif_encoder_t* enc); swif_status_t swif_decoder_reset_coding_window (swif_decoder_t* dec); /** * Add this source symbol to the coding window. * Encoder: add a source symbol to the coding window. * Decoder: add a source symbol to the coding window under preparation. * * @param new_src_symbol_buf (encoder only) pointer to a buffer * containing the source symbol. The application MUST NOT * free nor modify this buffer as long as the source symbol * is in the coding window. * @param new_src_symbol_esi ESI of the source symbol to add. * @return */ swif_status_t swif_encoder_add_source_symbol_to_coding_window ( swif_encoder_t* enc, void* new_src_symbol_buf, esi_t new_src_symbol_esi); Roca (Ed.), et al. Expires May 20, 2020 [Page 18] Internet-Draft Generic API November 2019 swif_status_t swif_decoder_add_source_symbol_to_coding_window ( swif_decoder_t* dec, esi_t new_src_symbol_esi); /** * Remove this source symbol from the coding window. * * Encoder: remove a source symbol from the encoding window, e.g. * because the application knows that a source symbol has * been acknowledged by the peer (if applicable). Note that * the left side of the sliding window is automatically * managed by the codec and no action is needed from the * application. If needed a callback is available to inform * the application that a source symbol has been removed). * Decoder: remove a source symbol from the coding window under * preparation. * * @param old_src_symbol_esi ESI of the source symbol to remove from * the coding window. * @return */ swif_status_t swif_encoder_remove_source_symbol_from_coding_window ( swif_encoder_t* enc, esi_t old_src_symbol_esi); swif_status_t swif_decoder_remove_source_symbol_from_coding_window ( swif_decoder_t* dec, esi_t old_src_symbol_esi); Coding Window Functions at an Encoder and Decoder. 4.5. Coding Coefficients Functions at an Encoder and Decoder This section gathers functions used to manage the coding coefficients, both at an encoder and at a decoder. Since different FEC codecs will have different requirements, it is important to keep these functions separate from the build_repair_symbol() and decode_with_new_repair_symbol() functions. Several situations exist: o the application provides the list of coding coefficients to use for the next build_repair_symbol(); o the application provides a key (typically a PRNG seed) that the codec uses to produce the coding coefficients to use for the next build_repair_symbol(); o the choice of the coding coefficients is totally performed by the codec, in an autonomous manner (e.g., the codec includes an Roca (Ed.), et al. Expires May 20, 2020 [Page 19] Internet-Draft Generic API November 2019 algorithm that produces an appropriate seed based on various criteria, or the codec selects a set of coding coefficients based on various criteria). In that case the application needs to retrieve the list of coding coefficients or the key selected by the codec; /** * The following functions enable an encoder (resp. decoder) to * initialize the set of coefficients to be used for encoding * or associated to a received repair symbol. * * Encoder: calling one of them MUST be done before calling * build_repair_symbol(). * Decoder: calling one of them MUST be done before calling * decode_with_new_repair_symbol(). */ /** * Encoder: this function specifies the coding coefficients chosen by * the application if this is the way the codec works. * Decoder: communicate with this function the coding coefficients * associated to a repair symbol and carried in the packet * header. * * @param coding_coefs_tab * (IN) table of coding coefficients associated to each of * the source symbols currently in the encoding window. * The size (number of bits) of each coefficient depends on * the FEC Scheme. The allocation and release of this table * is under the responsibility of the application. * @param nb_coefs_in_tab * (IN) number of entries (i.e., coefficients) in the table. * @return */ swif_status_t swif_encoder_set_coding_coefs_tab ( swif_encoder_t* enc, void* coding_coefs_tab, uint32_t nb_coefs_in_tab); swif_status_t swif_decoder_set_coding_coefs_tab ( swif_decoder_t* dec, void* coding_coefs_tab, uint32_t nb_coefs_in_tab); /** * The coding coefficients may be generated in a deterministic manner, Roca (Ed.), et al. Expires May 20, 2020 [Page 20] Internet-Draft Generic API November 2019 * for instance by a PRNG known by the codec and a seed (perhaps with * other parameters) provided by the application. * The codec may also choose in an autonomous manner these coefficients. * This function is used to trigger this process. * When the choice is made in an autonomous manner, the actual coding * coefficient or key used by the codec can be retrieved with * swif_encoder_get_coding_coefs_tab(). * * @param key (IN) Value that can be used as a seed in case of a PRNG * for instance, or by a specific coding coefficients * function. Set to 0 if not required by a codec. * @param add_param * (IN) an opaque 32-bit integer that contains a codec * specific parameter if needed. Set to 0 if not used. * @return */ swif_status_t swif_encoder_generate_coding_coefs ( swif_encoder_t* enc, uint32_t key, uint32_t add_param); swif_status_t swif_decoder_generate_coding_coefs ( swif_decoder_t* dec, uint32_t key, uint32_t add_param); /** * This function enables the application to retrieve the set of coding * coefficients generated and used by build_repair_symbol(). This is * useful when the choice of coefficients is performed by the codec in * an autonomous manner but needs to be sent in the repair packet header. * This function is only used by an encoder. * * @param coding_coefs_tab * (OUT) pointer to a table of coding coefficients. * The size (number of bits) of each coefficient depends on * the FEC scheme. Upon return of this function, this table * is allocated and filled with coefficient values. The * release of this table is under the responsibility of the * application. * @param nb_coefs_in_tab * (IN/OUT) pointer to the number of entries (i.e., * coefficients) in the table. * Upon calling this function, this number must be zero. * Upon return of this function this variable is initialized * with the actual number of entries in the coeffs_tab[]. * @return Roca (Ed.), et al. Expires May 20, 2020 [Page 21] Internet-Draft Generic API November 2019 */ swif_status_t swif_encoder_get_coding_coefs_tab ( swif_encoder_t* enc, void** coding_coefs_tab, uint32_t* nb_coefs_in_tab); /** * Get information on the current coding window at the encoder. * This function stores the ESI of the first source symbol and * last source symbol in the coding window, as well as the number * of symbols. In theory the application should be able to recover * the information (it knows when new symbols are added and old * symbols removed), but it's easier to let the SWiF Codec care * about it. The number of source symbols is also returned. * In situations where there's no gap (i.e., when * swif_encoder_remove_source_symbol_from_coding_window() has not * been used), nss can also be calculated with first/last. However * it is more convenient to use nss directly (in particular in case * of wraping to zero of either first or last). * * @param enc * @param first (in/out) pointer to ESI of the first source * symbol in the coding window (inclusive) * @param last (in/out) pointer to ESI of the last source * symbol in the coding window (inclusive) * @param nss (in/out) pointer to number of source symbols * in the coding window * @return */ swif_status_t swif_encoder_get_coding_window_information ( swif_encoder_t* enc, esi_t* first, esi_t* last, uint32_t* nss); Coding Coefficients Functions at an Encoder and Decoder. 5. Security Considerations TBD 6. IANA Considerations This document has no IANA requirement. Roca (Ed.), et al. Expires May 20, 2020 [Page 22] Internet-Draft Generic API November 2019 7. Acknowledgments The authors would like to thank Marie-Jose Montpetit, Francois Michel, and Oumaima Attia for their valuable contributions to the IETF Hackathon SWiF-Codec project and their inputs to this document. 8. References 8.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . 8.2. Informative References [fecframe-ext] Roca, V. and A. Begen, "Forward Error Correction (FEC) Framework Extension to Sliding Window Codes", Transport Area Working Group (TSVWG) draft-ietf-tsvwg-fecframe-ext (Work in Progress), June 2018, . [RLC] Roca, V. and B. Teibi, "Sliding Window Random Linear Code (RLC) Forward Erasure Correction (FEC) Scheme for FECFRAME", Transport Area Working Group (TSVWG) draft- ietf-tsvwg-rlc-fec-scheme (Work in Progress), June 2018, . Authors' Addresses Vincent Roca INRIA Univ. Grenoble Alpes France EMail: vincent.roca@inria.fr Jonathan Detchart ISAE - Supaero France EMail: jonathan.detchart@isae-supaero.fr Roca (Ed.), et al. Expires May 20, 2020 [Page 23] Internet-Draft Generic API November 2019 Cedric Adjih INRIA France EMail: cedric.adjih@inria.fr Morten V. Pedersen Steinwurf ApS Denmark EMail: morten@steinwurf.com Roca (Ed.), et al. Expires May 20, 2020 [Page 24]