CDNI Metadata Model Extensions
Lumen TechnologiesUS
glenng1215@gmail.com
Lumen TechnologiesUS
wrpower@gmail.com
BroadpeakFrance
guillaume.bichot@gmail.com
TelefonicaSpain
alfonsosiloniz@gmail.com
Open Caching architecture is a use case of Content Delivery Networks Interconnection (CDNI) in which the
commercial Content Delivery Network (CDN) is the upstream CDN (uCDN) and the ISP caching layer serves as the
downstream CDN (dCDN). This document proposed extensions to the RFC8006 Metadata Model by way of a set of GenericMetadata objects that address extend the original CDNI capabilities to meet the more general needs of the CDN and Open Caching industry. Extensions to RFC8008 are also introduced to allow a dCDN to advertise support for these extended metadata capabilities.
The Streaming Video Alliance is a global association
that works to solve streaming video challenges in an effort to improve end-user experience
and adoption. The Open Caching Working Group of the
Streaming Video Alliance is focused on the delegation
of video delivery requests from commerical CDNs to a caching layer at the ISP's network.
Open Caching architecture is a specific use case of CDNI where the commercial CDN is the
upstream CDN (uCDN) and the ISP caching layer is the downstream CDN (dCDN).
The interchange of content delivery configuration metadata between the various entities
in the delivery ecosystem is essential for efficient interoperability. The need for an
industry-standard API and metadata model becomes increasingly important as content
and service providers automate more of their operations, and as technologies, such as
open caching, require coordination of content delivery configurations. In order to achieve this, the
Open Caching Configuration Interface Specification defines
an interface contemplating a set of use cases.
The following capabilities extend the Metadata Model:
Open Caching Configuration Metadata Enhanced Source definitions, with load balancing, failover, and extended authorization methods A rich set of Cache Control Policies and computed cache keys Rules for generating Dynamic CORS Headers Possibility to activate compression in the Edge independent of the origin content Traffic Types ServiceID Metadata Processing Stage Rules , enabling metadata to be applied conditionally at various stages in the CDN request/response pipeline.Request URI RewritesHTTP Header Modifications HTTP Status Modifications Synthetic HTTP Responses An Expression Language for matching rules and synthesis of dynamic values Private Features
For consistency with other CDNI documents this document follows the
CDNI convention of uCDN (upstream CDN) and dCDN (downstream CDN) to represent the commercial CDN and ISP caching
layer respectively.
This document defines and registers CDNI GenericMetadata objects (as defined in section 4 of ), registers additional CDNI Payload Types (section 7.1 of
), and adds capability objects (extending section 5 in )
The following terms are used throughout this document:
API - Application Programming Interface
AWS - Amazon Web Services
CDN - Content Delivery Network
CDNi - CDN Interconnect
CORS - Cross-Origin Resource Sharing
CP - Content Provider
dCDN - Downstream CDN
DNS - Domain Name System
FCI - Footprint and Capabilities Advertising Interface
HREF - Hypertext Reference (link)
HTTP - Hypertext Transfer Protocol
IETF - Internet Engineering Task Force
ISP - Internet Service Provider
JSON - JavaScript Object Notation
MEL - Metadata Expression Language
Object - A collection of properties.
OC - Open Caching
OCN - Open Caching Node
PatternMatch - An object which matches a string pattern
UA - User Agent
uCDN - Upstream CDN
URI - Uniform Resource Identifier
URN - Uniform Resource Name
VOD - Video-on-Demand
W3C - World Wide Web Consortium
Additionally, this document reuses the terminology defined in
,
,
,
,
, and
.
Specifically, we use the following CDNI acronyms:
uCDN, dCDN - Upstream CDN and Downstream CDN respectively (see
)
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14
when, and only when, they appear in all capitals, as shown here.
Section 4 of defines a set of GenericMetadata Object Types.
Below additional GenericMetadata Objects are defined to meet the more general needs of the CDNs and Open Caching industry.
Note: In the following sections, the term "mandatory-to-specify" is
used to convey which properties MUST be included when serializing a
given capability object. When mandatory-to-specify is defined as
"Yes" for an individual property, it means that if the object
containing that property is included in a message, then the
mandatory-to-specify property MUST also be included.
To meet the CDN industry requirements for origin authentication, two new authentication types are proposed to be registered via the CDNi Media Type Registration process, as described in .
auth-type: headerauth
Description: Header based authentication is used to pass an HTTP header secret-name and value secret-value to a uCDN when requesting content. The header name and value are agreed upon between parties out of band.auth-value: HeaderAuth object as defined in auth-type: awsv4auth
Description: Allows for the specification of a set of headers to be added to requests that are forwarded to an origin to enable Amazon Web Services AWS authentication as documented by AWS See Specifications Standards References.auth-value: AWSv4Auth object specifying the access parameters as defined in
The HeaderAuth metadata object is used in the auth-value property of the
Auth object, as defined in section 4.2.7 of , and may be applied to any
source by including or referencing it under its authentication property. This method of authentication provides a simple capability for a mutually agreed upon header to be added by the CDN to all requests sent to a specific origin. Note that if a dynamically generated header value is required, the RequestTransform capabilities within StageProcessing (See ) can be used.
Property: header-name
Description: Name of the authentication header.Type: StringMandatory-to-Specify: YesProperty: header-value
Description: Value of the authentication header typically a pre-shared key. Note that this value should not be disclosed it should be protected by some mechanism such as a secret-sharing API which is outside the scope of this specification.Type: StringMandatory-to-Specify: Yes
The AWSv4Auth metadata object is used in the auth-value property of the
Auth object as defined in RFC-8006 section 4.2.7, and may be applied to any
source by including or referencing it under its authentication property.
AWSv4 authentication causes upstream requests to have a signature applied,
following the method described in
. A hash-based signature is calculated over the request URI and specified
headers, and provided in an Authorization: header on the upstream request.
The signature is tied to a pre-shared secret key specific to an AWS
service, region, and key ID.
Property: access-key-id
Description: The preconfigured ID of the pre-shared authorization secret.Type: StringMandatory-to-Specify: YesProperty: secret-access-key
Description: The pre-shared authorization secret which is the basis of building the signature. This is a secret key that should not be disclosed it should be protected by some mechanism such as a secret-sharing API which is outside the scope of this specification.Type: StringMandatory-to-Specify: YesProperty: aws-region
Description: The AWS region name that is hosting the service and shares the key ID and corresponding pre-shared secret.Type: StringMandatory-to-Specify: YesProperty: aws-service
Description: The AWS service name that is serving the upstream requests.Type: StringMandatory-to-Specify: No. It defaults to s3 if not specified.Property: host-name
Description: The host name to use as part of the signature calculation.Type: StringMandatory-to-Specify: No. It defaults to using the value of the Host: header of the upstream request. This property is available in case the application needs to override that behavior.
Downstream CDNs often have the ability to compress HTTP response bodies in cases where the client has declared that it can accept compressed responses (via an Accept-Encoding header), but the source/origin has returned an uncompressed response
The specific compression algorithm used by the dCDN is negotiated by the client's Accept-Encoding header according to (including q= preferences) and the compression capabilities available on the dCDN.
MI.AllowCompress is a new GenericMetadata object that allows the uCDN to control the activation of response compression in the dCDN directly, and allows content providers to disable compression in cases where compressed responses are not handled properly by certain streaming devices. This can be achieved using a match expression on the user agent.
In addition, HeaderTransform allows the uCDN to normalize, or modify, the Accept-Encoding header to allow for fine-grain control over the selection of the compression algorithm (e.g., gzip, compress, deflate, br, etc.).
Properties of AllowCompress object are:
Property: allow-compress
Description: If set to True then the dCDN will try to compress the response to the client based on the Accept-Encoding request header.Type: BooleanMandatory-to-Specify: No. The default is False.
MI.CachePolicy is a new GenericMetadata object that allows for the uCDN to specify internal caching policies for the dCDN and external caching policies advertised to clients of the dCDN. overriding any cache control policy set in the response from the uCDN
Property: internal
Description: Specifies the internal cache control policy to be used by the dCDN. Type: Number in seconds encoded as string (e.g. 5 is a five second cache ) and/or a list of Enumeration [as-is|no-cache|no-store]Mandatory-to-Specify: No. The default is to use the cache control policy specified in the response from the uCDN.Property: external
Description: Specifies the external cache control policy to be used by clients of this dCDN.Type: Number in seconds encoded as string (e.g. 5 is a five second cache ) and/or a list of Enumeration [as-is|no-cache|no-store]Mandatory-to-Specify: No. The default is to use the cache control policy specified in the response from the uCDN.Property: force
Description: If set to True the metadata interface cache policy defined in the MI.CachePolicy will override any cache control policy set in the response from the uCDN. If set to False the MI.CachePolicy is only used if there is no cache control policy provided in the response from the uCDN.Type: BooleanMandatory-to-Specify: No. The default is False which will apply the MI.CachePolicy only if no policy is provided in the response from the uCDN.
While the properties provided by the standard CDNi metadata Cache object (See Section 4.2.6 ) provide some simple control over the construction of the cache key, it is typical in advanced CDN configurations to generate cache keys that are dynamically constructed via lightweight processing of various properties of the HTTP request and/or response. As an example, an origin may specify a cache key as a value returned in a specific HTTP response header.
MI.ComputedCacheKey is a new GenericMetadata object that allows for the specification of a cache key using the metadata expression language. See . Typical use cases would involve the construction of a cache key from one or more elements of the HTTP request. In cases where both the ComputedCacheKey and the Cache object are applied, the ComputedCacheKey will take precedence.
Property: expression
Description: The expression that specifies how the cache key shall be constructed.Type: String. An expression using CDNI-MEL to dynamically construct the cache key from elements of the HTTP request and/or response.Mandatory-to-Specify: Yes
Delegation of traffic between an uCDN over a dCDN based on HTTP redirection does change the domain name in the client requests. This represents a cross-origin request that must be managed appropriately using Cross-Origin Resource Sharing (CORS) headers in the responses in the dCDN.
The dynamic generation of CORS headers is typical in modern HTTP request processing and avoids CORS validation forwarded to the uCDN origin servers, particularly with the preflight OPTIONS requests. The CDNI metadata model requires extensions to specify how a dCDN should generate and evaluate these headers.
Simple CORS requests are those where both HTTP method and headers in the request are included in the safe list defined by the World Wide Web Consortium . The user agent request can include an origin header set to the URL domain of the webpage where a player runs. Depending on the metadata configuration, the logic to apply by the dCDN is:
Validation of the origin headerWildcard usageSet a default value for CORS response headersWhen a UA makes a request that includes a method or headers that are not included in the safe-list, the client will make a CORS preflight request using the OPTIONS method to the resource including the origin header. If CORS is enabled and the requests passes the origin validation, the OCN should respond with the set of headers that indicate what is permitted for that resource, including one or more of the following:
Allowed methods
Allowed credentials
Allowed request headers
max-age that the OPTIONS request is valid
Headers that can be exposed to the client
CrossoriginPolicy allows for the specification of dynamically generated CORS headers.
Property: allow-origin
Description: Validation of simple CORS requests.Type: ObjectValues: One element for each of the following properties.Mandatory-to-Specify: Yes
The AllowOrigin object has the following properties:
Property: allow-list
Description: List of valid URLs only scheme and host name that will be used to match the request origin header.Type: Array of PatternMatch objects (Section 4.1.5 of )Mandatory-to-Specify: YesProperty: wildcard-return
Description: If True the dCDN will include a wildcard in the Access-Control-Allow-Origin response header. If False the dCDN will reflect the request origin header in the Access-Control-Allow-Origin response header.Type: BooleanMandatory-to-Specify: YesProperty: expose-headers
Description: A list of values the dCDN will include in the Access-Control-Expose-Headers response header to a preflight request.Type: Array of stringsMandatory-to-Specify: NoProperty: allow-methods
Description: A list of values the dCDN will include in the Access-Control-Allow-Methods response header to a preflight request.Type: Array of stringsMandatory-to-Specify: NoProperty: allow-headers:
Description: A list of values the dCDN will include in the Access-Control-Allow-Headers response header to a preflight request.Type: Array of stringsMandatory-to-Specify: NoProperty: allow-credentials
Description: The value the dCDN will include in the Access-Control-Allow-Credentials response header to a preflight request.Type: BooleanMandatory-to-Specify: NoProperty: max-age
Description: The value the dCDN will include in the Access-Control-Max-Age response header to a preflight request.Type: IntegerMandatory-to-Specify: No
MI.NegativeCachePolicy is a new GenericMetadata object that allows for the specification of caching policies based on error response codes received from the origin, allowing for fine-grained control of the downstream caching of error responses. For example, it may be desirable to cache error responses at the uCDN for a short period of time to prevent an overwhelmed origin service from being flooded with requests
Property: error-codes
Description: Array of HTTP response error status codes (See Sextions 6.5 and 6.6 of ) that if returned from the uCDN will be cached using the cache policy defined by the cache-policy property.Type: Array of HTTP response error status codesMandatory-to-Specify: No. The default is to revert to behavior.Property: cache-policy
Description: MI.CachePolicy to apply to the HTTP response error status codes returned by the uCDN.Mandatory-to-Specify: Yes
MI.CacheBypassPolicy is a new GenericMetadata object that allows a client request to be set as non cacheable. It is expected that this feature will be used to allow clients to bypass cache when testing the uCDN fill path. Note, CacheBypassPolicy only applies to the current request. In addition, any content previously cached (by client requests that do not set CacheBypassPolicy) is not evicted.
Property: error-codes
Description: Array of HTTP response error status codes (See Sextions 6.5 and 6.6 of ) that if returned from the uCDN will be cached using the cache policy defined by the cache-policy property.Type: Array of HTTP response error status codesMandatory-to-Specify: No. The default is to revert to behavior.Property: cache-policy
Description: MI.CachePolicy to apply to the HTTP response error status codes returned by the uCDN.Mandatory-to-Specify: Yes
MI.OcnSelection is a new GenericMetadata object that allows the uCDN to indicate to the dCDN a preference in terms of OCN selection.
Property: ocn-delivery
Description: Instructs the dCDN to perform delegation operating a particular medium and/or a transport arrangement.Type: An OcnDeliver yObject as defined in
An OcnDelivery object contains the following properties:
Property: ocn-medium
Description: Instructs the dCDN to perform delegation operating a particular medium. The following values are specified: SATELLITE.Type: StringMandatory-to-Specify: No. Either the ocn-medium property or the ocn-transport property must be present.Property: ocn-transport
Description: Instructs the dCDN to perform delegation operating a particular transport arrangement. The following values are specified: MABR.Type: StringMandatory-to-Specify: No. Either the ocn-medium property or the ocn-transport property must be present.Mandatory-to-Specify: No. At least one of the two properties ocn-type or ocn-delivery must be present.Property: ocn-type
Description: Instructs the dCDN to perform delegation operating the type of open caching nodes.Type: A string corresponding to one of the open caching node types announced by the dCDN through the FCI interface.Mandatory-to-Specify: No. At least one of the two properties ocn-type or ocn-delivery must be present.Property: ocn-selection
Description: This property enforces the selection of OCNs considering the ocn-type and/or the ocn-delivery properties. False means best-effort.Type: string. attempt-or-failed and attempt-or-besteffort mean that the delegation must be attempted considering the ocn-type and/or the ocn-delivery properties. If not possible it is considered as an error and either fails configuration failure or the dCDN continues with a best-effort procedure. Last best effort means the dCDN tries its best to fulfill the requested ocn-selection policy.Mandatory-to-Specify: No. Best-effort is the default OCN selection policy.
MI.PrivateFeatureList is a new GenericMetadata configuration object as a base generic object that permits the control of private features. Note that the private features exposed by the dCDN can be advertised through a dedicated FCI object.
Property: features
Description: The list of feature configuration objects.Type: List array of MI.PrivateFeature objects .Mandatory-to-Specify: Yes
A MI.PrivateFeature object contains the following properties:
Property: feature-oid
Description: The owner organization that has specified that feature.Type: StringMandatory-to-Specify: YesProperty: feature-type
Description: Indicates the typename of the private feature configuration object.Type: StringMandatory-to-Specify: YesProperty: feature-value
Description: Feature configuration object.Type: Format type is defined by the value of the feature-type property above.Mandatory-to-Specify: Yes
A ProcessingStages object is a type of GenericMetadata that describes the matching rules, metadata, and transformations to be applied at specific stages in the request processing pipeline.
It is typical in CDN configurations to define matching rules and metadata that are to be applied at specific stages in the request processing pipeline. For example, it may be required to append a host header prior to forwarding a request to an origin, or modify the response returned from an origin prior to storing in the cache. The following four processing stages are defined:
clientRequest - Rules run on the client request prior to further processing.originRequest - Rules run prior to making a request to the origin.originResponse - Rules run after a response is received from the origin and before being placed in the cache.clientResponse - Rules run prior to sending the response to the client. If the response is from the cache, rules are applied to the response retrieved from the cache prior to sending to the client.
Each of the four processing stages is represented by an array of StageRules objects, with each StageRules object defining match criteria along with metadata that should be applied if the match applies to true. It should be noted that all of the StageRules objects in the array are evaluated and processed in order. A possible future extension to this processing model could allow for an if-else structure, where processing for a stage is halted upon matching of a StageRule match expression.
Each of the four processing stages is represented by an array of StageRules objects, with each StageRules object defining criteria along with metadata that should be applied if the match applies to True. It should be noted that the StageRules objects in the array are evaluated and processed in order. A possible future extension to this processing model could allow for an if-else structure, where processing for a stage is halted upon the first match of a StageRules expression.
Property: client-request
Description: Allows for the specification of conditional metadata to be applied at the client request processing stages as defined in the Rule Processing Stages section. The StageRules in the array are evaluated in order.Type: Array of StageRules objectsMandatory-to-Specify: NoProperty: origin-request
Description: Allows for the specification of conditional metadata to be applied at origin request processing stages as defined in the Rule Processing Stages section. The StageRules in the array are evaluated in order.Type: Array of StageRules objectsMandatory-to-Specify: NoProperty: origin-response
Description: Allows for the specification of conditional metadata to be applied at origin response processing stages as defined in the Rule Processing Stages section. The StageRules in the array are evaluated in order.Type: Array of StageRules objectsMandatory-to-Specify: NoProperty: client-response
Description: Allows for the specification of conditional metadata to be applied at client response processing stages as defined in the Rule Processing Stages section. The StageRules in the array are evaluated in order.Type: Array of StageRules objectsMandatory-to-Specify: No
A StageRules object is used within the context of ProcessingStages to define elements in a list of match rules and stage-specific metadata and transformations that should be applied conditionally on a rich expression match.
Property: match
Description: An ExpressionMatch object encapsulating a rich expression using the CDNi Metadata Expression Language CDNI-MEL to evaluate aspects of the HTTP request and/or response. The stage-metadata rules are only applied if the match evaluates to True or if no match expression is providedType: ExpressionMatch objectMandatory-to-Specify: No. The stage-metadata rules are always applied if no match expression is provided. This would be the case when stage-metadata should be applied unconditionally within the context of the higher-level host and path matches.Property: stage-metadata
Description: Specifies the set of StageMetadata to be applied at the processing stage if the match expression evaluates to True or is not present.Type: Array of StageMetadata objects applied in order.Mandatory-to-Specify: Yes
CDN and open caching systems often require a rich set of matching rules, with full regular expressions and Boolean combinations of matching parameters for host, path, and header elements of a request. In typical CDN implementations, this capability is provided by a rich expression language that can be embedded in the metadata configurations
The ExpressionMatch object contains the rich expression that must evaluate to True for the StageMetadata to be applied for the specific StageRules. Defining expressions as stand-alone objects allows for sets of reusable match expressions to be reused via metadata reference linking.
Property: expression
Description: A rich expression using CDNI-MEL to evaluate aspects of the HTTP request and/or response. See documentation on the Metadata Expression Language for details on the expression of matching variables and syntax.Type: String using CDNI-MEL syntax. See Mandatory-to-Specify: Yes
The StageMetadata object contains GenericMetadata and HTTP request/response transformations that should be applied for a StageRules match. The following table defines the processing stages where request and response transformations are possible:
Stagerequest-transformresponse-transformclientRequestYesYesoriginRequestYesYesoriginResponseYesNoclientResponseYesNo
Note that for the stages where both request and response transformations are allowed, it is possible to specify both. This may be the case if, for example, the request URI needs alteration for cache-key generation and the response headers need to be manipulated.
Property: generic-metadata
Description: Specifies the set of GenericMetadata to be applied for a StageRules match. A typical use case would be the application of a CachePolicy or TimeWindowACL conditionally on matching HTTP headers. Support for this capability is optional and can be advertised via feature-flags in the FCI interface.Type: Array of GenericMetadata applied in order. Note that not all GenericMetadata object types may be applicable at all processing stages.Mandatory-to-Specify: No. The generic-metadata property would not be needed when StageMetadata is used to only specify request or response transformations such as modifications of HTTP headers.Property: request-transform
Description: Specifies a transformation to be applied to the HTTP request for a StageRules match. The transformation can be the modification of any request header and/or the modification of the URI. Modifications are applied such that downstream processing stages receive the modified HTTP request as their input. Support for this capability is optional and can be advertised via feature-flags in the FCI interface.Type: RequestTransform objectMandatory-to-Specify: NoProperty: response-transform
Description: Specifies a transformation to be applied to the HTTP response for a StageRules match. The transformation can be the modification of any response header HTTP response status code or the generation of a synthetic response. Modifications are applied such that downstream processing stages receive the modified HTTP response as their input. Support for this capability is optional and can be advertised via feature-flags in the FCI interface.Type: ResponseTransform objectMandatory-to-Specify: No
The RequestTransform object contains metadata for transforming the HTTP request for a specific StageRules object. The transformation can be the modification of any request header and/or the modification of the URI. Modifications are applied such that downstream processing stages receive the modified HTTP request as their input.
Property: headers
Description: A HeaderTransform object specifying HTTP request headers to add replace or delete.Type: HeaderTransform objectMandatory-to-Specify: NoProperty: uri
Description: Replacement value for the HTTP request.Type: String. Either a literal static string or an expression using CDNI-MEL to dynamically construct a URI value from elements of the HTTP request and/or response.Mandatory-to-Specify: NoProperty: uri-is-expression
Description: Flag to signal whether the URI is a static string literal or a CDNI-MEL expression that needs to be dynamically evaluated.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the URI is a string literal and does not need to be evaluated.
The ResponseTransform object contains metadata for transforming the HTTP response for a StageRules match. The transformation can be the modification of any response header, HTTP response status code, or the generation of a synthetic response. Modifications are applied such that downstream processing stages receive the modified HTTP response as their input.
Property: headers
Description: A HeaderTransform object specifying HTTP response headers to add replace or delete.Type: HeaderTransform objectMandatory-to-Specify: NoProperty: response-status
Description: Replacement value for the HTTP response status code.Type: Integer. Either a static integer or an expression using CDNI-MEL that evaluates to an integer to dynamically generate an HTTP status code based on elements of the HTTP request and/or response. Expressions that do not evaluate to an integer shall be considered invalid and result in no override of origin-provided response status.Mandatory-to-Specify: NoProperty: status-is-expression
Description: Flag to signal whether the response-status is a static integer or a CDNI-MEL expression that needs to be dynamically evaluated to generate an HTTP response status code.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the response-status is a static integer and does not need to be evaluated.Property: synthetic
Description: Specification of a complete replacement of any HTTP response that may have been generated in an earlier processing stage with a synthetic response. Use of this property to specify a synthetic response would override any response transformations or status codes specified by other properties.Type: SyntheticResponse objectMandatory-to-Specify: No
It is quite common in CDN configurations to specify a synthetic response be generated based on inspection of aspects of the original request or the origin response.
The SyntheticResponse object allows for the specification of a synthetic response to be generated in response to the HTTP request being processed. The synthetic response can contain a set of response headers, a status code, and a response body, and is a complete replacement for any HTTP response elements generated in an earlier processing stage.
A dynamically generated Content-Length HTTP response header is generated based on the length of the generated response body.
Property: headers
Description: An array of HTTP header objects that specify the full set of headers to be applied to the synthetic response.Type: Array of HTTP header objectsMandatory-to-Specify: No although it would be unusual to not specify minimal standard response headers such as Content-Type.Property: response-status
Description: HTTP response status code.Type: Integer. Either a static integer or an expression using CDNI-MEL that evaluates to an integer to dynamically generate an HTTP status code based on elements of the upstream HTTP request and/or response. Expressions that do not evaluate to an integer shall be considered invalid and result in a 500 status for the synthetic response.Mandatory-to-Specify: YesProperty: status-is-expression
Description: Flag to signal whether the response-status is a static integer or a CDNI-MEL expression that needs to be dynamically evaluated to generate an HTTP response status code.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the response-status is a static integer and does not need to be evaluated.Property: body
Description: Body for the synthetic HTTP response. The response body can either be static or dynamically constructed from a rich expression.Type: String. Either a literal static string or an expression using CDNI-MEL to dynamically construct a response body from elements of the HTTP request and/or response.Mandatory-to-Specify: No. If absent an empty HTTP response with a zero-value Content-Length header is generated.Property: body-is-expression
Description: Flag to signal whether the synthetic response body is a static string literal or a CDNI-MEL expression that needs to be dynamically evaluated.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the body is a string literal and does not need to be evaluated.
In processing HTTP requests, it is often required to modify HTTP request or response headers at one of the processing stages, requiring CDNi metadata to have the capability to update any field in an HTTP request or response header. It should be noted that certain HTTP headers (such as Set-Cookie) have multiple occurrences in a request or response, thereby requiring that we allow for add and replace designations for header modification.
The HeaderTransform object specifies how HTTP headers should be added, replaced, or deleted from HTTP requests and responses.
Property: add
Description: List of HTTP headers name/value pairs that should be added to the HTTP request or response. Note that any existing headers in the request or response with the same names of those added are not affected resulting in multiple headers with the same name.Type: Array of HTTPHeader objects containing header name/value pairsMandatory-to-Specify: NoProperty: replace
Description: List of HTTP headers name/value pairs that should be added to the HTTP request or response replacing any previous headers with the same name.Type: Array of HTTPHeader objects containing header name/value pairsMandatory-to-Specify: NoProperty: delete
Description: List of names of HTTP headers that should be deleted from theHTTP request or response. If a named header appears multiple times all occurrences are deleted.Type: Array of strings with each string naming an HTTP header to deleteMandatory-to-Specify: No
The HTTPHeader object contains a name/value pair for an HTTP header to add or replace in a request or response. The CDNI-MEL expression language can be used to dynamically generate response values.
Property: name
Description: Name of the HTTP header.Type: StringMandatory-to-Specify: YesProperty: value
Description: New value of the named HTTP header.Type: String. Either a static string or an expression using CDNI-MEL to dynamically construct a header value from elements of the HTTP request and/or response.Mandatory-to-Specify: YesProperty: value-is-expression
Description: Flag to signal whether the value is a static string literal or a CDNI-MEL expression that needs to be dynamically evaluated.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the value is a string literal and does not need to be evaluated.
MI.RequestedCapacityLimits is a new GenericMetadata object that allows the uCDN to communicate to the dCDN a desired change in the advertised traffic delegation limits for a given host and footprint.
Property: requested-limits
Description: A set of requested changes to the dCDNs advertised FCI.CapacityLimits .Type: Array of RequestedCapacityLimit objectsMandatory-to-Specify: Yes
RequestedCapacityLimit objects contain the following properties:
Property: limit-type
Description: The limit type for which the change is requested corresponding to the limit types of FCI.CapacityLimits .Type: String. One of egress requests storage-size storage-objects sessions or cache-sizeMandatory-to-Specify: YesProperty: limit-value
Description: The requested new value for the limit. Units of this value are dependent upon the limit-type and are defined in the SVA Capacity Insights Interface Specification.Type: IntegerMandatory-to-Specify: YesProperty: footprints
Description: dCDN footprints with advertised FCI.CapacityLimits for which this request applies.Type: List of CDNI footprint objects from the CDNI Metadata Footprint Types registry of Mandatory-to-Specify: Yes
MI.RequestRouting is a new GenericMetadata object that allows the uCDN to force the dCDN request routing mode(s) to be applied when working in iterative redirection mode. The list of redirection modes supported by the dCDN is advertised through the FCI.RedirectionMode object. See Section 5.5 of . The list of request routing modes supported by the dCDN is advertised through the FCI.RequestRouting objectProperty: request-routing-modes
Description: Instructs the dCDN to perform request routing according to one or more preferred modes among those supported and advertised by the dCDN through the FCI.RequestRouting object. One must understand that forcing instead of letting the dCDN request router select one particular request routing mode may trigger some inefficiency in the request routing process.Type: List array of iterative request routing modes, defined as Enumeration of [DNS|HTTP|MANIFESTREWRITE] encoded as an uppercase string Mandatory-to-Specify: No. By default all request routing modes supported by the dCDN can be used by the dCDN as part of its request routing process.
CDN configurations typically have multiple layers of identifiers that group configurations by customer account to facilitate logging, billing, and support operations. The metadata model sis extended to allow for the association service identifier metadata to a host or path match and to allow for these IDs to be dynamically generated via an expression language. For example, it may be necessary to extract a portion of the Request URI path to derive a service identifier (e.g.: /news/* maps to one serviceID and /movies/* maps to a different serviceID)
A MI.ServiceIDs is a new GenericMetadata object that allows for the specification of two tiers of CDN-specific identifiers and names. The interpretation of these identifiers is implementation specific.
Property: service-id
Description: A provider-specific identifier for the service typically a customer account identifier.Type: String. Either a literal static string or an expression using CDNI-MEL to dynamically construct the ID from elements of the HTTP request and/or response.Mandatory-to-Specify: NoProperty: service-id-is-expression
Description: Flag to signal whether the service-id is a static string literal or a CDNI-MEL expression that needs to be dynamically evaluated.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the service-id is a string literal and does not need to be evaluated.Property: service-name
Description: Human-readable name for the service-id.Type: StringMandatory-to-Specify: NoProperty: property-id
Description: A provider-specific identifier for the property typically identifies a child configuration within the parent service-id.Type: String. Either a literal static string or an expression using CDNI-MEL to dynamically construct the ID from elements of the HTTP request and/or response.Mandatory-to-Specify: NoProperty: property-id-is-expression
Description: Flag to signal whether the property-id is a static string literal or a CDNI-MEL expression that needs to be dynamically evaluated.Type: BooleanMandatory-to-Specify: No. The default is False indicating that the property-id is a string literal and does not need to be evaluated.Property: property-name
Description: Human-readable name for the property-id.Type: StringMandatory-to-Specify: No
MI.SourceMetadataExtended is an alternative to the CDNi standard MI.SourceMetadata object (See Section 4.2.1 of ), which adds a property to specify load balancing across multiple sources, as well as a SourceExtended sub-object with additional attributes to the CDNi standard Source object (See Section 4.2.1.1 of ). While both SourceMetadataExtended and SourceMetadata can be provided for backward compatibility, a dCDN that advertises capability for SourceMetadataExtended will ignore SourceMetadata if both are provided for a given host or path match.
Property: sources
Description: Sources from which the dCDN can acquire content listed in order of preference.Type: Array of SourceExtended objectsMandatory-to-Specify: No. Default is to use static configuration out-of-band from the CDNI metadata interface.Property: load-balance
Description: Specifies load balancing rules for the set of sources.Type: LoadBalanceMetadata objectMandatory-to-Specify: No
SourceExtended is an alternative to the CDNi standard Source (Section 4.2.1.1 of ) object with additional metadata. Open Caching use cases requires additional capabilities not covered in as:
Web root path specification for the source.
Support for additional forms of origin authentication
Validate a host header in the request different than the defined in endpoints
Whether a Source response of HTTP redirect is passed through to the user agent or followed by the dCDN to acquire a content
It inherits all the attributes of the Source object (acquistion-auth, endpoints, and protocol), with the following additions:
Property: origin-host
Description: HTTP host header to pass to the endpoints when retrieving content from a uCDN. The host MUST conform to the Domain Name System DNS syntax defined in and .Type: StringMandatory-to-Specify: No. The default is to use the host name passed by the dCDN.Property: webroot
Description: The path element that is prepended to a resources URI beforeretrieving content from a uCDN.Type: StringMandatory-to-Specify: No. The default is to use the original URI.Property: follow-redirects
Description: If the follow-redirects property is set to True HTTP redirect responses returned from a uCDN will be followed when retrieving content. Otherwise the HTTP redirect response is returned to the client.Type: BooleanMandatory-to-Specify: No. The default is True i.e. follow redirect responses from the uCDN.Property: timeout-ms
Description: A timeout in milliseconds to apply when connecting to a uCDN. If the connection is not established within timeout-ms this source is abandoned and the next source in the MI.SourceMetadataExtended sources array is tried. Once a connection is established timeout-ms is used on subsequent reads of data from the uCDN.Type: IntegerMandatory-to-Specify: No. The default is to revert to RFC-8006 behavior.Property: failover-errors
Description: Array of HTTP response error status codes (See Sections 6.5 and 6.6 of ) that if returned from the uCDN will trigger a failover to the next source in the MI.SourceMetadataExtended sources array. If the uCDN returns an HTTP error code that is not in the failover-errors array that error code is returned to the client of the dCDN.Type: Array of HTTP response error status codes.Mandatory-to-Specify: No. The default is to revert to RFC-8006 behavior.
The LoadBalanceMetadata object defines how content acquisition requests are distributed over the SourceExtended objects listed in the SourceMetadataExtended object.
It permits the following capabilities:
Multi-origin failover: ability to specify a list of origins that can act as fallbacks to the primary origin. Failure rules can specify types of errors and timeout values that trigger failover
Multi-origin load balancing: The ability to specify a list of origins that can be selected by one of several balancing rules (round robin, content hash, IP hash).
Origin Shielding Definitions - The ability to designate CDN origin shield locations associated with a specific origin
Parameters for LoadBalanceMetadata are:
Property: balance-algorithm
Description: Specifies the algorithm to be used when distributing content acquisition requests over the sources in a SourceMetadataExtended object. The available algorithms are random content-hash and ip-hash
random: Requests are distributed over the sources in proportion to their associated weights.content-hash: Requests are distributed over the sources in a consistent fashion based on the balance-path-pattern property.ip-hash: Requests are distributed over the sources in a consistent fashion based on the IP address of the client requestor.Type: Enumeration [random|content-hash|ip-hash] encoded as a lowercase string.Mandatory-to-Specify: No. The default is to use sources in preference order as defined in the SourceMetadataExtended object.Property: balance-weights
Description: This property specifies the relative frequency that a source is used when distributing requests. For example if there are three SourceExtended objects in a SourceMetadataExtended object with balance-weights 1 2 1 source 1 will receive 14 of the requests source 2 will receive 24 of the requests source 3 will receive 14 of the requests.Type: Array of integersMandatory-to-Specify: No. The default is to use sources in preference order as defined in the SourceMetadataExtended object.Property: balance-path-pattern
Description: This property specifies a regular expression pattern to apply to the URI when calculating the content hash used by the balance-algorithm. For example balance-path-pattern: prod...ts would distribute requests based on the URI but excluding the prod prefix and the .ts segment file.Type: String regular expressionMandatory-to-Specify: No. The default is to use the original URI.
MI.StaleContentCachePolicy is a new GenericMetadata object that allows the uCDN to specify the policy how the dCDN should process requests for stale content. For example, this policy allows the content provider to specify that stale content be served from cache for a specified time period while refreshes from the origin occur asynchronously.
Property: stale-while-revalidating
Description: Instructs the dCDN to serve a stale version of a resource while refreshing the resource with the uCDN. When set to True the dCDN will return a previously cached version of a resource while the resource is refreshed with the uCDN in the background.Type: BooleanMandatory-to-Specify: No. The default is False which waits for the uCDN to refresh a resource before responding to the client.Property: stale-if-error
Description: Instructs the dCDN to serve a stale version of a resource if an error was received when trying to refresh the resource with the uCDN. When set the dCDN will return a previously cached version of a resource instead of caching the error response. Per Section 4 an error is any situation that would result in a 500 502 503 or 504 HTTP response status code being returnedType: Array of HTTP response error status codesMandatory-to-Specify: No. The default is to cache the error response received from the uCDN.Property: failed-refresh-ttl
Description: Instructs the dCDN to serve a stale version of a resource for the number of seconds specified in failed-refresh-ttl before trying to revalidate the resource with the uCDN. Use of failed-refresh-ttl allows the load to be reduced on the uCDN during times of system stress.Type: IntegerMandatory-to-Specify: No
Content delivery networks often apply different infrastructure, network routes, and internal metadata for different types of traffic. Knowing those differences, a dCDN provider can implement specific strategies that will maximize performance and thereby provide more available capacity to the upstream provider. it should be noted that the dCDNs handling of the traffic types is implementation-specific and not prescribed here.
TrafficType metadata defines a set of descriptors that characterize either the type or usage of the traffic, enabling dCDNs to apply any internal configuration rules without exposing an unnecessary amount of internal details.
Property: traffic-type
Description: A literal that defines the traffic type. uCDN will use the literal that is most representative of the traffic being delegated. The following literals can be specified:
vodliveobject-downloadType: StringMandatory-to-Specify: YesProperty: hints
Description: Other traffic characteristics that the uCDN can indicate to the dCDN as suggestions for service optimization. Accepts free-form unconstrained values.Type: Array of stringsMandatory-to-Specify: No
Section 5 of describes the FCI Capability Advertisement Object, which
includes a CDNI Capability Object as well as the capability object type (a CDNI Payload Type).
The section also defines the Capability Objects per such type. Below we define additional Capability Objects.
In most cases, the presence or absence of a GenericMetadata object name in FCI.Metadata (as described above), is sufficient to convey support for a capability. There are cases, however, where more fine-grained capabilities declarations are required. Specifically, a dCDN may support some, but not all, of the capabilities specified by one of the new GenericMetadata objects. In these cases, new FCI objects are created to allow a dCDN to express these fine-grained capabilities.
Note: In the following sections, the term "mandatory-to-specify" is
used to convey which properties MUST be included when serializing a
given capability object. When mandatory-to-specify is defined as
"Yes" for an individual property, it means that if the object
containing that property is included in an FCI message, then the
mandatory-to-specify property MUST also be included.
This object is used to to indicate the support of authentication methods to be used for content acquisition (while interacting with an origin server) and authorization methods to be used for content delivery.
This documents defines two new authentication methods (See ), while there is one other authorization method under specification in CDNI called URI signingProperty: authe-types
Description: List of supported authentication methods possibly required for content acquisitionType: List of supported authentication methods (e.g. as defined in )Mandatory-to-Specify: No. No authentication method is supported in this case.Property: autho-types
Description: List of supported authorization methods possibly required for content deliveryType: List of supported authorization methods (e.g. URI signing)Mandatory-to-Specify: No. No authorization method is supported in this case.
This object is used to signal the set of features that are supported in relation with the ProcessingStages configuration object. Those optional features depend on the CDNI-MEL language support.
Property: features
Description: List of supported optional processing stages features. Note that these features all have some dependencies on support of the CDNi MEL expression language.Type: Enumeration [ExpressionMatch|RequestTransform|ResponseTransform] encoded as stringMandatory-to-Specify: No. None of these optional features are supported in this case.
This object is used to signal the supported features related to the SourceMetadataExtended configuration object.
Property: load-balance
Description: List of supported load balancing algorithms in relation to the SourceMetadataExtended configuration object see GenericMetadata: SourceMetadataExtendedType: Enumeration [random|content-hash|ip-hash] encoded as lowercase stringsMandatory-to-Specify: No. load balancing is not supported among sources.
If the FCI.SourceMetadataExtended object is not exposed neither advertised or if the load-balance array is empty, the dCDN does not support the usage of the load-balance property attached to the SourceMetadataExtended configuration object (see SourceMetadataExtended).
This object is used by the dCDN to signal/announce the supported request routing modes. This can be optionally used by the uCDN to further select a subset of those modes when operating one of the iterative delegation modes. See Property: request-routing-modes
Description: List of supported request routing modes by the dCDN. This information is useful when the uCDN decides to perform a delegation in iterative mode.Type: Enumeration [DNS|HTTP-R|MANIFESTREWRITE] encoded as uppercase stringsMandatory-to-Specify: No. If the dCDN does not advertise the supported request routing modes they are all supported by default.
This object is used by the dCDN to signal/announce the list of supported private features. See Property: features
Description: The list of supported private featureType: List array of nested objects each containing the following properties:
Property: feature-oid
Description: The owner/organization that has specified the feature.Type: StringMandatory-to-Specify: YesProperty: feature-type
Description: Indicates the typename of the private feature configuration object.Type: StringMandatory-to-Specify: Yes
This object is used by the dCDN to signal/announce the supported OCN types and/or their transport arrangement and/or medium supported by OCNs.
Property: ocn-delivery-list
Description: List of supported medium and/or transport arrangements.
Type: List of OcnDeliveryList objects Property: ocn-type-list
oDescription: List of supported OCN types. Examples include: HOME or EDGE.oType: Array of stringsoMandatory-to-Specify: NoProperty: ocn-medium
Description: This property lists the supported mediums.Type: List of strings. The following values are specified: SATELLITEMandatory-to-Specify: NoProperty: ocn-transport
Description: Instructs the dCDN to perform delegation operating a particular transport arrangement. The following values are specified: MABR.Type: List of stringsMandatory-to-Specify: No
The CDNI Metadata Expression Language provides a syntax with a rich set of variables, operators, and built-in functions to facilitate use cases within the extended CDNi metadata model.
Enables expression matching to dynamically determine if StageMetadata should be applied at a StageRules match.
Enables the dynamic construction of a value to be used in scenarios such as constructing a service identifier or cache key, setting an HTTP header, rewriting a request URI, setting a response status code, or dynamically generating a response body for a SyntheticResponse.
Expressions can evaluate to a Boolean, string, or integer, depending on the use case:
UsageDescriptionEvaluation ResultsExpressionMatch.expressionDynamically determines if StageMetadata should be applied at a specific StageRules.Boolean. Expressions that do not evaluate to True or False shall be considered as False.RequestTransform.uriRewrites request URI that will be presented to all downstream stages.StringResponseTransform.response-statusDynamically sets a response status code to replace the status-code returned by the origin.Integer (HTTP status code)SyntheticResponse.response-statusDynamically sets a response status code for a synthetically constructed response.Integer (HTTP status code)SyntheticResponse.bodyDynamically constructs a response body.StringHTTPHeader.valueDynamically constructs a header value.StringComputedCacheKey.expressionDynamically constructs a cache key.StringServiceIDs.properry-id,ServiceIDs.service-idDynamically constructs service and property identifiers.StringVariableMeaningreq.h.]]>Request header ]]>req.uriRequest URI (includes query string and fragment identifier, if any)req.uri.pathRequest URI pathreq.uri.pathqueryRequest path and query stringreq.uri.queryRequest query stringreq.uri.query.]]>Request query string value associated with ]]>req.methodRequest HTTP method (GET, POST, others)resp.h.]]>Response header ]]>resp.statusResponse status codeOperatorTypeResult TypeMeaning==infixBooleanEquality test!=infixBooleanInequality test!infixBooleanLogical NOT operator]]>infixBooleanGreater than testinfixBooleanLess than test=]]>infixBooleanGreater than or equal testinfixBooleanLess than or equal*=infixBooleanGlob style matchinfixBooleanRegular expression match (see https://www.pcre.org/ for details on PCRE RegEx matching)ipmatchinfixBooleanMatch against IP address or CIDR (IPv4 and IPv6)+infixNumericAddition-infixNumericSubtraction*infixNumericMultiplication/infixNumericDivision%infixUnsigned or IntegerModulus.infixStringConcatenation? :ternary*Conditional operator: ]]> ? ]]> : ]]>
Evaluates ]]> if ]]> is true, ]]> otherwise.( )groupingUsed to override precedence and for function calls.KeywordMeaningandLogical ANDorLogical ORnotLogical NOT (see also the ! operator)nilNo value (distinct from empty value)trueBoolean constant: truefalseBoolean constant: falseFunctionActionArgument(s)Returnsinteger(e)Converts expression to integer.1integerreal(e)Converts expression to real.1realstring(e)Converts expression to string.1stringboolean(e)Converts expression to Boolean.1BooleanFunctionActionArgument(s)Returnsupper(e)Converts a string to uppercase. Useful for case-insensitive comparisons.1stringlower(e)Converts a string to lowercase. Useful for case-insensitive comparisons.1stringFunctionActionArgument(s)Returnsmatch(string Input, string Match)Regular expression Match is applied to Input and the matching element (if any) is returned. Empty string is returned if there is no match.
See https://www.pcre.org/ for details on PCRE RegEx matching.2stringmatch_replace(string Input, string Match, string Replace)Regular expression Match is applied to Input arg and replaced with the Replace arg upon successful match. Returns updated (replaced) version of Input.3stringadd_query(string Input, string q, string v)Add query string element q with value v to the Input string. If v is nil, then just add the query string element q.
The query element q and value v must conform to the format defined in: https://datatracker.ietf.org/doc/html/rfc39862stringremove_query(string Input, string q)Remove (all occurrences of) query string element q from the Input string.2stringpath_element(string Input, integer n)Return the path element n from Input. -1 returns the last element.2stringpath_element(string Input, integer n, integer m)Return the path elements from position n to m.3string
To ensure reliable service, all CDNI Metadata configurations MUST be validated for syntax errors before they are ingested into a dCDN. That is, existing configurations should be kept as the live running configuration until the new configuration has passed validation. If errors are detected in a new configuration, the configuration MUST be rejected. A HTTP 500 Internal Server Error should be returned with a message that indicates the source of the error (line number, and configuration element that caused the error).
Examples of compile-time errors:
Configuration does not parse relative to the CDNI Metadata JSON schema
Unknown CDNI Metadata object referenced in the configuration
CDNI Metadata object parse error
Missing mandatory CDNI Metadata property
Unknown CDNI Metadata property
Incorrect type for a CDNI Metadata property value
CDNI-MEL
Unknown CDNI-MEL variable name referenced in an expression
Unknown CDNI-MEL operator, key-word, or functions referenced in an expression
Incorrect number of arguments used in a CDNI-MEL expression operator or function
Incorrect type of argument used in a CDNI-MEL expression operator or function
If a runtime error is detected when processing a request, the request should be terminated, and a HTTP 500 'Internal Server Error' returned to the caller. To avoid security leaks, sensitive information MUST be removed from the error message before it is returned to an external client. In addition to returning the HTTP 500 error, the dCDN SHOULD log additional diagnostics information to assist in troubleshooting.
Examples of runtime errors:
Failure to allocate memory (or other server resources) when evaluating a CDNI-MEL expression
Incorrect runtime argument type in a CDNI-MEL expression. E.g., trying to convert a non-numeric string to a number
Sets the MI.ComputedCacheKey to the value of the X-Cache-Key header from the client request.Sets the MI.ComputedCacheKey to the lowercase version of the URI.
ExpressionMatch where the expression is true if the user-agent (glob) matches *Safari* and the referrer equals www.example.com.
Adds X-custom-response-header with a value equal to the value of user-agent - host header.
Adds a Set-Cookie header with a dynamically computed cookie value (concatenating user agent and host name) and forces a 403 response.
Extracts the first path element from the URI. For example, if the URI = /789/second/third/test.txt, property-id is set to the first-path (789).
TBD. Two new auth-types will be introduced for use with SourceMetadataExtended: AWSv4Auth and HeaderAuth
This specification is in accordance with the CDNI Request Routing:
Footprint and Capabilities Semantics. As such, it is subject to the security and privacy considerations as
defined in Section 8 of
and in Section 7 of
respectively.
MORE - TBD
The authors would like to express their gratitude to the members of the Streaming Video Alliance Open Caching Working Group for their guidance / contribution / reviews ...)
Cross-Origin Resource Sharing
URI Signing for CDN Interconnection (CDNI)
CDNI Capacity Insights Capability Advertisment Extensions
Streaming Video Alliance Home Page
Open Caching Home Page
Authenticating Requests (AWS Signature Version 4)
Open Caching - Configuration Interface Functional Specification (Parts 1,2,3)
Lumen Technologies
Lumen Technologies
Broadpeak
Telefonica