| Network Working Group | A. Morton |
| Internet-Draft | AT&T Labs |
| Intended status: Standards Track | M. Bagnulo |
| Expires: June 15, 2016 | UC3M |
| P. Eardley | |
| BT | |
| K. D'Souza | |
| AT&T Labs | |
| December 13, 2015 |
Initial Performance Metric Registry Entries
draft-morton-ippm-initial-registry-02
This memo defines the Initial Entries for the Performance Metrics Registry.
Section 4 has been edited to reflect recent discussion on the ippm-list: * Removed the combination or "Raw" and left 95th percentile. * Hanging Indent on Run-time parameters (Fixed parameters use bullet lists and other indenting formats. * Payload format for measurement has been removed. * Explanation of Conditional delay distribution.
Still need: * suggestion of standard naming format for parameters. * usecase(s) for machine parse-able registry columns. * Contribution of YANG references for alternate data formats.
Note: lambda parameter description is correct in section 4, elsewhere needs fix.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 15, 2016.
Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Note: Efforts to synchronize structure and terminology with [I-D.ietf-ippm-metric-registry] will likely be incomplete until both drafts are stable.
This memo defines the Initial set of entries for the Performance Metric Registry. The registry will contain Active Performance Metrics, especially those defined in RFCs prepared in the IP Performance Metrics (IPPM) Working Group of the IETF, according to their framework [RFC2330]. Three aspects make IPPM metric registration difficult: (1) Use of the Type-P notion to allow users to specify their own packet types. (2) Use of Flexible input variables, called Parameters in IPPM definitions, some which determine the quantity measured and others which should not be specified until execution of the measurement. (3) Allowing flexibility in choice of statistics to summarize the results on a stream of measurement packets. This memo uses terms and definitions from the IPPM literature, primarily [RFC2330], and the reader is assumed familiar with them or may refer questions there as necessary.
Although there are several standard templates for organizing specifications of performance metrics (see [RFC2679] for an example of the traditional IPPM template, based to large extent on the Benchmarking Methodology Working Group's traditional template in [RFC1242], and see [RFC6390] for a similar template), none of these templates were intended to become the basis for the columns of an IETF-wide registry of metrics. While examinating aspects of metric specifications which need to be registered, it was clear that none of the existing metric templates fully satisfies the particular needs of a registry.
[I-D.ietf-ippm-metric-registry] defines the overall structure for a Performance Metric Registry and provides guidance for the process to examine proposed metrics and maitain Registered Metrics.
This document defines the initial set of Performance Metrics Registry entries; all are active metrics, or those where the packets measured have been specially generated for the purpose of measurement.
A row in the registry corresponds to one Registered Performance Metric, with entries in the various columns specifying the metric.
As discussed in [I-D.ietf-ippm-metric-registry], each entry (row) must be tightly defined; the definition must leave open only a few parameters that do not change the fundamental nature of the measurement (such as source and destination addresses), and so promotes comparable results across independent implementations. Also, each registered entry must be based on existing reference RFCs (or other standards) for performance metrics, and must be operationally useful and have significant industry interest. This is ensured by expert review for every entry before IANA action.
This section defines the categories and columns of the registry. Below, categories are described at the 3.x heading level, and columns are at the 3.x.y heading level. The Figure below illustrates this organization. An entry (row) therefore gives a complete description of a Registered Metric.
Registry Categories and Columns, shown as
Category
------------------
Column | Column |
Summary
--------------------------------
ID | Name | URIs | Description |
Metric Definition
-----------------------------------------
Reference Definition | Fixed Parameters |
Method of Measurement
---------------------------------------------------------------
Reference | Packet | Traffic | Sampling | Run-time | Role |
Method | Generation | Filter | dist. | Param | |
| Stream |
Output
----------------------------
Type | Reference | Units |
| Definition | |
Administrative information
----------------------------------
Status |Request | Rev | Rev.Date |
Comments and Remarks
--------------------
Each column serves as a check-list item and helps to avoid omissions during registration and expert review. In some cases an entry (row) may have some columns without specific entries, marked Not Applicable (NA).
This section gives an initial registry entry for the UDP Round-trip Latency.
Note: If each Registry entry should only produce a "raw" output or a statistical summary, then the "Output" Category can be split and this section can become two closely-related metrics.
This category includes multiple indexes to the registry entries, the element ID and metric name.
<insert numeric identifier, an integer>
<insert name according to metric naming convention>
Act_IP_UDP_Round-trip_Delay_95th-percentile_Poisson
URN: Prefix urn:ietf:params:performance:metric...<name>
URL: http://<TBD by IANA>/<name>
This metric assesses the delay of a stream of packets exchanged between two hosts (or measurement points), and the Output is the Round-trip delay for all successfully exchanged packets expressed as the 95th percentile of their conditional delay distribution.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, September 1999.
<specific section reference and additional clarifications, if needed>
Section 2.4 of [RFC2681] provides the reference definition of the singleton (single value) Round-trip delay metric. Section 3.4 of [RFC2681] provides the reference definition expanded to cover a multi-value sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].
Note that although the definition of "Round-trip-Delay between Src and Dst at T" is directionally ambiguous in the text, this metric tightens the definition further to recognize that the host in the "Src" role will send the first packet to "Dst", and ultimately receive the corresponding return packet from "Dst" (when neither are lost).
Finally, note that the variable "dT" is used in [RFC2681] to refer to the value of Round-trip delay in metric definitions and methods. The variable "dT" has been re-used in other IPPM literature to refer to different quantities, and cannot be used as a global variable name.
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
Type-P:
Tmax: a loss threshold waiting time
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
The methodology for this metric is defined as Type-P-Round-trip-Delay-Poisson-Stream in section 2.6 of RFC 2681 [RFC2681] and section 3.6 of RFC 2681 [RFC2681] using the Type-P and Tmax defined under Fixed Parameters.
The reference method distinguishes between long-delayed packets and lost packets by implementing a maximum waiting time for packet arrival. Tmax is the waiting time used as the threshold to declare a packet lost. Lost packets SHALL be designated as having undefined delay.
The calculations on the delay (RTT) SHALL be performed on the conditional distribution, conditioned on successful packet arrival within Tmax. Also, when all packet delays are stored, the process which calculates the RTT value MAY enforce the Tmax threshold on stored values before calculations. See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
The reference method requires some way to distinguish between different packets in a stream and establish correspondence between sending times and receiving times for each successfully-arriving packet. Sequence numbers or other send-order information MUST be retained at the Src or included with each packet to dis-ambiguate packet reordering if it occurs. The chosen measurement protocol will dictate the format of sequence numbers and time-stamps, if they are conveyed in the packet payload. The sequence numbers or timestamps MUST be determined by the measurement process itself after the Fixed and Runtime parameters are passed to that process.
Refer to Section 4.4 of [RFC6673] for expanded discussion of the instruction to "send a Type-P packet back to the Src as quickly as possible" in Section 2.6 of RFC 2681 [RFC2681]. Section 8 of [RFC6673] presents additional requirements which shall be included in the method of measurement for this metric.
This section gives the details of the packet traffic which is the basis for measurement. In IPPM metrics, this is called the Stream, and can easily be described by providing the list of stream parameters.
<section/specification references, and description of any new generation parameters, if needed>
Section 11.1.3 of [RFC2330] provides three methods to generate Poisson sampling intervals. the reciprocal of lambda is the average packet spacing, thus the Run-time Parameter is Reciprocal_lambda = 1/lambda, in seconds.
>>> Check with Sam, most likely it is this...
Method 3 SHALL be used, where given a start time (Run-time Parameter), the subsequent send times are all computed prior to measurement by computing the pseudo-random distribution of inter-packet send times, (truncating the distribution as specified in the Run-time Parameter, Trunc), and the Src sends each packet at the computed times.
Note that Trunc is the upper limit on inter-packet times in the Poisson distribution. A random value greater than Trunc is set equal to Trunc instead.
The measured results based on a filtered version of the packets observed, and this section provides the filter details (when present).
<section reference>.
NA
<insert time distribution details, or how this is diff from the filter>
NA
Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.
<list of run-time parameters, and their data formats>
>>> should Poisson run-time params be fixed instead? probably yes if modeling a specific version of MBA tests.
<lists the names of the different roles from the measurement method>
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
Percentile -- for the conditional distribution of all packets with a valid value of Round-trip delay (undefined delays are excluded), a single value corresponding to the 95th percentile.
<describe the data format for each type of result>
For all outputs ---
Raw -- REMOVED IN VERSION 01
Percentile -- for the conditional distribution of all packets with a valid value of Round-trip delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3 of [RFC3393] for details on the percentile statistic (where Round-trip delay should be substituted for "ipdv").
The percentile = 95.
The time value of the result is expressed in units of seconds, with resolution of 0.0001 seconds (0.1 ms), and with lossless conversion to/from the 32-bit NTP timestamp as per section 6 of RFC [RFC5905]
<pointer to section/spec where output type/format is defined>
See the Data Format column for references.
<insert units for the measured results, and the reference specification>.
The 95th Percentile of Round-trip Delay is expressed in seconds.
<current or depricated>
name or RFC, etc.
1.0
YYYY-MM-DD
Additional (Informational) details for this entry
This section gives an initial registry entry for a Packet Delay Variation metric.
Note: If each Registry entry should only produce a "raw" output or a statistical summary, then the "Output" Category can be split and this section can become two closely-related metrics.
This category includes multiple indexes to the registry entries, the element ID and metric name.
<skipping some Summary columns for now>
<insert numeric identifier, an integer>
<insert name according to metric naming convention>
Act_IP-UDP-One-way-pdv-95th-percentile-Poisson
URL: ??
URI: Prefix urn:ietf:params:performance:metric<add name>
An assessment of packet delay variation with respect to the minimum delay observed on the stream.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998. [RFC2330]
Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002. [RFC3393]
Morton, A. and B. Claise, "Packet Delay Variation Applicability Statement", RFC 5481, March 2009. [RFC5481]
Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, June 2010.[RFC5905]
<specific section reference and additional clarifications, if needed>
See sections 2.4 and 3.4 of [RFC3393]. Singleton delay differences measured are referred to by the variable name "ddT".
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
See section 2.6 and 3.6 of [RFC3393] for singleton elements.
<list of generation parameters and section/spec references if needed>
Poisson distributed as described in [RFC2330], with the following Parameters.
<insert the measured results based on a filtered version of the packets observed, and this section provides the filter details (when present), and section reference>.
NA
<insert time distribution details, or how this is diff from the filter>
NA
<list of run-time parameters, and any reference(s)>.
<lists the names of the different roles from the measurement method>
Src - the host that sends the stream of packets.
Dst - the host that receives the stream of packets.
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
Raw -- for each packet sent, pairs of values.
Percentile -- for the conditional distribution of all packets with a valid value of one-way delay (undefined delays are excluded), a single value corresponding to the 95th percentile of the singletons, ddT.
<describe the data format for each type of result>
For all Output types
Raw -
Percentile -- for the conditional distribution of all packets with a valid value of one-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3 of [RFC3393] for details on the percentile statistic (where pdv should be substituted for "ipdv").
The percentile = 95.
Data format is a 32-bit signed floating point value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
<pointer to section/spec where output type/format is defined>
see Data Format column.
<insert units for the measured results, and the reference specification>.
See section 3.3 of [RFC3393] for singleton elements, ddT. The units are seconds, and the same units are used for 95th percentile.
[RFC2330] recommends that when a time is given, it will be expressed in UTC.
The timestamp format (for T, Tf, etc.) is the same as in [RFC5905] (64 bits) and is as follows: the first 32 bits represent the unsigned integer number of seconds elapsed since 0h on 1 January 1900; the next 32 bits represent the fractional part of a second that has elapsed since then.
<current or depricated>
<name of individual or RFC, etc.>
1.0
YYYY-MM-DD
<Additional (Informational) details for this entry>
Lost packets represent a challenge for delay variation metrics. See section 4.1 of [RFC3393] and the delay variation applicability statement[RFC5481] for extensive analysis and comparison of PDV and an alternate metric, IPDV.
This section gives an initial registry entry for DNS Response Latency. RFC 2681 [RFC2681] defines a Round-trip delay metric. We build on that metric by specifying several of the input parameters to precisely define a metric for measuring DNS latency.
This category includes multiple indexes to the registry entries, the element ID and metric name.
<skipping some admin columns for now>
<insert numeric identifier, an integer>
<insert name according to metric naming convention>
URL: ??
URI: Prefix urn:ietf:params:performance:metric
This metric assesses the response time, the interval from the query transmission to the response.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, November 1987. (and updates)
Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, September 1999.
<specific section reference and additional clarifications, if needed>
Section 2.4 of [RFC2681] provides the reference definition of the singleton (single value) Round-trip delay metric. Section 3.4 of [RFC2681] provides the reference definition expanded to cover a multi-value sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].
For DNS Response Latency, the entities in [RFC1035] must be mapped to [RFC2681]. The Local Host with its User Program and Resolver take the role of "Src", and the Foreign Name Server takes the role of "Dst".
Note that although the definition of "Round-trip-Delay between Src and Dst at T" is directionally ambiguous in the text, this metric tightens the definition further to recognize that the host in the "Src" role will send the first packet to "Dst", and ultimately receive the corresponding return packet from "Dst" (when neither are lost).
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
Type-P:
Observation: reply packets will contain a DNS response and may contain RRs.
Timeout: Tmax = 5 seconds (to help disambiguate queries)
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
The methodology for this metric is defined as Type-P-Round-trip-Delay-Poisson-Stream in section 2.6 of RFC 2681 [RFC2681] and section 3.6 of RFC 2681 [RFC2681] using the Type-P and Timeout defined under Fixed Parameters.
The method requires sequence numbers or other send-order information to be retained at the Src or included with each packet to dis-ambiguate packet reordering if it occurs. Sequence number is part of the payload described under Fixed Parameters.
DNS Messages bearing Queries provide for random ID Numbers, so more than one query may be launched while a previous request is outstanding when the ID Number is used.
IF a DNS response does not arrive within Tmax, the result is undefined. The Message ID SHALL be used to disambiguate the successive queries.
>>> This would require support of ID generation and population in the Message. An alternative would be to use a random Source port on the Query Message, but we would choose ONE before proceding.
Refer to Section 4.4 of [RFC6673] for expanded discussion of the instruction to "send a Type-P packet back to the Src as quickly as possible" in Section 2.6 of RFC 2681 [RFC2681]. Section 8 of [RFC6673] presents additional requirements which shall be included in the method of measurement for this metric.
This section gives the details of the packet traffic which is the basis for measurement. In IPPM metrics, this is called the Stream, and can easily be dscribed by providing the list of stream parameters.
<list of generation parameters and section/spec references if needed>
Section 11.1.3 of RFC 2681 [RFC2330] provides three methods to generate Poisson sampling intervals. the reciprocal of lambda is the average packet rate, thus the Run-time Parameter is 1/lambda.
>>> Check with Sam, most likely it is this...
Method 3 is used, where given a start time (Run-time Parameter), the subsequent send times are all computed prior to measurement by computing the pseudo-random distribution of inter-packet send times, (truncating the distribution as specified in the Run-time Parameters), and the Src sends each packet at the computed times.
The measured results based on a filtered version of the packets observed, and this section provides the filter details (when present).
<section reference>.
NA
<insert time distribution details, or how this is diff from the filter>
NA
Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.
<list of run-time parameters, and their data formats>
[RFC5905] (32 bits) and is as follows: the first 16 bits represent the integer number of seconds; the next 16 bits represent the fractional part of a second.
The format for 1/lambda and Upper limit of Poisson Dist. are the short format in
>>> should Poisson run-time params be fixed instead? probably yes if modeling a specific version of MBA tests.
<lists the names of the different roles from the measurement method>
Src - launches each packet and waits for return transmissions from Dst.
Dst - waits for each packet from Src and sends a return packet to Src.
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
For all output types:
Raw -- for each packet sent, pairs of values.
>>> and the status of the response, only assigning values to successful query-response pairs.
Percentile -- for the conditional distribution of all packets with a valid value of Round-trip delay (undefined delays are excluded), a single value corresponding to the 95th percentile.
<describe the data format for each type of result>
Raw -- for each packet sent, pairs of values as follows:
Percentile -- for the conditional distribution of all packets with a valid value of Round-trip delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3 of [RFC3393] for details on the percentile statistic (where Round-trip delay should be substituted for "ipdv").
The percentile = 95.
Data format is a 32-bit signed floating point value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
<pointer to section/spec where output type/format is defined>
See the Data Format column for references.
<insert units for the measured results, and the reference specification>.
Round-trip Delay, dT, is expressed in seconds.
The 95th Percentile of Round-trip Delay is expressed in seconds.
<current or depricated>
name or RFC, etc.
1.0
YYYY-MM-DD
Additional (Informational) details for this entry
This section gives an initial registry entry for the UDP Poisson One-way Delay.
Note: Each Registry "Name" below specifies a single registry entry, whose output format varies according to a component of the name that specifies one form of statistical summary.
IANA is asked to assign a different numeric identifiers to each Name. All other column entries are the same, thus this section is proposes five closely-related registry entries. As a result, IANA is also asked to assign corresponding URIs and URLs.
This category includes multiple indexes to the registry entries, the element ID and metric name.
<insert numeric identifier, an integer, one corresponding to each name below>
<insert name according to metric naming convention>
Act_IP_UDP_Poisson_UDP-Payload-250_One-way_Delay_<statistic>
Act_IP_UDP_Poisson_UDP-Payload-250_One-way_Delay_Percentile95
Act_IP_UDP_Poisson_UDP-Payload-250_One-way_Delay_Mean
Act_IP_UDP_Poisson_UDP-Payload-250_One-way_Delay_Min
Act_IP_UDP_Poisson_UDP-Payload-250_One-way_Delay_Max
Act_IP_UDP_Poisson_UDP-Payload-250_One-way_Delay_Std_Dev
URI: Prefix urn:ietf:params:performance:metric...<name>
URL: http:\\www.iana.org\ ... <name>
This metric assesses the delay of a stream of packets exchanged between two hosts (or measurement points), and reports the <statistic> One-way delay for all successfully exchanged packets based on their conditional delay distribution.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, September 1999.
Morton, A., and Stephan, E., "Spatial Composition of Metrics", RFC 6049, January 2011.
<specific section reference and additional clarifications, if needed>
Section 3.4 of [RFC2679] provides the reference definition of the singleton (single value) One-way delay metric. Section 4.4 of [RFC2679] provides the reference definition expanded to cover a multi-value sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].
Only successful packet transfers with finite delay are included in the sample, as prescribed in section 4.1.2 of [RFC6049].
NOTE: RFC2679 will be replaced by 2679-bis on approval, see draft-ietf-ippm-2679-bis-01.
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
Type-P:
Timeout, Tmax: 3 seconds
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
The methodology for this metric is defined as Type-P-One-way-Delay-Poisson-Stream in section 3.6 of [RFC2679] and section 4.6 of [RFC2679] using the Type-P and Timeout defined under Fixed Parameters.
The method requires sequence numbers or other send-order information to be retained at the Src or included with each packet to dis-ambiguate packet reordering if it occurs. Sequence number is part of the TWAMP payload described under Fixed Parameters.
This section gives the details of the packet traffic which is the basis for measurement. In IPPM metrics, this is called the Stream, and can easily be dscribed by providing the list of stream parameters.
<list of generation parameters and section/spec references if needed>
Section 11.1.3 of RFC 2681 [RFC2330] provides three methods to generate Poisson sampling intervals. The reciprocal of lambda is the average packet rate, thus the Run-time Parameter is 1/lambda.
Method 3 or equivalent SHALL used, where given a start time (Run-time Parameter), the subsequent send times are all computed prior to measurement by computing the pseudo-random distribution of inter-packet send times, (truncating the distribution as specified in the Run-time Parameters), and the Src sends each packet at the computed times.
NA
NA
Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.
<list of run-time parameters, and their data formats>
[RFC5905] (32 bits) and is as follows: the first 16 bits represent the integer number of seconds; the next 16 bits represent the fractional part of a second.
The format for 1/lambda and Upper limit of Poisson Dist. are the short format in
>>> should Poisson run-time params be fixed instead? probably yes if modeling a specific version of tests. Note in the NAME, i.e. Poisson3.3
<lists the names of the different roles from the measurement method>
Src - launches each packet and waits for return transmissions from Dst. This is the TWAMP Session-Sender.
Dst - waits for each packet from Src and sends a return packet to Src. This is the TWAMP Session-Reflector.
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
See subsection titles below for Types.
<describe the data format for each type of result>
For all output types ---
The 95th percentile SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3 of [RFC3393] for details on the percentile statistic (where Round-trip delay should be substituted for "ipdv").
The percentile = 95.
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
The mean SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.2.2 of [RFC6049] for details on calculating this statistic, and 4.2.3 of [RFC6049].
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
The minimum SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for details on calculating this statistic, and 4.3.3 of [RFC6049].
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
The maximum SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for a closely related method for calculating this statistic, and 4.3.3 of [RFC6049]. The formula is as follows:
Max = (FiniteDelay [j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
<pointer to section/spec where output type/format is defined>
See the Data Format column for references.
<insert units for the measured results, and the reference specification>.
The <statistic> of One-way Delay is expressed in seconds.
The 95th Percentile of One-way Delay is expressed in seconds.
<current or depricated>
name or RFC, etc.
1.0
YYYY-MM-DD
Additional (Informational) details for this entry
This section gives an initial registry entry for the UDP Periodic One-way Delay.
Note: Each Registry "Name" below specifies a single registry entry, whose output format varies according to a component of the name that specifies one form of statistical summary.
IANA is asked to assign a different numeric identifiers to each Name. All other column entries are the same, thus this section is proposes five closely-related registry entries. As a result, IANA is also asked to assign corresponding URIs and URLs.
This category includes multiple indexes to the registry entries, the element ID and metric name.
<insert numeric identifier, an integer, one corresponding to each name below>
<insert name according to metric naming convention>
Act_IP_UDP_Periodic-var_UDP-Payload-142_One-way_Delay_<statistic>
Act_IP_UDP_Periodic-var_UDP-Payload-142_One-way_Delay_Percentile95
Act_IP_UDP_Periodic-var_UDP-Payload-142_One-way_Delay_Mean
Act_IP_UDP_Periodic-var_UDP-Payload-142_One-way_Delay_Min
Act_IP_UDP_Periodic-var_UDP-Payload-142_One-way_Delay_Max
Act_IP_UDP_Periodic-var_UDP-Payload-142_One-way_Delay_Std_Dev
URI: Prefix urn:ietf:params:performance:metric...<name>
URL: http:\\www.iana.org\ ... <name>
This metric assesses the delay of a stream of packets exchanged between two hosts (or measurement points), and reports the <statistic> One-way delay for all successfully exchanged packets based on their conditional delay distribution.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, September 1999.
Morton, A., and Stephan, E., "Spatial Composition of Metrics", RFC 6049, January 2011.
<specific section reference and additional clarifications, if needed>
Section 3.4 of [RFC2679] provides the reference definition of the singleton (single value) One-way delay metric. Section 4.4 of [RFC2679] provides the reference definition expanded to cover a multi-value sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].
Only successful packet transfers with finite delay are included in the sample, as prescribed in section 4.1.2 of [RFC6049].
NOTE: RFC2679 will be replaced by 2679-bis on approval, see draft-ietf-ippm-2679-bis-01.
ANY other conditions, ...
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
Type-P:
Timeout, Tmax: 3 seconds
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
The methodology for this metric is defined as Type-P-One-way-Delay-Poisson-Stream in section 3.6 of [RFC2679] and section 4.6 of [RFC2679] using the Type-P and Timeout defined under Fixed Parameters.
The method requires sequence numbers or other send-order information to be retained at the Src or included with each packet to dis-ambiguate packet reordering if it occurs. Sequence number is part of the TWAMP payload described under Fixed Parameters.
This section gives the details of the packet traffic which is the basis for measurement. In IPPM metrics, this is called the Stream, and can easily be dscribed by providing the list of stream parameters.
<list of generation parameters and section/spec references if needed>
Section 3 of [RFC3432] prescribes the method for generating Periodic streams using associated parameters.
NOTE: an initiation process with a number of control exchanges resulting in unpredictable start times (within a time interval) may be sufficient to avoid synchronization of periodic streams, and therefore a valid replacement for selecting a start time at random from a fixed interval.
These stream parameters will be specified as Run-time parameters.
NA
NA
Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.
<list of run-time parameters, and their data formats>
[RFC5905] (32 bits) and is as follows: the first 16 bits represent the integer number of seconds; the next 16 bits represent the fractional part of a second.
The format for incT and dT are the short format in
>>> should Periodic run-time params be fixed instead? probably yes if modeling a specific version of tests. Note in the NAME, i.e. Poisson3.3
<lists the names of the different roles from the measurement method>
Src - launches each packet and waits for return transmissions from Dst. This is the TWAMP Session-Sender.
Dst - waits for each packet from Src and sends a return packet to Src. This is the TWAMP Session-Reflector.
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
See subsection titles in Data Format for Types.
<describe the data format for each type of result>
For all output types ---
The 95th percentile SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3 of [RFC3393] for details on the percentile statistic (where Round-trip delay should be substituted for "ipdv").
The percentile = 95.
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
The mean SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.2.2 of [RFC6049] for details on calculating this statistic, and 4.2.3 of [RFC6049].
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
The minimum SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for details on calculating this statistic, and 4.3.3 of [RFC6049].
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
The maximum SHALL be calculated using the conditional distribution of all packets with a finite value of One-way delay (undefined delays are excluded), a single value as follows:
See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for a closely related method for calculating this statistic, and 4.3.3 of [RFC6049]. The formula is as follows:
Max = (FiniteDelay [j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
Data format is a 32-bit signed value, *similar to* the 32-bit short NTP Time format in Section 6 of [RFC5905] and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.
<pointer to section/spec where output type/format is defined>
See the Data Format column for references.
<insert units for the measured results, and the reference specification>.
The <statistic> of One-way Delay is expressed in seconds.
<current or depricated>
name or RFC, etc.
1.0
YYYY-MM-DD
Additional (Informational) details for this entry
This section gives an initial registry entry for ....
This category includes multiple indexes to the registry entries, the element ID and metric name.
<skipping the admin columns for now>
<insert numeric identifier, an integer>
<insert name according to metric naming convention>
URL: ??
URI: Prefix urn:ietf:params:performance:metric
TBD.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, September 1999.
<specific section reference and additional clarifications, if needed>
Section 2.4 of [RFC2681] provides the reference definition of the singleton (single value) Round-trip delay metric. Section 3.4 of [RFC2681] provides the reference definition expanded to cover a multi-value sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].
Note that although the definition of "Round-trip-Delay between Src and Dst at T" is directionally ambiguous in the text, this metric tightens the definition further to recognize that the host in the "Src" role will send the first packet to "Dst", and ultimately receive the corresponding return packet from "Dst" (when neither are lost).
<<< Check how the Methodology also makes this clear (or not) >>>
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
Type-P:
Timeout: 3 seconds
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
This section gives the details of the packet traffic which is the basis for measurement. In IPPM metrics, this is called the Stream, and can easily be dscribed by providing the list of stream parameters.
<list of generation parameters and section/spec references if needed>
The measured results based on a filtered version of the packets observed, and this section provides the filter details (when present).
<section reference>.
<insert time distribution details, or how this is diff from the filter>
Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.
<list of run-time parameters>
<reference(s)>.
<lists the names of the different roles from the measurement method>
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
<describe the data format for each type of result>
<pointer to section/spec where output type/format is defined>
<insert units for the measured results, and the reference specification>.
<current or depricated>
name or RFC, etc.
1.0
YYYY-MM-DD
Additional (Informational) details for this entry
This section gives an initial registry entry for ....
This category includes multiple indexes to the registry entries, the element ID and metric name.
<skipping the Summary columns for now>
<insert numeric identifier, an integer>
<insert name according to metric naming convention>
URL: ??
URI: Prefix urn:ietf:params:performance:metric
TBD.
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.
<Full bibliographic reference to an immutable doc.>
<specific section reference and additional clarifications, if needed>
<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.
<for metric, insert relevant section references and supplemental info>
<list of generation parameters and section/spec references if needed>
<insert the measured results based on a filtered version of the packets observed, and this section provides the filter details (when present), and section reference>.
<insert time distribution details, or how this is diff from the filter>
<list of run-time parameters, and any reference(s)>.
<lists the names of the different roles from the measurement method>
This category specifies all details of the Output of measurements using the metric.
<insert name of the output type, raw or a selected summary statistic>
<describe the data format for each type of result>
<pointer to section/spec where output type/format is defined>
<insert units for the measured results, and the reference specification>.
<current or depricated>
<name of individual or RFC, etc.>
1.0
YYYY-MM-DD
Additional (Informational) details for this entry
This section is MAY BE DELETED or adapted before submission.
This section gives an example registry entry for the end-point metric described in RFC 7003 [RFC7003], for RTCP-XR Burst/Gap Discard Metric reporting.
This category includes multiple indexes to the registry entries, the element ID and metric name.
An integer having enough digits to uniquely identify each entry in the Registry.
A metric naming convention is TBD.
Prefix urn:ietf:params:performance:metric
current
Alcelip Mornuley
1.0
2014-07-04
TBD.
[RFC3611] [RFC4566] [RFC6776] [RFC6792] [RFC7003]
This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters. Section 3.2 of [RFC7003] provides the reference information for this category.
Packets Discarded in Bursts:
The total number of packets discarded during discard bursts. The measured value is unsigned value. If the measured value exceeds 0xFFFFFD, the value 0xFFFFFE MUST be reported to indicate an over-range measurement. If the measurement is unavailable, the value 0xFFFFFF MUST be reported.
Fixed Parameters are input factors that must be determined and embedded in the measurement system for use when needed. The values of these parameters is specified in the Registry.
Threshold: 8 bits, set to value = 3 packets.
The Threshold is equivalent to Gmin in [RFC3611], i.e., the number of successive packets that must not be discarded prior to and following a discard packet in order for this discarded packet to be regarded as part of a gap. Note that the Threshold is set in accordance with the Gmin calculation defined in Section 4.7.2 of [RFC3611].
Interval Metric flag: 2 bits, set to value 11=Cumulative Duration
This field is used to indicate whether the burst/gap discard metrics are Sampled, Interval, or Cumulative metrics [RFC6792]:
I=10: Interval Duration - the reported value applies to the most recent measurement interval duration between successive metrics reports.
I=11: Cumulative Duration - the reported value applies to the accumulation period characteristic of cumulative measurements.
Senders MUST NOT use the values I=00 or I=01.
This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations. For the Burst/Gap Discard Metric, it appears that the only guidance on methods of measurement is in Section 3.0 of [RFC7003] and its supporting references. Relevant information is repeated below, although there appears to be no section titled "Method of Measurement" in [RFC7003].
Metrics in this block report on burst/gap discard in the stream arriving at the RTP system. Measurements of these metrics are made at the receiving end of the RTP stream. Instances of this metrics block use the synchronization source (SSRC) to refer to the separate auxiliary Measurement Information Block [RFC6776], which describes measurement periods in use (see [RFC6776], Section 4.2).
This metrics block relies on the measurement period in the Measurement Information Block indicating the span of the report. Senders MUST send this block in the same compound RTCP packet as the Measurement Information Block. Receivers MUST verify that the measurement period is received in the same compound RTCP packet as this metrics block. If not, this metrics block MUST be discarded.
Since RTCP-XR Measurements are conducted on live RTP traffic, the complete description of the stream is contained in SDP messages that proceed the establishment of a compatible stream between two or more communicating hosts. See Run-time Parameters, below.
The output type defines the type of result that the metric produces.
The measured results are apparently expressed in packets, although there is no section of [RFC7003] titled "Metric Units".
Run-Time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete. However, the values of these parameters is not specified in the Registry, rather these parameters are listed as an aid to the measurement system implementor or user (they must be left as variables, and supplied on execution).
The Data Format of each Run-time Parameter SHALL be specified in this column, to simplify the control and implementation of measurement devices.
SSRC of Source: 32 bits As defined in Section 4.1 of [RFC3611].
SDP Parameters: As defined in [RFC4566]
Session description v= (protocol version number, currently only 0)
o= (originator and session identifier : username, id, version number, network address)
s= (session name : mandatory with at least one UTF-8-encoded character)
i=* (session title or short information) u=* (URI of description)
e=* (zero or more email address with optional name of contacts)
p=* (zero or more phone number with optional name of contacts)
c=* (connection information—not required if included in all media)
b=* (zero or more bandwidth information lines) One or more Time descriptions ("t=" and "r=" lines; see below)
z=* (time zone adjustments)
k=* (encryption key)
a=* (zero or more session attribute lines)
Zero or more Media descriptions (each one starting by an "m=" line; see below)
m= (media name and transport address)
i=* (media title or information field)
c=* (connection information — optional if included at session level)
b=* (zero or more bandwidth information lines)
k=* (encryption key)
a=* (zero or more media attribute lines — overriding the Session attribute lines)
An example Run-time SDP description follows:
v=0
o=jdoe 2890844526 2890842807 IN IP4 192.0.2.5
s=SDP Seminar i=A Seminar on the session description protocol
u=http://www.example.com/seminars/sdp.pdf e=j.doe@example.com (Jane Doe)
c=IN IP4 233.252.0.12/127
t=2873397496 2873404696
a=recvonly
m=audio 49170 RTP/AVP 0
m=video 51372 RTP/AVP 99
a=rtpmap:99 h263-1998/90000
TBD.
These registry entries represent no known security implications for Internet Security. Each referenced Metric contains a Security Considerations section.
IANA is requested to populate The Performance Metric Registry defined in [I-D.ietf-ippm-metric-registry] with the values defined above.
<more is needed here>
The authors thank Brian Trammell for suggesting the term "Run-time Parameters", which led to the distinction between run-time and fixed parameters implemented in this memo, for identifying the IPFIX metric with Flow Key as an example, and for many other productive suggestions. Thanks to Peter Koch, who provided several useful suggestions for disambiguating successive DNS Queries in the DNS Response time metric.
The authors also acknowledge the constructive reviews and helpful suggestions from Barbara Stark, Juergen Schoenwaelder, Tim Carey, and participants in the LMAP working group.
| [I-D.ietf-ippm-metric-registry] | Bagnulo, M., Claise, B., Eardley, P. and A. Morton, "Registry for Performance Metrics", Internet Draft (work in progress) draft-ietf-ippm-metric-registry, 2014. |
| [RFC1035] | Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, DOI 10.17487/RFC1035, November 1987. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC2330] | Paxson, V., Almes, G., Mahdavi, J. and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, DOI 10.17487/RFC2330, May 1998. |
| [RFC2679] | Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679, September 1999. |
| [RFC2680] | Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, DOI 10.17487/RFC2680, September 1999. |
| [RFC2681] | Almes, G., Kalidindi, S. and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, September 1999. |
| [RFC3339] | Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002. |
| [RFC3393] | Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, DOI 10.17487/RFC3393, November 2002. |
| [RFC3432] | Raisanen, V., Grotefeld, G. and A. Morton, "Network performance measurement with periodic streams", RFC 3432, DOI 10.17487/RFC3432, November 2002. |
| [RFC4737] | Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S. and J. Perser, "Packet Reordering Metrics", RFC 4737, DOI 10.17487/RFC4737, November 2006. |
| [RFC5357] | Hedayat, K., Krzanowski, R., Morton, A., Yum, K. and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, DOI 10.17487/RFC5357, October 2008. |
| [RFC5905] | Mills, D., Martin, J., Burbank, J. and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010. |
| [RFC6049] | Morton, A. and E. Stephan, "Spatial Composition of Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011. |
| [RFC6673] | Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, DOI 10.17487/RFC6673, August 2012. |
| [RFC6991] | Schoenwaelder, J., "Common YANG Data Types", RFC 6991, DOI 10.17487/RFC6991, July 2013. |