Advanced Unidirectional Route
Assessment (AURA)Universidad de Buenos AiresAv. Paseo Colón 850Buenos AiresC1063ACVArgentina+54 11 5285-0716ihameli@cnet.fi.uba.arhttp://cnet.fi.uba.ar/ignacio.alvarez-hamelin/AT&T Labs200 Laurel Avenue SouthMiddletownNJ07748USA+1 732 420 1571+1 732 368 1192acm@research.att.comTU WienGusshausstrasse 25/E389Vienna1040Austria+43 1 58801 38813+43 1 58801 38898Joachim.Fabini@tuwien.ac.athttp://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/Cisco Systems, Inc.7200-11 Kit Creek RoadResearch Triangle ParkNC27709USAcpignata@cisco.comDeutsche TelekomHeinrich Hertz Str. 3-7Darmstadt64295Germany+49 6151 5812747Ruediger.Geib@telekom.deThis memo introduces an advanced unidirectional route assessment
(AURA) metric and associated measurement methodology, based on the IP
Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC
2330 in the areas of path-related terminology and path description,
primarily to include the possibility of parallel subpaths between a
given Source and Destination pair, owing to the presence of multi-path
technologies.The IETF IP Performance Metrics (IPPM) working group first created a
framework for metric development in . This
framework has stood the test of time and enabled development of many
fundamental metrics. It has been updated in the area of metric
composition , and in several areas related to
active stream measurement of modern networks with reactive properties
.The framework motivated the development of
"performance and reliability metrics for paths through the Internet,"
and Section 5 of defines terms that support
description of a path under test. However, metrics for assessment of
path components and related performance aspects had not been attempted
in IPPM when the framework was written.This memo takes up the route measurement challenge and specifies a
new route metric, two practical frameworks for methods of measurement
(using either active or hybrid active-passive methods ), and Round-Trip Delay and link information discovery
using the results of measurements. All route measurements are limited by
the willingness of hosts along the path to be discovered, to cooperate
with the methods used, or to recognize that the measurement operation is
taking place (such as when tunnels are present).Section 7 of presented a simple example of
a "route" metric along with several other examples. The example is
reproduced below (where the reference is to Section 5 of ):"route: The path, as defined in Section 5, from A to B at a given
time."This example provides a starting point to develop a more complete
definition of route. Areas needing clarification include:In practice, the route will be assessed over a
time interval, because active path detection methods like rely on TTL limits for their operation and cannot
accomplish discovery of all hosts using a single packet.The legacy route definition lacks the option
to cater for packet-dependent routing. In this memo, we assess the
route for a specific packet of Type-P, and reflect this in the
metric definition. The methods of measurement determine the
specific Type-P used.Parallel paths are a reality of the
Internet and a strength of advanced route assessment methods, so
the metric must acknowledge this possibility. Use of Equal Cost
Multi-Path (ECMP) and Unequal Cost Multi-Path (UCMP) technologies
are common sources of parallel subpaths.May contain hosts that do not
decrement TTL or Hop Limit, but may have two or more exchange
links connecting "discoverable" hosts or routers. Parallel
subpaths contained within clouds cannot be discovered. The
assessment methods only discover hosts or routers on the path that
decrement TTL or Hop Count, or cooperate with interrogation
protocols. The presence of tunnels and nested tunnels further
complicate assessment by hiding hops.Although the
definition of a hop was a link-host pair, only hosts that are
discoverable or have the capability to cooperate with
interrogation protocols where link information may be exposed.The refined definition of Route metrics begins in the
sections that follow.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 when, and only
when, they appear in all capitals, as shown here.The purpose of this memo is to add new route metrics and methods of
measurement to the existing set of IPPM metrics.The scope is to define route metrics that can identify the path taken
by a packet or a flow traversing the Internet between two hosts.
Although primarily intended for hosts communicating on the Internet with
IP, the definitions and metrics are constructed to be applicable to
other network domains, if desired. The methods of measurement to assess
the path may not be able to discover all hosts comprising the path, but
such omissions are often deterministic and explainable sources of
error.Also, to specify a framework for active methods of measurement which
use the techniques described in at a minimum, and a
framework for hybrid active-passive methods of measurement, such as the
Hybrid Type I method described in (intended only for single
administrative domains), which do not rely on ICMP and provide a
protocol for explicit interrogation of nodes on a path. Combinations of
active methods and hybrid active-passive methods are also in-scope.Further, this memo provides additional analysis of the round-trip
delay measurements made possible by the methods, in an effort to
discover more details about the path, such as the link technology in
use.This memo updates Section 5 of in the areas
of path-related terminology and path description, primarily to include
the possibility of parallel subpaths between a given Source and
Destination address pair (possibly resulting from Equal Cost Multi-Path
(ECMP) and Unequal Cost Multi-Path (UCMP) technologies).There are several simple non-goals of this memo. There is no attempt
to assess the reverse path from any host on the path to the host
attempting the path measurement. The reverse path contribution to delay
will be that experienced by ICMP packets (in active methods), and may be
different from delays experienced by UDP or TCP packets. Also, the round
trip delay will include an unknown contribution of processing time at
the host that generates the ICMP response. Therefore, the ICMP-based
active methods are not supposed to yield accurate, reproducible
estimations of the Round-Trip Delay that UDP or TCP packets will
experience.This section sets requirements for the following components to
support the Route Metric:A Host as defined in (a
computer capable of IP communication, includes routers), a.k.a. RFC
2330 Host.A Node is any network function on the path
capable of IP-layer Communication, includes RFC 2330 Hosts.The unique address for Nodes
communicating within the network domain. For Nodes communicating on
the Internet with IP, it is the globally routable IP address(es)
which the Node uses when communicating with other Nodes under normal
or error conditions. The Node Identity revealed (and its connection
to a Node Name through reverse DNS) determines whether interfaces to
parallel links can be associated with a single Node, or appear to
identify unique Nodes.Nodes that convey their Node
Identity according to the requirements of their network domain, such
as when error conditions are detected by that Node. For Nodes
communicating with IP packets, compliance with Section 3.2.2.4 of
when discarding a packet due to TTL or Hop
Limit Exceeded condition, MUST result in sending the corresponding
Time Exceeded message (containing a form of Node identity) to the
source. This requirement is also consistent with section 5.3.1 of
for routers.Nodes that MUST respond to direct
queries for their Node identity as part of a previously agreed and
established interrogation protocol. Nodes SHOULD also provide
information such as arrival/departure interface identification,
arrival timestamp, and any relevant information about the Node or
specific link which delivered the query to the Node.A Hop MUST contain a Node Identity, and MAY
contain arrival and/or departure interface identification, round
trip delay, and an arrival timestamp.A route that treats equally a class C of
different types of packets (unrelated to address classes of the
past). Knowledge of such a class allows any one of the types of
packets within that class to be used for subsequent measurement of
the route.Type-P-Route-Ensemble-Method-Variant, abbreviated as Route
Ensemble.Note that Type-P depends heavily on the chosen method and
variant.This section lists the REQUIRED input factors to specify a Route
metric.Src, the address of a Node (such as the globally routable IP
address).Dst, the address of a Node (such as the globally routable IP
address).i, the limit on the number of Hops a specific packet may visit
as it traverses from the Node at Src to the Node at Dst (such as
the TTL or Hop Limit).MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops).T0, a time (start of measurement interval)Tf, a time (end of measurement interval)MP(address), Measurement Point at address, such as Src or Dst,
usually at the same node stack layer as "address".T, the Node time of a packet as measured at MP(Src), meaning
Measurement Point at the Source.Ta, the Node time of a reply packet's *arrival* as measured at
MP(Src), assigned to packets that arrive within a "reasonable"
time (see parameter below).Tmax, a maximum waiting time for reply packets to return to the
source, set sufficiently long to disambiguate packets with long
delays from packets that are discarded (lost), such that the
distribution of Round-Trip Delay is not truncated.F, the number of different flows simulated by the method and
variant.flow, the stream of packets with the same n-tuple of designated
header fields that (when held constant) result in identical
treatment in a multi-path decision (such as the decision taken in
load balancing). Note: The IPv6 flow label MAY be included in the
flow definition if the MP(Src) is a Tunnel End Point (TEP)
complying with guidelines.Type-P, the complete description of the packets for which this
assessment applies (including the flow-defining fields).This section defines the REQUIRED measurement components of the
Route metrics (unless otherwise indicated):M, the total number of packets sent between T0 and Tf.N, the smallest value of i needed for a packet to be received at
Dst (sent between T0 and Tf).Nmax, the largest value of i needed for a packet to be received at
Dst (sent between T0 and Tf). Nmax may be equal to N.Next define a *singleton* definition for a Hop on the path, with
sufficient indexes to identify all Hops identified in a measurement
interval.A Hop, designated h(i,j), the IP address and/or identity of
Discoverable Nodes (or Cooperating Nodes) that are i hops away from
the Node with address = Src and part of Route j during the measurement
interval, T0 to Tf. As defined here, a Hop singleton measurement MUST
contain a Node Identity, hid(i,j), and MAY contain one or more of the
following attributes:a(i,j) Arrival Interface ID (e.g., when is supported)d(i,j) Departure Interface ID (e.g., when is supported)t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by
the Hop, or approximated from the sending time of the packet that
revealed the Hop)Measurements of Round-Trip Delay (for each packet that reveals
the same Node Identity and flow attributes, then this attribute is
computed, see next section)Node Identities and related information can be ordered by their
distance from the Node with address Src in Hops h(i,j). Based on this,
two forms of Routes are distinguished:A Route Ensemble is defined as the combination of all routes
traversed by different flows from the Node at Src address to the Node
at Dst address. A single Route traversed by a single flow (determined
by an unambiguous tuple of addresses Src and Dst, and other identical
flow criteria) is a member of the Route Ensemble and called a Member
Route.Using h(i,j) and components and parameters, further define:When considering the set of Hops in the context of a single flow, a
Member Route j is an ordered list {h(1,j), ... h(Nj, j)} where h(i-1,
j) and h(i, j) are 1 hop away from each other and Nj satisfying
h(Nj,j)=Dst is the minimum count of Hops needed by the packet on
Member Route j to reach Dst. Member Routes must be unique. The
uniqueness property requires that any two Member routes j and k that
are part of the same Route Ensemble differ either in terms of minimum
hop count Nj and Nk to reach the destination Dst, or, in the case of
identical hop count Nj=Nk, they have at least one distinct Hop: h(i,j)
!= h(i,k) for at least one i (i=1..Nj).All the optional information collected to describe a Member Route,
such as the arrival interface, departure interface, and Round Trip
Delay at each Hop, turns each list item into a rich structure. There
may be information on the links between Hops, possibly information on
the routing (arrival interface and departure interface), an estimate
of distance between Hops based on Round-Trip Delay measurements and
calculations, and a time stamp indicating when all these additional
details were valid.The Route Ensemble from Src to Dst, during the measurement interval
T0 to Tf, is the aggregate of all m distinct Member Routes discovered
between the two Nodes with Src and Dst addresses. More formally, with
the Node having address Src omitted:where the following conditions apply: i <= Nj <= Nmax
(j=1..m)Note that some h(i,j) may be empty (null) in the case that systems
do not reply (not discoverable, or not cooperating).h(i-1,j) and h(i,j) are the Hops on the same Member Route one hop
away from each other.Hop h(i,j) may be identical with h(k,l) for i!=k and j!=l ; which
means there may be portions shared among different Member Routes
(parts of Member Routes may overlap).RTD(i,j,T) is defined as a singleton of the Round-Trip Delay between the Node with address =
Src and the Node at Hop h(i,j) at time T.RTL(i,j,T) is defined as a singleton of the Round-trip Loss between the Node with address = Src
and the Node at Hop h(i,j) at time T.Depending on the way that Node Identity is revealed, it may be
difficult to determine parallel subpaths between the same pair of
Nodes (i.e. multiple parallel links). It is easier to detect parallel
subpaths involving different Nodes.If a pair of discovered Nodes identify two different addresses,
then they will appear to be different Nodes.If a pair of discovered Nodes identify two different IP
addresses, and the IP addresses resolve to the same Node name (in
the DNS), then they will appear to be the same Nodes.If a discovered Node always replies using the same network
address, regardless of the interface a packet arrives on, then
multiple parallel links cannot be detected in that network domain.
This condition may apply to traceroute-style methods, but may not
apply to other hybrid methods based on In-situ Operations,
Administration, and Maintenance (IOAM).If parallel links between routers are aggregated below the IP
layer, then from Node point of view, all these links share the
same pair of IP addresses. The existence of these parallel links
can't be detected at IP layer. This applies to other network
domains with layers below them, as well. This condition may apply
to traceroute-style methods, but may not apply to other hybrid
methods based on IOAM.When a route assessment employs IP packets (for example), the
reality of flow assignment to parallel subpaths involves layers above
IP. Thus, the measured Route Ensemble is applicable to IP and higher
layers (as described in the methodology's packet of Type-P and flow
parameters).An Information Model and an XML Data Model for Storing Traceroute
Measurements is available in . The measured
information at each hop includes four pieces of information: a
one-dimensional hop index, Node symbolic address, Node IP address, and
RTD for each response.The description of Hop information that may be collected according
to this memo covers more dimensions, as defined in Section 3.3 above.
For example, the Hop index is two-dimensional to capture the
complexity of a Route Ensemble, and it contains corresponding Node
identities at a minimum. The models need to be expanded to include
these features, as well as Arrival Interface ID, Departure Interface
ID, and Arrival Timestamp, when available. The original sending
Timestamp from the Src Node anchors a particular measurement in
time.There are two classes of methods described in this section, active
methods relying on the reaction to TTL or Hop Limit Exceeded condition
to discover Nodes on a path, and Hybrid active-passive methods that
involve direct interrogation of cooperating Nodes (usually within a
single domain). Description of these methods follow.This section describes the method employed by current open source
tools, thereby providing a practical framework for further advanced
techniques to be included as method variants. This method is
applicable for use across multiple administrative domains.Internet routing is complex because it depends on the policies of
thousands of Autonomous Systems (AS). While most of the routers
perform load balancing on flows using Equal Cost Multiple Path (ECMP),
a few still divide the workload through packet-based techniques. The
former scenario is defined according to ,
while the latter generates a round-robin scheme to deliver every new
outgoing packet. ECMP uses a hashing function to ensure that every
packet of a flow is delivered by the same path, and this avoids
increasing the packet delay variation and possibly producing
overwhelming packet reordering in TCP flows.Taking into account that Internet protocol was designed under the
“end-to-end” principle, the IP payload and its header do
not provide any information about the routes or path necessary to
reach some destination. For this reason, the popular tool traceroute
was developed to gather the IP addresses of each hop along a path
using the ICMP protocol . Traceroute also
measures RTD from each hop. However, the growing complexity of the
Internet makes it more challenging to develop an accurate traceroute
implementation. For instance, the early traceroute tools would be
inaccurate in the current network, mainly because they were not
designed to retain a flow state. However, evolved traceroute tools,
such as Paris-traceroute and
Scamper , expect to encounter ECMP and achieve
more accurate results when they do, where Scamper ensures traceroute
packets will follow the same path in 98% of cases.Today's traceroute tools send Type-P of packets, either ICMP, UDP,
or TCP. UDP and TCP are used when a particular characteristic needs to
be verified, such as filtering or traffic shaping on specific ports
(i.e., services). supports IPv6 traceroute
measurements, keeping the FlowLabel constant in all packets.Paris-traceroute allows its users to measure RTD in every hop of
the path for a particular flow. Furthermore, either Paris-traceroute
or Scamper is capable of unveiling the many available paths between a
source and destination (which are visible to this method). This task
is accomplished by repeating complete traceroute measurements with
different flow parameters for each measurement; Paris-traceroute
provides “exhaustive” mode while scamper provides
“tracelb” (stands for traceroute load balance). The
Framework for IP Performance Metrics (IPPM) (
updated by) has the flexibility to
require that the Round-Trip Delay measurement
uses packets with the constraints to assure that all packets in a
single measurement appear as the same flow. This flexibility covers
ICMP, UDP, and TCP. The accompanying methodology of needs to be expanded to report the sequential hop
identifiers along with RTD measurements, but no new metric definition
is needed.The advanced route assessment methods used in Paris-traceroute
keep the critical fields constant for every packet
to maintain the appearance of the same flow. In IPv6, it is sufficient
to be routed identically if the IP source and destination addresses
and the FlowLabel are constant, see . In IPv4,
certain fields of the IP header and the first four bytes of the IP
payload should remain constant in a flow. In the IPv4 header, the IP
source and destination addresses, protocol number, and Diffserv fields
identify flows. The first four payload bytes include the UDP and TCP
ports, and the ICMP type, code, and checksum fields.Maintaining a constant ICMP checksum in IPv4 is most challenging,
as the ICMP sequence number or identifier fields will usually change
for different probes of the same path. Probes should use arbitrary
bytes in the ICMP data field to offset changes to sequence number and
identifier, thus keeping the checksum constant.Finally, it is also essential to route the resulting ICMP Time
Exceeded messages along a consistent path. In IPv6, the fields above
are sufficient. In IPv4, the ICMP Time Exceeded message will contain
the IP header and the first eight bytes of the IP payload, which
affects its ICMP checksum. The TCP sequence number, UDP Length, and
UDP checksum will affect this value, and should remain constant.Formally, to maintain the same flow in the measurements to a
particular hop, the Type-P-Route-Ensemble-Method-Variant packets
should be:TCP case: For IPv4, the fields Src, Dst, port-Src, port_Dst,
sequence number, and Diffserv Field SHOULD be the same. For IPv6,
the field FlowLabel, Src and Dst SHOULD be the same.UDP case: For IPv4, the fields Src, Dst, port-Src, port-Dst,
Diffserv should be the same, and the UDP-checksum SHOULD change to
keep the IP checksum of the ICMP time exceeded reply constant.
Then, the data length should be fixed, and the data field is used
to make it so (consider that ICMP checksum uses its data field,
which contains the original IP header plus 8 bytes of UDP, where
TTL, IP identification, IP checksum, and UDP checksum changes).
For IPv6, the field FlowLabel, and Source and Destination
addresses SHOULD be the same.ICMP case: For IPv4, the Data field SHOULD compensate
variations on TTL or Hop Limit, IP identification, and IP checksum
for every packet. There is no need to consider ICMPv6 because only
FlowLabel of IPv6 and Source and Destination addresses are used,
and all of them SHOULD be constant.Then, the way to identify different hops and attempts of the same
flow is:TCP case: The IP identification field.UDP case: The IP identification field.ICMP case: The IP identification field, and ICMP Sequence
number.The Active Route Assessment Methods described above have the
ability to discover portions of a path where ECMP load balancing is
present, observed as two or more unique Member Routes having one or
more distinct Hops which are part of the Route Ensemble. Likewise,
attempts to deliberately vary the flow characteristics to discover
all Member Routes will reveal portions of the path which are
flow-invariant.Section 9.2 of describes Temporal
Composition of metrics, and introduces the possibility of a
relationship between earlier measurement results and the results for
measurement at the current time (for a given metric). There is value
in establishing a Temporal Composition relationship for Route
Metrics. However, this relationship does not represent a forecast of
future route conditions in any way.For Route Metric measurements, the value of Temporal Composition
is to reduce the measurement iterations required with repeated
measurements. Reduced iterations are possible by inferring that
current measurements using fixed and previously measured flow
characteristics:will have many common hops with previous measurements.will have relatively time-stable results at the ingress and
egress portions of the path when measured from user locations,
as opposed to measurements of backbone networks and across
inter-domain gateways.may have greater potential for time-variation in path
portions where ECMP load balancing is observed (because
increasing or decreasing the pool of links changes the hash
calculations).Optionally, measurement systems may take advantage of the
inferences above when seeking to reduce measurement iterations,
after exhaustive measurements indicate that the time-stable
properties are present. Repetitive Active Route measurement
systems:SHOULD occasionally check path portions which have exhibited
stable results over time, particularly ingress and egress
portions of the path.SHOULD continue testing portions of the path that have
previously exhibited ECMP load balancing.SHALL trigger re-assessment of the complete path and Route
Ensemble, if any change in hops is observed for a specific (and
previously tested) flow.There is an opportunity to apply the
notion of equal treatment for a class of packets, "...very useful to
know if a given Internet component treats equally a class C of
different types of packets", as it applies to Route measurements.
The notion of class C was examined further in as it applied to load-balancing flows over
parallel paths, which is the case we develop here. Knowledge of
class C parameters (unrelated to address classes of the past) on a
path potentially reduces the number of flows required for a given
method to assess a Route Ensemble over time.First, recognize that each Member Route of a Route Ensemble will
have a corresponding class C. Class C can be discovered by testing
with multiple flows, all of which traverse the unique set of hops
that comprise a specific Member Route.Second, recognize that the different classes depend primarily on
the hash functions used at each instance of ECMP load balancing on
the path.Third, recognize the synergy with Temporal Composition methods
(described above), where evaluation intends to discover time-stable
portions of each Member Route, so that more emphasis can be placed
on ECMP portions that also determine class C.The methods to assess the various class C characteristics benefit
from the following measurement capabilities:flows designed to determine which n-tuple header fields are
considered by a given hash function and ECMP hop on the path,
and which are not. This operation immediately narrows the search
space, where possible, and partially defines a class C.a priori knowledge of the possible types of hash functions in
use also helps to design the flows for testing (major router
vendors publish information about these hash functions, examples
are here .ability to direct the emphasis of current measurements on
ECMP portions of the path, based on recent past measurement
results (the Routing Class of some portions of the path is
essentially "all packets").There are many examples where passive monitoring of a flow at an
Observation Point within the network can detect unexpected Round
Trip Delay or Delay Variation. But how can the cause of the
anomalous delay be investigated further --from the Observation Point
-- possibly located at an intermediate point on the path?In this case, knowledge that the flow of interest belongs to a
specific Routing Class C will enable measurement of the route where
anomalous delay has been observed. Specifically, Round-Trip Delay
assessment to each Hop on the path between the Observation Point and
the Destination for the flow of interest may discover high or
variable delay on a specific link and Hop combination.The determination of a Routing Class C which includes the flow of
interest is as described in the section above, aided by computation
of the relevant hash function output as the target.The Hybrid Type I methods provide an alternative method for Route
Member assessment. As mentioned in the Scope section, provides a possible set of data
fields that would support route identification.In general, nodes in the measured domain would be equipped with
specific abilities:Store the identity of nodes that a packet has visited in header
data fields, in the order the packet visited the nodes.Support of a "Loopback" capability, where a copy of the packet
is returned to the encapsulating node, and the packet is processed
like any other IOAM packet on the return transfer.In addition to node identity, nodes may also identify the ingress
and egress interfaces utilized by the tracing packet, the time of day
when the packet was processed, and other generic data (as described in
section 4 of ). Interface
identification isn't necessarily limited to IP, i.e. different links
in a bundle (LACP) could be identified. Equally well, links without
explicit IP addresses can be identified (like with unnumbered
interfaces in an IGP deployment).Note that the Type-P packet specification for this method will
likely be a partial specification, because most of the packet fields
are determined by the user traffic. The packet (encapsulation)
header(s) added by the Hybrid method can certainly be specified in
Type-P, in unpopulated form.In principle, there are advantages if the entity conducting Route
measurements can utilize both forms of advanced methods (active and
hybrid), and combine the results. For example, if there are Nodes
involved in the path that qualify as Cooperating Nodes, but not as
Discoverable Nodes, then a more complete view of Hops on the path is
possible when a hybrid method (or interrogation protocol) is applied
and the results are combined with the active method results collected
across all other domains.In order to combine the results of active and hybrid/interrogation
methods, the network Nodes that are part of a domain supporting an
interrogation protocol have the following attributes: Nodes at the ingress to the domain SHOULD be both Discoverable
and Cooperating, and SHOULD reveal the same Node Identity in
response to both active and hybrid methods.Any Nodes within the domain that are both Discoverable and
Cooperating SHOULD reveal the same Node Identity in response to
both active and hybrid methods.Nodes at the egress to the domain SHOULD be both Discoverable
and Cooperating, and SHOULD reveal the same Node Identity in
response to both active and hybrid methods.When Nodes follow these requirements, it becomes a simple matter to
match single domain measurements with the overlapping results from a
multidomain measurement.In practice, Internet users do not typically have the ability to
utilize the OAM capabilities of networks that their packets traverse,
so the results from a remote domain supporting an interrogation
protocol would not normally be accessible. However, a network operator
could combine interrogation results from their access domain with
other measurements revealing the path outside their domain.The aim of this method is to use packet probes to unveil the paths
between any two end-Nodes of the network. Moreover, information derived
from RTD measurements might be meaningful to identify:Intercontinental submarine linksSatellite communicationsCongestionInter-domain pathsThis categorization is widely accepted in the literature and among
operators alike, and it can be trusted with empirical data and several
sources as ground of truth (e.g., ) but it is an
inference measurement nonetheless .The first two categories correspond to the physical distance
dependency on Round-Trip Delay (RTD), the next one binds RTD with
queuing delay on routers, and the last one helps to identify different
ASes using traceroutes. Due to the significant contribution of
propagation delay in long-distance hops, RTD will be on the order of
100ms on transatlantic hops, depending on the geolocation of the vantage
points. Moreover, RTD is typically higher than 480ms when two hops are
connected using geostationary satellite technology (i.e., their orbit is
at 36000km). Detecting congestion with latency implies deeper
mathematical understanding since network traffic load is not stationary.
Nonetheless, as the first approach, a link seems to be congested if,
after sending several traceroute probes, it is possible to detect
congestion observing different statistics parameters (e.g., see ). Finally, to recognize distinctive ASes in the same
traceroute path is challenging, because more data is needed, like AS
relationships and RIR delegations among other (for more detail, please
consult ).Several articles have shown that network traffic presents a
self-similar nature which is
accountable for filling the queues of the routers. Moreover, router
queues are designed to handle traffic bursts, which is one of the most
remarkable features of self-similarity. Naturally, while queue length
increases, the delay to traverse the queue increases as well and leads
to an increase on RTD. Due to traffic bursts generating short-term
overflow on buffers (spiky patterns), every RTD only depicts the
queueing status on the instant when that packet probe was in transit.
For this reason, several RTD measurements during a time window could
begin to describe the random behavior of latency. Loss must also be
accounted for in the methodology.To understand the ongoing process, examining the quartiles provides a
non-parametric way of analysis. Quartiles are defined by five values:
minimum RTD (m), RTD value of the 25% of the Empirical Cumulative
Distribution Function (ECDF) (Q1), the median value (Q2), the RTD value
of the 75% of the ECDF (Q3) and the maximum RTD (M). Congestion can be
inferred when RTD measurements are spread apart, and consequently, the
Inter-Quartile Range (IQR), the distance between Q3 and Q1, increases
its value.This procedure requires the algorithm presented in to compute quartile values "on the fly”.This procedure allows us to update the quartiles value whenever a new
measurement arrives, which is radically different from classic methods
of computing quartiles because they need to use the whole dataset to
compute the values. This way of calculus provides savings in memory and
computing time.To sum up, the proposed measurement procedure consists of performing
traceroutes several times to obtain samples of the RTD in every hop from
a path, during a time window (W), and compute the quartiles for every
hop. This procedure could be done for a single Member Route flow, with
parameter E set as False, or for every detected Route Ensemble flow
(E=True).The identification of a specific Hop in traceroute is based on the IP
origin address of the returned ICMP Time Exceeded packet, and on the
distance identified by the value set in the TTL field inserted by
traceroute. As this specific Hop can be reached by different paths, also
the IP source and destination addresses of the traceroute packet need to
be recorded. Finally, different return paths are distinguished by
evaluating the ICMP Time Exceeded TTL (of the reply message): if this
TTL is constant for different paths containing the same Hop, the return
paths have the same distance. Moreover, this distance can be estimated
considering that the TTL value is normally initialized with values 64,
128, or 255. The 5-tuple (origin IP, destination IP, reply IP, distance,
response TTL) unequivocally identifies every measurement.This algorithm below runs in the origin of the traceroute. It returns
the Qs quartiles for every Hop and Alt (alternative paths because of
balancing). Notice that the "Alt" parameter condenses the parameters of
the 5-tuple (origin IP, destination IP, reply IP, distance, response
TTL), i.e., one for each possible combination.During the time W, lines 6 and 7 assure that the measurement loop is
made. Line 8 and 13 set a timer for each cycle of measurements. A cycle
comprises the traceroutes packets, considering every possible Hop and
the alternatives paths in the Alt variable (ensured in lines 9-12). In
line 9, the advance-traceroute could be either Paris-traceroute or
Scamper, which will use the “exhaustive” mode or
“tracelb” option if E is set True, respectively. The
procedure returns a list of tuples (m,Q1,Q2,Q3,M) for each intermediate
hop, or "Alt" in as a function of the 5-tuple, in the path towards the
Dst. Finally, lines 10 through 12 stores each measurement into the
real-time quartiles computation.Notice there are cases where the even having a unique hop at distance
h from the Src to Dst, the returning path could have several
possibilities, yielding in different total paths. In this situation, the
algorithm will return more "Alt" for this particular hop.The security considerations that apply to any active measurement of
live paths are relevant here as well. See and
.The active measurement process of "changing several fields to keep
the checksum of different packets identical" does not require special
security considerations because it is part of synthetic traffic
generation, and is designed to have minimal to zero impact on network
processing (to process the packets for ECMP).For applicable Hybrid methods, the security considerations in apply.When considering privacy of those involved in measurement or those
whose traffic is measured, the sensitive information available to
potential observers is greatly reduced when using active techniques
which are within this scope of work. Passive observations of user
traffic for measurement purposes raise many privacy issues. We refer the
reader to the privacy considerations described in the Large Scale
Measurement of Broadband Performance (LMAP) Framework , which covers active and passive techniques.This memo makes no requests of IANA. We thank the good folks at IANA
for having checked this section anyway.The original 3 authors acknowledge Ruediger Geib, for his penetrating
comments on the initial draft, and his initial text for the Appendix on
MPLS. Carlos Pignataro challenged the authors to consider a wider scope,
and applied his substantial expertise with many technologies and their
measurement features in his extensive comments. Frank Brockners also
shared useful comments, so did Footer Foote. We thank them all!A Node assessing an MPLS path must be part of the MPLS domain where
the path is implemented. When this condition is met, RFC 8029 provides a
powerful set of mechanisms to detect “correct operation of the
data plane, as well as a mechanism to verify the data plane against the
control plane” .MPLS routing is based on the presence of a Forwarding Equivalence
Class (FEC) Stack in all visited Nodes. Selecting one of several Equal
Cost Multi Path (ECMP) is however based on information hidden deeper in
the stack. Late deployments may support a so called "Entropy label" for
this purpose. State of the art deployments base their choice of an ECMP
member interface on the complete MPLS label stack and on IP addresses up
to the complete 5 tuple IP header information (see Section 2.4 of ). Load Sharing based on IP information decouples this
function from the actual MPLS routing information. Thus, an MPLS
traceroute is able to check how packets with a contiguous number of ECMP
relevant IP addresses (and an identical MPLS label stack) are forwarded
by a particular router. The minimum number of equivalent MPLS paths
traceable at a router should be 32. Implementations supporting more
paths are available.The MPLS echo request and reply messages offering this feature must
support the Downstream Detailed Mapping TLV (was Downstream Mapping
initially, but the latter has been deprecated). The MPLS echo response
includes the incoming interface where a router received the MPLS Echo
request. The MPLS Echo reply further informs which of the n addresses
relevant for the load sharing decision results in a particular next hop
interface and contains the next hop’s interface address (if
available). This ensures that the next hop will receive a properly coded
MPLS Echo request in the next step route of assessment. explains how a central Path Monitoring
System could be used to detect arbitrary MPLS paths between any routers
within a single MPLS domain. The combination of MPLS forwarding, Segment
Routing and MPLS traceroute offers a simple architecture and a powerful
mechanism to detect and validate (segment routed) MPLS paths.Avoiding traceroute anomalies with Paris tracerouteCOMPARISON OF HASH STRATEGIES FOR FLOW-BASED LOAD
BALANCINGMeasuring load-balanced paths in the InternetScamper: a scalable and extensible packet prober for active
measurement of the InternetSelf-Similar Network Traffic and Performance Evaluation (1st
ed.)An empirical mixture model for large-scale RTT
measurementsThe P 2 algorithm for dynamic calculation of quartiles and
histograms without storing observationsChallenges in inferring Internet interdomain
congestionbdrmap: Inference of Borders Between IP
Networksbdrmap: Inference of Borders Between IP NetworksIn and out of Cuba: Characterizing Cuba's
connectivity