BIER-TE-based OAM, Replication and EliminationCisco SystemsVillage d'Entreprises Green Side400, Avenue de RoumanilleBatiment T3Biot - Sophia Antipolis06410FRANCE+33 4 97 23 26 34pthubert@cisco.comEcole PolytechniqueRoute de SaclayPalaiseau91128FRANCE+33 6 73 73 35 09zacharie.brodard@polytechnique.eduTelecom Bretagne2, rue de la Châtaigneraie Cesson-Sévigné35510FRANCE+33 7 53 70 97 34hao.jiang@telecom-bretagne.euDetNet
This specification leverages Bit Index Explicit Replication - Traffic
Engineering to control in the data plane the DetNet Replication and
Elimination activities, and to provide traceability on links where
replication and loss happen, in a manner that is abstract to the
forwarding information.
Deterministic Networking (DetNet) provides a capability to carry
unicast or multicast data flows for real-time applications with extremely
low data loss rates and known upper bound maximum latency .
DetNet applies to multiple environments where there is a desire to replace
a point to point serial cable or a multidrop bus by a switched or routed
infrastucture, in order to scale, lower costs, and simplify management.
One classical use case is found in particular in the context of the
convergence of IT with Operational Technology (OT), also referred to as
the Industrial Internet. But there are many others use cases
, for instance in
in professional audio and video, automotive, radio fronthauls, etc..
The DetNet data plane alternatives
studies the applicability of existing and emerging dataplane
techniques that can be leveraged to
enable DetNet properties in IP networks. One critical feature in
the dataplane is traceability, the capability to control the activity
of intermediate nodes on a packet. For instance, if Replication and Elimination
is applied to a packet, then it is desirable to determine which node performed
a certain copy of that packet that is circulating in the network.
Traceability belongs to Operations, Administration, and Maintenance (OAM)
which is the toolset for fault detection and isolation, and for performance
measurement. More can be found on OAM Tools in
"An Overview of Operations,
Administration and Maintenance (OAM) Tools".
This document proposes a new set to OAM tools based on
Bit Indexed Explicit Replication
(BIER) and more specifically
BIER Traffic Engineering
(BIER-TE) to control the process or Replication and Elimination, and provide
traceability of these operations, in the DetNet dataplane. An
adjacency, which is represented by a bit in the BIER header, can correspond
in the dataplane to an Ethernet hop, a Label Switched Path, or it can
correspond to an IPv6 loose or strict source routed path.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in .BIER
is a network plane replication technique that was initially
intended as a new method for multicast distribution. In a nutshell, a BIER
header includes a bitmap that explicitly signals the listeners that are
intended for a particular packet, which means that 1) the sender is aware of
the individual listeners and 2) the BIER control plane is a simple extension
of the unicast routing as opposed to a dedicated multicast data plane, which
represents a considerable reduction in OPEX. For this reason, the technology
faces a lot of traction from Service Providers.
The simplicity of the BIER technology makes it very versatile as a network
plane signaling protocol. Already, a new Traffic Engineering variation is
emerging that uses bits to signal segments along a TE path. While
BIER is mainly a multicast technology that typically leverages a
unicast distributed control plane through IGP extensions, BIER-TE
is mainly
a unicast technology that leverages a central computation to setup path,
compute segments and install the mapping in the intermediate nodes.
BIER-TE supports a Traffic Engineered forwarding plane
by explicit hop-by-hop forwarding and loose hop forwarding of packets.
From the BIER-TE architecture, the key differences over BIER are:
BIER-TE replaces in-network autonomous path calculation by
explicit paths calculated off path by the BIER-TE controller host.
In BIER-TE every BitPosition of the BitString of a BIER-TE packet
indicates one or more adjacencies - instead of a BFER as in BIER.
BIER-TE in each BFR has no routing table but only a BIER-TE
Forwarding Table (BIFT) indexed by SI:BitPosition and populated
with only those adjacencies to which the BFR should replicate
packets to.
The generic view of an adjacency can be over a link, a tunnel or along a
path segment.
With Segment Routing a
segment can be signaled as an MPLS label, or an IPv6 routing header . A
segment may be loosely of strictly source routed, depending on the need for
full non-congruence and the confidence that loose routing may still achieve
that need.
In a nutshell, BIER-TE is used as follows:
A controller computes a complex path, sometimes called a track, which takes
the general form of a ladder. The steps and the side rails between them
are the adjacencies that can be activated on demand on a per-packet basis
using bits in the BIER header.
The controller assigns a BIER domain, and inside that domain, assigns bits
to the adjacencies. The controller assigns each bit to a replication node
that sends towards the adjacency, for instance the ingress router into a
segment that will insert a routing header in the packet. A single bit may
be used for a step in the ladder, indicating the other end of the step in
both directions.
The controller activates the replication by deciding the setting of the
bits associated with the adjacencies. This decision can be modified at any
time, but takes the latency of a controller round trip to effectively take
place. Below is an example that uses Replication and Elimination to protect
the A->C adjacency.
Bit #AdjacencyOwnerExample Bit Setting1I->AI12A->BA1B->AB3I->CI04A->CA15B->DB16C->DC1D->CD7C->EC18D->ED0Replication and Elimination Protecting A->C
The BIER header with the controlling BitString is injected in the packet by
the ingress node of the deterministic path. That node may act as a
replication point, in which case it may issue multiple copies of the packet
For each of its bits that is set in the BIER header, the owner replication
point resets the bit and transmits towards the associated adjacency;
to achieve this, the replication point copies the packet and inserts the
relevant data plane information, such as a source route header, towards the
adjacency that corresponds to the bit
AdjacencyBIER BitStringI->A01011110A->B00011110B->D00010110D->C00010010A->C01001110BitString in BIER Header as Packet Progresses
Adversely, an elimination node on the way strips the data plane information
and performs a bitwise AND on the BitStrings from the various copies of the
packet that it has received, before it forwards the packet with the
resulting BitString.
OperationBIER BitStringD->C00010010A->C01001110--------AND in C00000010C->E00000000BitString Processing at Elimination Point C
In this example, all the transmissions succeeded and the BitString at
arrival has all the bits reset - note that the egress may be an Elimination
Point in which case this is evaluated after this node has performed its AND
operation on the received BitStrings).
Failing AdjacencyEgress BIER BitStringI->AFrame LostI->BNot TriedA->C00010000A->B01001100B->D01001100D->C01001100C->EFrame LostD->ENot TriedBitString indicating failures
But if a transmission failed along the way, one (or more) bit is never
cleared. provides the possible outcomes of a
transmission. If the frame is lost, then it is probably due to a failure in
either I->A or C->E, and the controller should enable I->B and D->E to
find out. A BitString of 00010000 indicates unequivocally a transmission
error on the A->C adjacency, and a BitString of 01001100 indicates a loss
in either A->B, B->D or D->C; enabling D->E on the next packets may provide
more information to sort things out.
In more details:
The BIER header is of variable size, and a DetNet network of a limited size
can use a model with 64 bits if 64 adjacencies are enough, whereas a larger
deployment may be able to signal up to 256 adjacencies for use
in very complex paths. The format of this header is common to
BIER and BIER-TE.
For the DetNet data plane, a replication point is an ingress point for more
than one adjacency, and an elimination point is an egress point for more than
one adjacency.
A pre-populated state in a replication node indicates which bits are
served by this node and to which adjacency each of these bits corresponds.
With DetNet, the state is typically installed by a controller entity such as
a PCE.
The way the adjacency is signaled in the packet is fully abstracted in the
bit representation and must be provisioned to the replication nodes and
maintained as a local state, together with the timing or shaping information
for the associated flow.
The DetNet data plane uses BIER-TE to control which adjacencies are used
for a given packet. This is signaled from the path ingress, which sets the
appropriate bits in the BIER BitString to indicate which replication must
happen.
The replication point clears the bit associated to the adjacency where the
replica is placed, and the elimination points perform a logical AND of the
BitStrings of the copies that it gets before forwarding.
As is apparent in the examples above, clearing the bits enables to trace a
packet to the replication points that made any particular copy. BIER-TE also
enables to detect the failing adjacencies or sequences of adjacencies along a
path and to activate additional replications to counter balance the failures.
Finally, using the same BIER-TE bit for both directions of the steps of the
ladder enables to avoid replication in both directions along the crossing
adjacencies. At the time of sending along the step of the ladder, the bit may
have been already reset by performing the AND operation with the copy from
the other side, in which case the transmission is not needed and does not
occur (since the control bit is now off).
BIER-TE occupies a particular position in the DetNet dataplane. In the one
hand it is optional, and only useful if replication and elimination is
taking place. In the other hand, it has unique capabilities to:
control which replication take place on a per packet basis, so that
replication points can be configured but not actually utilizedtrace the replication activity and determine which node replicated a
particular packetmeasure the quality of transmission of the actual data packet along
the replication segments and use that in a control loop to adapt the
setting of the bits and maintain the reliability. A research-stage implementation of the forwarding plane fir a 6TiSCH IOT use case
was developed at Cisco's Paris Innovation Lab (PIRL) by Zacharie Brodard.
It was implemented on OpenWSN Open-source firmware and tested on the OpenMote-CC2538
hardware. It implements the header types 15,16, 17, 18 and 19 (bit-by-bit encoding
without group ID) in order to allow a BIER-TE protocol over IEE802.15.4e.
This work was complemented with a Controller-based control loop by Hao Jiang.
The controller builds the complex paths (called Tracks in 6TiSCH) and decides
the setting oif the BitStrings in real time in order to optimize the delivery ratio
within a minimal energy budget.
Links:
github: https://github.com/zach-b/openwsn-fw/tree/BIER
OpenWSN firmware: https://openwsn.atlassian.net/wiki/pages/viewpage.action?pageId=688187
OpenMote hardware: http://www.openmote.com/
TBD.
This document has no IANA considerations.
The method presented in this document was discussed and worked out together with the DetNet Data Plane Design Team:
Jouni KorhonenJános FarkasNorman FinnOlivier MarceGregory MirskyPascal ThubertZhuangyan Zhuang
The authors also like to thank the DetNet chairs Lou Berger and Pat Thaler,
as well as Thomas Watteyne, 6TiSCH co-chair, for their contributions
and support to this work.