IP Flow Information Export S.K. Kashima
Internet-Draft NTT
Intended status: Standards Track A.K. Kobayashi
Expires: April 02, 2013 NTT East
P.A. Aitken
Cisco Systems, Inc.
October 2012

Information Elements for Data Link Layer Traffic Measurement
draft-ietf-ipfix-data-link-layer-monitoring-01

Abstract

This document describes Information Elements related to data link layer. They are used by the IP Flow Information Export (IPFIX) protocol for encoding measured data link layer traffic information.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http:/⁠/⁠datatracker.ietf.org/⁠drafts/⁠current/⁠.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on April 02, 2013.

Copyright Notice

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http:/⁠/⁠trustee.ietf.org/⁠license-⁠info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.


Table of Contents

1. Introduction

Ethernet [IEEE802.1D] and VLAN (Virtual LAN) [IEEE802.1Q-2005] technologies used to be used only in Local Area Networks. Recently, they have been used in Wide Area Networks, e.g., L2-VPN services. Accordingly, the IEEE802.1Q standard has been enhanced to IEEE802.1ad [IEEE802.1ad-2005] and IEEE802.1ah [IEEE802.1ah-2008]. And, Ethernet in data center also has been enhanced for server virtualization and I/O consolidation.

While these innovations provide flexibility, scalability, and mobility to an existing network architecture, it increases the complexity of traffic measurement due to the existence of various Ethernet header formats. To cope with this, a more sophisticated method is required.

IPFIX/PSAMP helps to resolve these problems. However, the PSAMP Information Model [RFC5477] and the IPFIX Information Model [RFC5101] are not yet enough for Information Elements related to data link layer, e.g., Ethernet header forms. This document describes the Information Elements related to data link layers that enable a more sophisticated traffic measurement method.

1.1. Conventions Used in This Document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

2. Extended Ethernet Technology

2.1. Wide-Area Ethernet Summary

Provider Bridge [IEEE802.1ad-2005] and Provider Backbone Bridge [IEEE802.1ah-2008], which are standards for the Wide-Area Ethernet, are described below.

Provider Backbone Bridge enables a wide-area Ethernet service to be improved from a flat network to a hierarchical network co-existing Provider Bridge and Provider Backbone Bridge.

Frame formats used in Wide-Area Ethernet are shown in Appendix A.

2.2. Data Center Bridging Summary

In data center networks, Ethernet needs to be enhanced to provide the flexibility, mobility for server virtualization, and I/O consolidation. In IEEE802.1 Data Center Bridging Task Group, several Ethernet header formats are proposed to enable a simplifying networks and server managements.

The one of the enhanced methods is Bridge Port Extension [IEEE802.1BR], which brings a traffic exchange point to upper bridges. Bridge Port Extension introduces a Ethernet format named Extension Tag (E-TAG) in addition to existing Service VLAN Tag (S-TAG) and Customer VLAN Tag (C-TAG) to move the policy enhancement to upper bridges in data center network. On the other hand, the complexity for traffic measurement would be increased, because multiple Ethernet header formats as shown in Appendix B co-exist in the same network.

2.3. Multiple Path Ethernet Summary

Inside data center networks, Ethernet need to be enhanced to support multiple path for effective utilization of bandwidth. Existing Ethernet does not support multiple path because it is based on STP (Spanning Tree Protocol) [IEEE802.1D].

Two solutions for multiple path Ethernet are discussed for standardization. One is Shortest Path Bridging [IEEE802.1aq] in IEEE 802, the other is TRILL (Transparent Interconnection of Lots of Links) [RFC6325] in IETF. Both solutions realize multiple path with IS-IS [ISO_IEC.10589_2002] and Layer 2 over Layer 2 encapsulation. Shortest Path Bridging uses IEEE802.1ah [IEEE802.1ah-2008] header as an encapsulation technology and TRILL uses an unique header as shown in Section 3 of [RFC6325].

In multiple path Ethernet network, a traffic measurement based on MAC address and VLAN Tag is more important than single path Ethernet network in which traffic measurement is based on only VLAN Tag.

2.4. VXLAN Summary

Inside and between data center networks, the existing Ethernet and VLAN technologies have two problems. One is that 12 bit of VLAN ID does not have enough length for identifying a lot of tenants. The other is that it is difficult to place same VLAN in multiple data centers.

The solution for the problems is VXLAN [I-D.mahalingam-dutt-dcops-vxlan]. VXLAN has 24 bit length of identifier named VXLAN Segment ID/VXLAN Network Identifier (VNI) and uses Layer 2 over Layer 3 encapsulation with the frame format as show in Section 5 of [I-D.mahalingam-dutt-dcops-vxlan].

In VXLAN networks, a traffic measurement based on multiple kinds of IP/MAC address and new VXLAN header is more important than existing Ethernet network in which traffic measurement is based on only VLAN Tag.

3. New Information Elements

The following Information Elements whose ElementId are from 312 to TBD03 are necessary for enabling the IPFIX/PSAMP traffic measurement for data link layer, which is not limited to Ethernet because the method can be applied to other data link protocols as well.

The following Information Elements whose ElementId are from TBD04 to TBD08 are necessary for enabling the IPFIX/PSAMP traffic measurement for IEEE 802.1ah.

The following Information Elements whose ElementId are TBD09 from to TBD22 are octet counter or packet length for layer 2, and are necessary for enabling the IPFIX/PSAMP traffic measurement for data link layer.

Note that 312 and 315 are proposed IDs, subject to approval by IANA.

+-----+------------------------------------+
| ID  | Name                               |
+-----+------------------------------------+
| 312 | dataLinkFrameSize                  |
| 315 | dataLinkFrameSection               |
|TBD01| dataLinkFrameType                  |
|TBD02| sectionOffset                      |
|TBD03| sectionObservedOctets              |
|TBD04| dot1qServiceInstanceTag            |
|TBD05| dot1qServiceInstanceId             |
|TBD06| dot1qServiceInstancePriority       |
|TBD07| dot1qCustomerDestinationMacAddress |
|TBD08| dot1qCustomerSourceMacAddress      |
|TBD09| l2octetDeltaCount                  |
|TBD10| postMCastL2OctetDeltaCount         |
|TBD11| postL2OctetDeltaCount              |
|TBD12| minimumL2TotalLength               |
|TBD13| maximumL2TotalLength               |
|TBD14| l2octetTotalCount                  |
|TBD15| droppedL2OctetDeltaCount           |
|TBD16| droppedL2OctetTotalCount           |
|TBD17| ignoredL2OctetTotalCount           |
|TBD18| notSentL2OctetTotalCount           |
|TBD19| postL2OctetTotalCount              |
|TBD20| postMCastL2OctetTotalCount         |
|TBD21| l2octetDeltaSumOfSquares           |
|TBD22| l2octetTotalSumOfSquares           |
+-----+------------------------------------+

3.1. dataLinkFrameSize

Description:

Abstract Data Type: unsigned16

Data Type Semantics: quantity

ElementId: 312

Status: current

3.2. dataLinkFrameSection

Description:

Abstract Data Type: octetArray

ElementId: 315

Status: current

3.3. dataLinkFrameType

Description:

Abstract Data Type: unsigned16

Data Type Semantics: identifier

ElementId: TBD01

Status: current

3.4. sectionOffset

Description:

Abstract Data Type: unsigned16

Data Type Semantics: quantity

ElementId: TBD02

Status: current

3.5. sectionObservedOctets

Description:

Abstract Data Type: unsigned16

Data Type Semantics: quantity

ElementId: TBD03

Status: current

3.6. dot1qServiceInstanceTag

Description:

Abstract Data Type: octetArray

Data Type Semantics: identifier

ElementId: TBD04

Status: current

3.7. dot1qServiceInstanceId

Description:

Abstract Data Type: unsigned32

Data Type Semantics: identifier

ElementId: TBD05

Status: current

3.8. dot1qServiceInstancePriority

Description:

Abstract Data Type: unsigned8

Data Type Semantics: identifier

ElementId: TBD06

Status: current

3.9. dot1qCustomerDestinationMacAddress

Description:

Abstract Data Type: macAddress

Data Type Semantics: identifier

ElementId: TBD07

Status: current

3.10. dot1qCustomerSourceMacAddress

Description:

Abstract Data Type: macAddress

Data Type Semantics: identifier

ElementId: TBD08

Status: current

3.11. l2OctetDeltaCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: deltaCounter

ElementId: TBD09

Status: current

Units: octets

3.12. postMCastL2OctetDeltaCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: deltaCounter

ElementId: TBD10

Status: current

Units: octets

3.13. postL2OctetDeltaCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: deltaCounter

ElementId: TBD11

Status: current

Units: octets

3.14. minimumL2TotalLength

Description:

Abstract Data Type: unsigned64

ElementId: TBD12

Status: current

Units: octets

3.15. maximumL2TotalLength

Description:

Abstract Data Type: unsigned64

ElementId: TBD13

Status: current

Units: octets

3.16. l2octetTotalCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD14

Status: current

Units: octets

3.17. droppedL2OctetDeltaCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: deltaCounter

ElementId: TBD15

Status: current

Units: octets

3.18. droppedL2OctetTotalCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD16

Status: current

Units: octets

3.19. ignoredL2OctetTotalCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD17

Status: current

Units: octets

3.20. notSentL2OctetTotalCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD18

Status: current

Units: octets

3.21. postL2OctetTotalCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD19

Status: current

Units: octets

3.22. postMCastL2OctetTotalCount

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD20

Status: current

Units: octets

3.23. l2octetDeltaSumOfSquares

Description:

Abstract Data Type: unsigned64

Data Type Semantics: deltaCounter

ElementId: TBD21

Status: current

Units: octets

3.24. l2octetTotalSumOfSquares

Description:

Abstract Data Type: unsigned64

Data Type Semantics: totalCounter

ElementId: TBD22

Status: current

Units: octets

4. Modification of Existing Information Elements Related to Packet Section

--- This is open issue. ---

New Information Elements related to packet section (ie, sectionOffset and sectionObservedOctets) can be applied to not only dataLinkFrameSection but also all kinds of packet section. In this case, existing Information Elements Description should be modified as follows:

4.1. ipHeaderPacketSection

Description:

Abstract Data Type: octetArray

ElementId: 313

Status: current

4.2. ipPayloadPacketSection

Description:

Abstract Data Type: octetArray

ElementId: 314

Status: current

4.3. mplsLabelStackSection

Description:

Abstract Data Type: octetArray

ElementId: 316

Status: current

4.4. mplsPayloadPacketSection

Description:

Abstract Data Type: octetArray

ElementId: 317

Status: current

5. Modification of Existing Information Elements Related to VLAN Tag

Information Elements related to Backbone Service Instance Tag (I-TAG) and Backbone VLAN Tag (B-TAG) are required in order to monitor IEEE802.1ah traffic with IPFIX/PSAMP. New Information Elements related to I-TAG are added in section 3 because I-TAG has different structure and semantics from Service VLAN Tag (S-TAG) and Customer VLAN Tag (C-TAG). On the other hand, Information Elements related to B-TAG reuse existing Information Elements related to C-TAG and S-TAG because B-TAG has same structure and semantics with C-TAG and S-TAG, Though they reuse existing Information Elements, it required to modify existing Descriptions and Reference as follows:

5.1. dot1qVlanId

Description:

Abstract Data Type: unsigned16

Data Type Semantics: identifier

ElementId: 243

Status: current

Reference:

(1)
[IEEE802.3-2005]
(2)
[IEEE802.1ad-2005]
(3)
[IEEE802.1ah-2008]

5.2. dot1qPriority

Description:

Abstract Data Type: unsigned8

Data Type Semantics: identifier

ElementId: 244

Status: current

Reference:

(1)
[IEEE802.3-2005]
(2)
[IEEE802.1ad-2005]
(3)
[IEEE802.1ah-2008]

5.3. dot1qCustomerVlanId

Description:

Abstract Data Type: unsigned16

Data Type Semantics: identifier

ElementId: 245

Status: current

Reference:

(1)
[IEEE802.1ad-2005]
(2)
[IEEE802.1Q-2005]
(3)
[IEEE802.1ah-2008]

5.4. dot1qCustomerPriority

Description:

Abstract Data Type: unsigned8

Data Type Semantics: identifier

ElementId: 246

Status: current

Reference:

(1)
[IEEE802.1ad-2005]
(2)
[IEEE802.1Q-2005]
(3)
[IEEE802.1ah-2008]

6. The relationship between Ethernet header fields and Information Elements

The following figures shows summary of various Ethernet header fields and the Informational Elements which would be used to represent each of the fields.


<----6----><----6----><----4----><------2------>
+----------+----------+----------+--------------+
+          +          +          +              +
+   C-DA   +   C-SA   +   C-TAG  +  Length/Type +
+    a     +    b     +     c    +       d      +
+----------+----------+----------+--------------+

a.(Existing Information Element) destinationMacAddress 80
b.(Existing Information Element) sourceMacAddress 56
c.(Existing Information Elements) dot1qVlanId 243, dot1qPriority 244
d.(Existing Information Element) ethernetType 256


<----6----><----6----><----4----><----4----><------2------>
+----------+----------+----------+----------+--------------+
+          +          +          +          +              +
+   C-DA   +   C-SA   +   S-TAG  +   C-TAG  +  Length/Type +
+    a     +    b     +     c    +     d    +       e      +
+----------+----------+----------+----------+--------------+

a.(Existing Information Element) destinationMacAddress 80
b.(Existing Information Element) sourceMacAddress 56
c.(Existing Information Elements) dot1qVlanId 243, dot1qPriority 244
d.(Existing Information Elements) dot1qCustomerVlanId 245, dot1qCustomerPriority 246
e.(Existing Information Element) ethernetType 256


<----6----><----6----><----4----><------18------><----4----><------2------>
+----------+----------+----------+---------------+----------+--------------+
+          +          +          +               +          +              +
+   B-DA   +   B-SA   +   B-TAG  +     I-TAG     +   C-TAG  +  Length/Type +
+    a     +    b     +     c    +       d       +     e    +       f      +
+----------+----------+----------+---------------+----------+--------------+

a.(Existing Information Element) destinationMacAddress 80
b.(Existing Information Element) sourceMacAddress 56
c.(Existing Information Elements) dot1qVlanId 243, dot1qPriority 244
d.(New Information Elements) defined in section 3.6, 3.7, 3.8, 3.9 and 3.10 of this draft
e.(Existing Information Elements) dot1qCustomerVlanId 245, dot1qCustomerPriority 246
f.(Existing Information Element) ethernetType 256

7. Security Considerations

The recommendations in this document do not introduce any additional security issues to those already mentioned in [RFC5101] and [RFC5477].

8. IANA Considerations

This document requests that the Information Element IDs are allocated as shown in section 3

In addition, the dataLinkFrameType Information Element requires the creation of new IANA registries.

And this document requests that the existing Information Element Description are modified as shown in section 4 and 5

9. References

9.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

9.2. Informative References

[IEEE802.3-2005] , , "IEEE Standard for Information Technology - Telecommunications and Information Exchange Between Systems - Local and Metropolitan Area Networks - Specific Requirements Part 3: Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications", IEEE Std 802.3-2005, December 2005.
[IEEE802.1D] , , "IEEE Standards for Local and Metropolitan Area Networks: Media Access Control (MAC) Bridges", IEEE Std 802.1D-2004, June 2004.
[IEEE802.1Q-2005] , , "IEEE Standards for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks", IEEE Std 802.1Q-2005, May 2006.
[IEEE802.1ad-2005] , , "IEEE Standards for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks Amendment 4: Provider Bridges", IEEE Std 802.1ad-2005, May 2006.
[IEEE802.1ah-2008] , , "IEEE Standards for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks Amendment 7: Provider Backbone Bridges", IEEE Std 802.1ah-2008, August 2008.
[IEEE802.1BR] , , "Draft Standards for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks Amendment: Bridge Port Extension", IEEE Std P802.1BR/D1.1, February 2011.
[IEEE802.1aq] , , "Draft Standards for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks Amendment: Shortest Path Bridging", IEEE Std P802.1aq/D3.6, February 2011.
[ISO_IEC.10589_2002] , , "Information technology -- Telecommunications and information exchange between systems -- Intermediate System to Intermediate System intra-domain routing information exchange protocol for use in conjunction with the protocol for providing the connectionless-mode network service (ISO 8473)", ISO Standard 10589:2002, November 2002.
[ISO_IEC.7498-1_1994] , , "Information technology -- Open Systems Interconnection -- Basic Reference Model: The Basic Mode", ISO Standard 7498-1:1994, June 1996.
[RFC5101] Claise, B., "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information", RFC 5101, January 2008.
[RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F. and G. Carle, "Information Model for Packet Sampling Exports", RFC 5477, March 2009.
[RFC6313] Claise, B., Dhandapani, G., Aitken, P. and S. Yates, "Export of Structured Data in IP Flow Information Export (IPFIX)", RFC 6313, July 2011.
[RFC6325] Perlman, R., Eastlake, D., Dutt, D., Gai, S. and A. Ghanwani, "Routing Bridges (RBridges): Base Protocol Specification", RFC 6325, July 2011.
[I-D.mahalingam-dutt-dcops-vxlan] Mahalingam, M, Dutt, D, Duda, K, Agarwal, P, Kreeger, L, Sridhar, T, Bursell, M and C Wright, "VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks", Internet-Draft draft-mahalingam-dutt-dcops-vxlan-02, August 2012.

Appendix A. Frame Formats in Wide-Area Ethernet Network


0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        C-TAG TCI=0x8100       |C-PCP|C|         C-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        S-TAG TCI=0x88a8       |S-PCP|D|         S-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        S-TAG TCI=0x88a8       |S-PCP|D|         S-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        C-TAG TCI=0x8100       |C-PCP|C|         C-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              B-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              B-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        B-TAG TCI=0x88a8       |B-PCP|D|         B-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        I-TAG TCI=0x88e7       |I-PCP|D|U| Res |     I-SID     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|             I-SID             |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-DA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-SA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |          Length/Type          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              B-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              B-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        B-TAG TCI=0x88a8       |B-PCP|D|         B-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        I-TAG TCI=0x88e7       |I-PCP|D|U| Res |     I-SID     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|             I-SID             |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-DA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-SA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |        C-TAG TCI=0x8100       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|C-PCP|C|         C-VID         |          Length/Type          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Appendix B. Frame Formats in Data Center Network

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        S-TAG TCI=0x88a8       |S-PCP|D|         S-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        S-TAG TCI=0x88a8       |S-PCP|D|         S-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        C-TAG TCI=0x8100       |C-PCP|C|         C-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        E-TAG TCI=0x893F       |E-PCP|D|   Ingress_E-CID_base  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Res|GRP|                       E-CID_base                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|       Ingress_E-CID_ext       |           E-CID_ext           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              C-DA                             |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                              C-SA                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        E-TAG TCI=0x893F       |E-PCP|D|   Ingress_E-CID_base  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Res|GRP|                       E-CID_base                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|       Ingress_E-CID_ext       |           E-CID_ext           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        C-TAG TCI=0x8100       |C-PCP|C|         C-VID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length/Type          |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
~                         Customer Data                         ~
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Appendix C. Template Formats Example

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        Set ID (0x0002)        |             Length            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|      Template ID (0x0103)     |      Field Count (0x0008)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   ingressInterface (0x000A)   |     Field Length (0x0004)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   egressInterface (0x000E)    |     Field Length (0x0004)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|observationTimeSeconds (0x0142)|     Field Length (0x0008)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   dataLinkFrameSize (0x0138)  |     Field Length (0x0002)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| dataLinkFrameSection (0x013B) |     Field Length (0xFF40)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   dataLinkFrameType (0x015B)  |     Field Length (0x0002)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     sectionOffset (0x015C)    |     Field Length (0x0002)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|sectionObservedOctets (0x015D) |     Field Length (0x0002)     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Authors' Addresses

Shingo Kashima NTT Information Sharing Platform Lab. Midori-Cho 3-9-11 Musashino-shi, Tokyo 180-8585 Japan Phone: +81 422 59 3894 EMail: kashima@nttv6.net
Atsushi Kobayashi NTT EAST IT Innovation Department 25F 3-20-2 Nishi-shinjuku Shinjuku-ku, Tokyo 163-1425 Japan Phone: +81 3 5353 3636 EMail: akoba@nttv6.net
Paul Aitken Cisco Systems, Inc. 96 Commercial Quay Commercial Street, Edinburgh EH6 6LX United Kingdom Phone: +44 131 561 3616 EMail: paitken@cisco.com