Internet Draft A. DeKok Expiration: March 2003 IDT, Inc. Working Group: Forces October, 2003 A Discussion of Metadata in ForCES draft-dekok-forces-metadata-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document describes a model for discussing metadata in ForCES. It defines multiple kinds of metadata, and shows which ones are in scope for ForCES, and which ones are out of scope for ForCES. It further defines a vocabulary for discussing metadata, which should help to avoid much of the historical confusion and disagreement surrounding discussions of metadata. Table of Contents Abstract...........................................................1 1. Introduction ...................................................2 2. An exposition of metadata ......................................2 2.1 Internal versus External Metadata ...........................2 2.2 Implicit versus Explicit Metadata ...........................3 3. Further Exposition .............................................3 3.1 More detailed examples ......................................3 3.2 The concept of "scale" as it impacts metadata ...............4 4. Implication for ForCES .........................................4 Internet Draft A Discussion of Metadata in ForCES [Page 1] Internet draft March 2003 5. Security Considerations ........................................5 6. Intellectual Property Right ....................................5 7. Normative References ...........................................5 8. Informative References .........................................5 9. Acknowledgments ................................................5 10. Authors' Addresses ............................................6 1. Introduction Metadata has historically been understood to mean "data about data". While this definition is better than nothing, it has contributed significantly to misunderstanding and miscommunication, when disparate parties attempt to discuss the topic. Our goal in this document is to give a more detailed definition of metadata, to show where different kinds of metadata occur in network devices, and to explain those differences via specific examples. 2. An Exposition of Metadata Our presentation of metadata divides it by two orthogonal axes: Internal versus External, and Implicit versus Explicit. 2.1 Internal versus External Metadata We define Internal metadata as data which is produced and consumed entirely within a particular context (usually network device or chip). For example, an Ethernet MAC may have an internal timer associated with a packet it is trying to transmit. If a collision is detected during transmission, the MAC performs an exponential back-off, and attempts to re-transmit the packet. The timer associated with that exponential back-off is internal to the MAC, and usually cannot be seen by any driver software. Nevertheless, that timing data is in the MAC, and is associated with the current packet. We can therefore describe that timing data as metadata about the current packet. We define External metadata as data which is visible outside of a particular context (usually network device or chip). For example, a chip implementing an 8-port switch fabric may need to be told the output port to which an input packet will be redirected, if it does not make that decision itself. That information may be supplied via in-band data, such as a series of bits which precede the packet, or it may be supplied via other methods. In either case, the information which tells the switch fabric chip where to redirect the packet may be described as metadata, which is external to the switch fabric chip. Internet Draft A Discussion of Metadata in ForCES [Page 2] Internet draft March 2003 2.2 Implicit versus Explicit Metadata We define Implicit metadata as data which may be determined implicitely from context. For example, an Ethernet MAC is never told that the packets it receives are in Ethernet format. That information is determined implicitly by the fact that the physical form factor of Ethernet cabling ensures (generally) that only transmitters of Ethernet packets are connected to receivers of Ethernet packets. The receivers, therefore, are never told explicitly that the packets are in Ethernet format. Instead, they proceed under the assumption that the packets are Ethernet. We define Explicit metadata as data which is given as information in addition to the packet data. For example, a chip implementing an 8-port switch fabric may be given explicit information as to the output port to which an input packet will be redirected. That information often cannot be determined implicitly from context, but must be explicitly supplied by something external to the switch fabric chip. 3. Further exposition. The exposition of the axes used to describe metadata in Section 2 is admittedly somewhat naive. This section gives more detailed examples, and shows how the concept of "scale" impacts metadata. 3.1 More detailed examples TCP packets are carried within IP packets, and the IP packet contains a "protocol" field, with an explicit value of "6", for TCP. That protocol field can be viewed from the perspective of the TCP packet as Explicit, External metadata. In contrast, the format of the TCP packet is implicitely defined, and is never sent over the wire. Therefore, the TCP packet format can be viewed as Implicit, External metadata. IP MPLS can be viewed as associating 32 bits of Explicit, External metadata with an IP packet. That metadata informs MPLS-aware devices as to how the IP packet should be forwarded. The use of that metadata allows MPLS-aware devices to perform that forwarding without examining or modifying the IP packet. More examples should be inserted here. Internet Draft A Discussion of Metadata in ForCES [Page 3] Internet draft March 2003 3.2 The concept of "scale" as it impacts metadata The ForCES model document[2] describes how a network device may be modelled via multiple Function Elements (FE's), each of which may be modelled via multiple Logical Function Blocks (LFB's.) We now discuss how the view of metadata changes at each level of that model. The metadata exchanged between devices at one level of the model may be viewed as External, Explicit metadata at that level. For example, the metadata distributed between LFB's may include information such as classification result, which may be represented as a sequence of bits prefixing a packet. At a higher level, however, the lower-level metadata may not be visible. For example, an IPv4 forwarder may be represented as a set of LFB's which exchange packets and metadata. At the FE level, the metadata inside of the IPv4 forwarder may not be visible. The other FE's will simply see a packet entering the device, and some time later, a modified packet exiting the device. The discussion of scale may be repeated until it encompasses the Internet. Sets of LFB's form FE's, sets of FE's form network devices, sets of network devices form networks, and sets of networks form the Internet. At each level of scale, we can characterize metadata as in Section 2, even though that same metadata may have a different characterization at another scale. This multiple interpretation of metadata becomes even more difficult when we realize that different vendors have implementations which may exist at multiple scales, or at disparate scales. For example, one vendor may choose to implement an IPv4 forwarder in a monolithic chip, while another implements an IPv4 forwarder through a set of distinct chips. Both implementations must be controllable and addressable through ForCES, which is a difficult proposition to satisfy in general. 4. Implications for ForCES. The discussion of metadata within the context of ForCES has historically been contentious. We hope that this exposition of metadata helps to decrease the confusion surrounding metadata, by supplying a clear vocabulary which may be used to discuss the issue. Further, we can use the preceding characterization of metadata to narrow the scope of using metadata within the context of ForCES. Internet Draft A Discussion of Metadata in ForCES [Page 4] Internet draft March 2003 Specifically, Internal metadata should be described as outside of the scope of ForCES. Vendors of network devices should be free to choose any manner of implementing or representing metadata within their devices. Any attempt by ForCES to standardize the representation or use of Internal metadata would be an unwarranted burden on vendors. In contrast, as one of the goals of ForCES is to facilitate inter- vendor communication, External metadata is within the scope of ForCES. The group should create definitions of, and standards for, External metadata which permit different vendor implementations of devices to communicate metadata. That metadata may be Implicit (inputs to LFB 'X' are assumed to be outputs from LFB 'Y'), and Explicit (this packet is to go out port "4"). We hope that this discussion of metadata terminology, scope, and scale, helps in facilitating communication among members of ForCES. 5. Security Considerations There are no security considerations associated with this document. 6. Normative References [1] Khosravi, H. et al., "Requirements for Separation of IP Control and Forwarding", work in progress, draft-ietf-forces- requirements-10.txt, Intel Labs, July 2003. 7. Informative References [2] Yang, Lily et. al., "ForCES Forwarding Element Model", work in progress, draft-ietf-forces-model-01.txt, Intel Labs, October 2003. 8. Intellectual Property Right The authors are not aware of any intellectual property right issues pertaining to this document. 9. Acknowledgements The authors would also like to thank the following individuals for their valuable technical input: David Maxwell, Michael Richardson, Internet Draft A Discussion of Metadata in ForCES [Page 5] Internet draft March 2003 and Wojtek Fraczak. 10. Authors' Address Alan DeKok IDT Inc. 1575 Carling Ave. Ottawa, ON Canada K1Z 7M3 Email: alan.dekok@idt.com Internet Draft A Discussion of Metadata in ForCES [Page 6]