Network Working Group P. Calato Internet-Draft Riverstone Networks Inc Expires: May 21, 2004 J. Meyer Hewlett-Packard J. Quittek NEC Europe Ltd. November 21, 2003 Information Model for IP Flow Information Export draft-ietf-ipfix-info-02 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on May 21, 2004. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This document defines and information and data model for the IP Flow Information export (IPFIX) protocol. It is used by the IPFIX protocol for encoding measured traffic information and information related to the traffic measurement process. Although developed for the IPFIX protcol, the model is defined in an open way that easily allows using it in other protocols, interfaces, and applications. Calato, et al. Expires May 21, 2004 [Page 1] Internet-Draft IPFIX Information Model November 2003 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . 4 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Properties of an IPFIX Flow Attribute . . . . . . . . . . 6 4. Type Space . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1 int . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2 unsignedInt . . . . . . . . . . . . . . . . . . . . . . . 8 4.3 long . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.4 unsignedLong . . . . . . . . . . . . . . . . . . . . . . . 8 4.5 float . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.6 double . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.7 hexBinary . . . . . . . . . . . . . . . . . . . . . . . . 9 4.8 string . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.9 boolean . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.10 byte . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.11 unsignedByte . . . . . . . . . . . . . . . . . . . . . . . 9 4.12 short . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.13 unsignedShort . . . . . . . . . . . . . . . . . . . . . . 10 4.14 dateTime . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.15 ipdr:dateTimeMsec . . . . . . . . . . . . . . . . . . . . 10 4.16 ipdr:ipV4Addr . . . . . . . . . . . . . . . . . . . . . . 10 4.17 ipdr:ipV6Addr . . . . . . . . . . . . . . . . . . . . . . 10 4.18 ipdr:UUID . . . . . . . . . . . . . . . . . . . . . . . . 10 4.19 ipdr:dateTimeUsec . . . . . . . . . . . . . . . . . . . . 11 4.20 ipfix:dateTimeNsec . . . . . . . . . . . . . . . . . . . . 11 4.21 Integral Type Semantics . . . . . . . . . . . . . . . . . 11 4.21.1 Quantity . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.21.2 Counter . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.21.3 Identifier . . . . . . . . . . . . . . . . . . . . . . . . 12 4.21.4 Flags . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5. Extending the Information Model . . . . . . . . . . . . . 13 6. Flow Attributes . . . . . . . . . . . . . . . . . . . . . 14 6.1 sourceAddress . . . . . . . . . . . . . . . . . . . . . . 14 6.2 sourceAddressV6 . . . . . . . . . . . . . . . . . . . . . 14 6.3 destinationAddress . . . . . . . . . . . . . . . . . . . . 14 6.4 destinationAddressV6 . . . . . . . . . . . . . . . . . . . 14 6.5 protocolIdentifier . . . . . . . . . . . . . . . . . . . . 14 6.6 sourcePort . . . . . . . . . . . . . . . . . . . . . . . . 15 6.7 destinationPort . . . . . . . . . . . . . . . . . . . . . 15 6.8 ingressPort . . . . . . . . . . . . . . . . . . . . . . . 15 6.9 egressPort . . . . . . . . . . . . . . . . . . . . . . . . 16 6.10 packetCount . . . . . . . . . . . . . . . . . . . . . . . 16 6.11 byteCount . . . . . . . . . . . . . . . . . . . . . . . . 16 6.12 classOfService . . . . . . . . . . . . . . . . . . . . . . 17 6.13 flowLabel . . . . . . . . . . . . . . . . . . . . . . . . 17 6.14 flowCreationTime . . . . . . . . . . . . . . . . . . . . . 17 6.15 flowEndTime . . . . . . . . . . . . . . . . . . . . . . . 18 Calato, et al. Expires May 21, 2004 [Page 2] Internet-Draft IPFIX Information Model November 2003 6.16 sourceAS . . . . . . . . . . . . . . . . . . . . . . . . . 18 6.17 destinationAS . . . . . . . . . . . . . . . . . . . . . . 18 6.18 nextHopAS . . . . . . . . . . . . . . . . . . . . . . . . 18 6.19 tcpControlBits . . . . . . . . . . . . . . . . . . . . . . 19 6.20 ipV4SourceExporterAddress . . . . . . . . . . . . . . . . 19 6.21 ipV6SourceExporterAddress . . . . . . . . . . . . . . . . 19 6.22 droppedPacketCount . . . . . . . . . . . . . . . . . . . . 20 6.23 samplingInterval . . . . . . . . . . . . . . . . . . . . . 20 6.24 samplingAlgorithm . . . . . . . . . . . . . . . . . . . . 21 6.25 flowEndState . . . . . . . . . . . . . . . . . . . . . . . 21 6.26 droppedByteCount . . . . . . . . . . . . . . . . . . . . . 21 7. The Benefits of a Formal Machine Readable Information Model . . . . . . . . . . . . . . . . . . . . . . . . . . 22 8. Security Considerations . . . . . . . . . . . . . . . . . 23 References . . . . . . . . . . . . . . . . . . . . . . . . 24 Authors' Addresses . . . . . . . . . . . . . . . . . . . . 25 A. IPFIX IPDR Service Definition . . . . . . . . . . . . . . 26 B. Change History . . . . . . . . . . . . . . . . . . . . . . 37 B.1 Changes Since -01 Revision . . . . . . . . . . . . . . . . 37 B.2 Changes Since -00 Revision . . . . . . . . . . . . . . . . 39 Intellectual Property and Copyright Statements . . . . . . 41 Calato, et al. Expires May 21, 2004 [Page 3] Internet-Draft IPFIX Information Model November 2003 1. Introduction Many applications e.g., intrusion detection, traffic engineering, and accounting among others require the monitoring, measuring of IP traffic flows. It is hence important to have a standard way of exporting information related to IP flows. This document defines the base set of attributes which may be used when exporting IP flow information. It also defines the mechanism by which new data items may be added without changing the underlying exchange protocol. Calato, et al. Expires May 21, 2004 [Page 4] Internet-Draft IPFIX Information Model November 2003 2. Scope This document defines an information model for the IP Flow Information eXport (IPFIX) protocol. The model consists of of a set of information elements, each one defining a single attribute of a flow. For each individual attribute, the semantics is clearly specified and a data type is assigned to it. Calato, et al. Expires May 21, 2004 [Page 5] Internet-Draft IPFIX Information Model November 2003 3. Properties of an IPFIX Flow Attribute Flow attributes are modeled as information elements of the IPFIX information model. Each information element models a single flow attribute. For defining flow attributes, a template is used. Information elements defined in this specification, or by extension MUST have the following properties defined: Name - a unique and meaningful name for the field. The preferred spelling for the name is to use mixed case if the name is compound, with an initial lower case letter. (E.g. "sourceIpAddress"). Description - the semantics of this information element. Describes how this field is derived from the flow or other information available to the observer. Type - one of the types listed in the following section, "The Type Space". The type space for attributes is constrained to facilitate implementation. The existing type space does however encompass most basic types used in modern programming languages, as well as some derived types (such as IPAddress) which are common to this domain and useful to distinguish. FieldId - a numeric identifier administered by IANA. This is used for compact identification of an information item when encoding templates in the protocol. Information elements defined in this specification, or by extension MAY have the following properties defined: Vendor ID - when extension is done outside of the scope of the IANA IPFIX fieldId range, a vendorId MUST be provided. This identifier is based on IANA assigned enterprise identifiers. Reference - identifies additional specifications which more precisely define this item or provide additional context for its use. Units - if the field is a measure of some kind, the units identify what the measure is. Enumerated range - some items may have a specific set of numeric identifiers associated with a set of discrete values this element may take. The meaning of each discrete value and a human readable Calato, et al. Expires May 21, 2004 [Page 6] Internet-Draft IPFIX Information Model November 2003 name should be assigned. Range - some elements may only be able to take on a restricted set of values which can be expressed as a range (e.g. 0 through 511 inclusive). If this is the case, the valid inclusive range should be specified. Calato, et al. Expires May 21, 2004 [Page 7] Internet-Draft IPFIX Information Model November 2003 4. Type Space The following subsections describe the basic types from which the types of all IPFIX attributes should be constructed. By describing attributes in terms of a well defined type space, versus describing these details in each element declaration, greater consistency of the existing information model is expected. This should also simplify the process of extending the information model over time, and maintain this consistency. Still any attribute is free to restrict the type assigned to it further than the general type description in this section does. Please note that a protocol implementation may based on local configuration, chose to carry integer values in network byte order encodings in a byte size which differs (greater or smaller) than the size implied by the information elements type. For instance although byteCount is defined as an unsignedLong, which would require 8 bytes for each reported value. An implementation may send only a 4 byte quantity, if it knows that it will not exceed this amount for an individual flow The type names used are copied from the namespace defined by XML-Schema Datatypes. There are a few types which are useful to distinguish in the context of IPFIX, which do not exist in the XML-Schema namespace. The type extensions used by IPDR.org's NDM-U Specification, addresses these gaps and is called out in the list with the "ipdr:" namespace qualifier. 4.1 int The type "int" represents a integer numeric value in the range of -2147483648 to 2147483647. (i.e. a 32-bit integer) 4.2 unsignedInt The type "unsignedInt" represents an integer value in the range of 0 to 4294967295. (i.e. a 32-bit unsigned integer) 4.3 long The type "long" represents an integer value in the range of 9223372036854775807 to -9223372036854775808. (i.e. a 64-bit integer) 4.4 unsignedLong Calato, et al. Expires May 21, 2004 [Page 8] Internet-Draft IPFIX Information Model November 2003 The type "unsignedLong" represents an integer value in the range of 0 to 18446744073709551615. (i.e. a 64-bit unsigned integer) 4.5 float The type "float" corresponds to an IEEE single-precision 32-bit floating point type. 4.6 double The double datatype corresponds to IEEE double-precision 64-bit floating point type 4.7 hexBinary The type "hexBinary" represents a finite length string of octets. Note the name reflects the mechanism used in XML documents to represent the value using ASCII characters. 4.8 string The type "string" represents a finite length string of valid characters from the Unicode character encoding set. Unicode allows for ASCII and many other international character sets to be used. It is expected that strings will be encoded in UTF-8 format, which is identical in encoding for USASCII characters, but also accomodates other Unicode multibyte characters. 4.9 boolean The type "boolean" represents the values for binary logic. (i.e. "true/false" or "1/0"). 4.10 byte The type "byte" represents a integer numeric value in the range of -128 to 127. (i.e. an 8-bit integer) 4.11 unsignedByte The type "unsignedByte" represents a non-negative integer numeric value in the range of 0 to 255. (i.e. an 8-bit unsigned integer) 4.12 short The type "short" represents a integer numeric value in the range of 32767 to 32768. (i.e. an 16-bit integer) Calato, et al. Expires May 21, 2004 [Page 9] Internet-Draft IPFIX Information Model November 2003 4.13 unsignedShort The type "unsignedShort" represents a non-negative integer numeric value in the range of 0 to 65535. (i.e. an 16-bit unsigned integer) 4.14 dateTime The "dateTime" type represents a specific instant of time. It is further restricted from the basic XML dateTime type to having a precision of seconds and normalized to the GMT timezone. Such types are in common use on many Operating Systems and have the advantage that they can be stored in 32-bit integers. 4.15 ipdr:dateTimeMsec The "dateTimeMsec" type is defined in the IPDR namespace. It represents a specific instant of time. It is further restricted from the basic XML dateTime type to having a precision of milliseconds and normalized to the GMT timezone. Such types are in common use on many Operating Systems and have the advantage that they can be stored in 64-bit integers. 4.16 ipdr:ipV4Addr The "ipV4Addr" type indicates the value is an IP version 4 address. These addresses are typically stored as 32-bit integers on systems. 4.17 ipdr:ipV6Addr The "ipV6Addr" type indicates the value is an IP version 6 address. IPv6 addresses are octet strings of length 16. 4.18 ipdr:UUID The "UUID" type represents a universal unique id as defined in the OSF specification for Distributed Computing Environment (DCE). It's definition can be found in the OSF CAE Specification, Document C706, 1997, Appendix A, located at: http://www.opengroup.org/onlinepubs/ 009629399/ UUIDs are equivalent to Globally Unique Identifiers (GUIDs) used by Microsoft. UUIDs are 16 byte quantities which are generated in such a way that systems can independently generate their values, but still have a guarantee of global uniqueness of the generated value. Calato, et al. Expires May 21, 2004 [Page 10] Internet-Draft IPFIX Information Model November 2003 UUID's are typically written in the form f81d4fae-7dec-11d0-a765-00a0c91e6bf6. Which merely shows in hexadecimal the 16 byte value. Separators are introduced to segment the hex value into groupings of 4, 2, 2, 2 and 6 bytes. An open source C implementation of UUID generation is available in the appendix of the IETF draft, draftleach-uuids-guids-01.txt. This draft has expired, but an archived copy is available at: http:// www.ipdr.org/public/draft-leach-uuids-guids-01.txt Note: the IETF draft was allowed to expire because the group considered the OSF work a referenceable standard and did not chose to duplicate it. 4.19 ipdr:dateTimeUsec The dateTimeUsec type is defined in the IPDR namespace. It represents a specific instant of time. It is further restricted from the basic XML dateTime type to having a precision of microseconds and normalized to the GMT timezone. 4.20 ipfix:dateTimeNsec The dateTimeNsec type is defined in the IPFIX namespace, it allows for preservation of the granularity of time down to the nano-second (10**-9). Time models based on NTP (RFC1305) style encoding can identify time down to a granularity of 232 picoseconds (.232 nanoseconds). 4.21 Integral Type Semantics The integral types, 1,2,4 and 8 bytes long, signed or unsigned, may be qualified by additional semantic details. Specifically integral values may be called out as having the semanctic types: quantity, counter, identifier or flags. 4.21.1 Quantity A quantity value represents a discrete measured value pertaining to the record. This is distinguished from counters which represent an ongoing measured value whose "odometer" reading is captured as part of a given record. If no semantic qualifier is given, the integer should behave as a quantity. 4.21.2 Counter A measurement which is ongoing from the perspective of the exporter. Calato, et al. Expires May 21, 2004 [Page 11] Internet-Draft IPFIX Information Model November 2003 Basically the same semantics as counters in SNMP. Counters are unsigned and wrap back to zero after reaching the limit of the type. E.g. a long with counter semantics will continue to increment until reaching the value of 2**64 - 1. At this point the next increment will wrap its value to zero. To reduce incidence of wrapping, counters should be either of type unsignedInt or unsignedLong. 4.21.3 Identifier An integral value which serves as an identifier. Specifically mathematical operations on two identifiers (aside from the equality operation) are meaningless. E.g. Autonomous System Id 1 * Autonomous System Id 2 is meaningless. 4.21.4 Flags An integral value which actually represents a set of bit fields. Logical operations are appropriate on such values, but not other mathematical operations. Flags should always be of an unsigned type. Calato, et al. Expires May 21, 2004 [Page 12] Internet-Draft IPFIX Information Model November 2003 5. Extending the Information Model A key requirement for IPFIX is to allow for extending the set of information items which are reported for flows. This section defines the mechanism for extending this set. The IPFIX protocol carries flow records defined by a template. Multiple templates may be defined for a dialog between an exporter and a collector. A given template identifies the information items and their order. The means of identification of information items in a template is via a field ID. Field Id's are unique identifiers administered by IANA (ed. ? true for vendor specific fields?). Extension is done by defining new Information elements, including the set of necessary information and possibly additional optional information for each element. Each new information item MUST be assigned a unique fieldId as part of its definition. These unique field ids are the connection between the record structure communicated by the protocol using templates and a consuming application. Calato, et al. Expires May 21, 2004 [Page 13] Internet-Draft IPFIX Information Model November 2003 6. Flow Attributes 6.1 sourceAddress Description: IPv4 source address taken from the packet header. Type: ipdr:ipV4Addr. Field Id: 8 6.2 sourceAddressV6 Description: IPv6 source address taken from the packet header. Type: ipdr:ipV6Addr. Field Id: 27 6.3 destinationAddress Description: IPv4 destination address taken from the packet header. Type: ipdr:ipV4Addr. Field Id: 12 6.4 destinationAddressV6 Description: IPv6 destination address taken from the packet header. Type: ipdr:ipV6Addr. Field Id: 28 6.5 protocolIdentifier Description: Protocol number identified in the IP packet. Calato, et al. Expires May 21, 2004 [Page 14] Internet-Draft IPFIX Information Model November 2003 In the Internet Protocol version 4 (IPv4) [RFC791] there is a field, called "Protocol", to identify the next level protocol. This is an 8 bit field. In Internet Protocol version 6 (IPv6) [RFC1883] this field is called the "Next Header" field. These numbers are administered by IANA. Type: unsignedByte. Semantics: identifier. Reference: Additional information on this element can be found at http://www.iana.org/assignments/protocol-numbers. Field Id: 4 6.6 sourcePort Description: This information element is used to report UDP source port [see RFC 768] or TCP source port [see RFC 793] as taken from the IP header. Type: unsignedShort. Semantics: identifier. Field Id: 7 6.7 destinationPort Description: This information element is used to report UDP destination port [see RFC 768] or TCP destination port [see RFC 793] as taken from the IP header. Type: unsignedShort. Semantics: identifier. Field Id: 11 6.8 ingressPort Description: The ifIndex where the packets for the flow are being received. Calato, et al. Expires May 21, 2004 [Page 15] Internet-Draft IPFIX Information Model November 2003 ifIndex is defined by RFC 2863. Type: unsignedInt. Semantics: identifier. Field Id: 10 6.9 egressPort Description: The ifIndex where the packets for the flow are exiting. ifIndex is defined by RFC 2863. Type: unsignedInt. Semantics: identifier. Field Id: 14 6.10 packetCount Description: Contains the number of packets in the flow, in the "downstream" (source-to-destination) direction. Type: unsignedLong. Units: The unit of measure is packets. Field Id: 2 6.11 byteCount Description: Contains the number of bytes in the flow, in the "downstream" (source-to-destination) direction. Type: unsignedLong. Units: The unit of measure is bytes. Field Id: 1 Calato, et al. Expires May 21, 2004 [Page 16] Internet-Draft IPFIX Information Model November 2003 6.12 classOfService Description: The class of service associated with a flow. Class of Service Received Class of Service Transmitted 1. IPv4, CoS value is defined by ToS in RFC 791 2. IPv6, CoS value is defined by Traffic Class in RFC 2460 3. MPLS, CoS value is defined by Exp in RFC 3032 4. VLAN, CoS value is defined by user_priority in IEEE802.1q[802.1q] and IEEE 802.1p[802.1p] Type: unsignedByte. Field Id: 5 6.13 flowLabel Description: The Flow Label information element contains the IPV6 Flow Label information as defined by RFC 2460. Note that a flow label only occupies 20 bits in the IPv6 header. Type: unsignedInt. Field Id: 31 Range: The valid range is 0..1048575. 6.14 flowCreationTime Description: The timestamp of the first packet of the flow. (Ed. note: current NFv9 protocol uses sysuptime vs. direct time. Not interesting from an info model perspective, an artifact (and an annoying one from a consumer perspective) of the protocol implementation details. How to address?) Type: dateTime. Field Id: 22 Calato, et al. Expires May 21, 2004 [Page 17] Internet-Draft IPFIX Information Model November 2003 6.15 flowEndTime Description: The timestamp of the last packet of the flow. (Ed. note: current NFv9 protocol uses sysuptime vs. direct time. Not interesting from an info model perspective, an artifact (and an annoying one from a consumer perspective) of the protocol implementation details. How to address?) Type: dateTime. Field Id: 21 6.16 sourceAS Description: The Autonomous System (AS) numbers for the source address associated with a flow. Autonomous System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4): Type: unsignedInt. Semantics: identifier. Field Id: 16 6.17 destinationAS Description: The Autonomous System (AS) numbers for the destination address associated wit a flow. Autonomous System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4). Type: unsignedInt. Semantics: identifier. Field Id: 17 6.18 nextHopAS Description: The Autonomous System (AS) numbers for the next hop IP. Autonomous System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4). Calato, et al. Expires May 21, 2004 [Page 18] Internet-Draft IPFIX Information Model November 2003 Type: unsignedInt. Semantics: identifier. Field Id: -1 6.19 tcpControlBits Description: The TCP control bits seen for this flow. Note a 0 value for each bit only indicates that the flag was not detected (i.e. it may have occurred but was not detected by the reporting CCE). TCP Control Bits are defined by RFC 793. Type: unsignedByte. Semantics: flags. Field Id: 6 6.20 ipV4SourceExporterAddress Description: The IPV4 address of the Exporter reporting the flow. This information is used by applications to later correlate the ingress/egress port with a specific Exporter. It is also used to maintain the source Exporter information when there is an intermediate proxy. For example, given the picture below: SW1 -------- P1 ------ Collector ^ | SW2---------- | Flows coming from SW1 and SW2 through proxy P1 would look to the Collector like the same Exporter connection. With the Source Exporter in the message the original Exporter address is maintained. Type: ipdr:ipV4Addr. Field Id: -1 6.21 ipV6SourceExporterAddress Calato, et al. Expires May 21, 2004 [Page 19] Internet-Draft IPFIX Information Model November 2003 Description: The IPv4 address of the Exporter reporting the flow. This information is used by applications to later correlate the ingress/ egress port with a specific Exporter. It is also used to maintain the source Exporter information when there is an intermediate proxy. For example, given the picture below: SW1 -------- P1 ------ Collector ^ | SW2---------- | Flows coming from SW1 and SW2 through proxy P1 would look to the Collector like the same Exporter connection. With the Source Exporter in the message the original Exporter address is maintained. Type: ipdr:ipV6Addr. Field Id: -1 6.22 droppedPacketCount Description: Contains the count of packets dropped at the observation point associated with the identified flow since the last report for this flow. Type: unsignedLong. Units: The unit of measure is packets. Field Id: -1 6.23 samplingInterval Description: When using Sampling, the rate at which packets is sampled. For example, a value of 100 indicates that one of every hundred packets is sampled. Type: unsignedInt. Field Id: 34 Calato, et al. Expires May 21, 2004 [Page 20] Internet-Draft IPFIX Information Model November 2003 6.24 samplingAlgorithm Description: The type of algorithm used for sampling data. Currently, the only sampling algorithm defined is: 0x02 packet-sampling Type: unsignedShort. Field Id: 35 6.25 flowEndState Description: The reason the flow has ended. 1. Inactivity timeout 2. End of flow detected (e.g. TCP FIN) 3. Forced end ???? 4. Cache full [enumerations in IPDR service def schemas are recommended to be of form string, w/ integer values (for Compact format) defined via annotation] Type: unsignedByte. Field Id: -1 6.26 droppedByteCount Description: Contains the count of octets dropped at the observation point associated with the identified flow since the last report for this flow. Type: unsignedLong. Units: The unit of measure is bytes. Field Id: -1 Calato, et al. Expires May 21, 2004 [Page 21] Internet-Draft IPFIX Information Model November 2003 7. The Benefits of a Formal Machine Readable Information Model Appendix A. expresses the IPFIX Information model as an XML-Schema. Using a formal and machine readable syntax for the Information model enables the creation of IPFIX aware tools which can automatically adapt to extensions to the information model, by simply reading updated information model specifications. The use of XML-Schema as the formal specification language is modeled after the techniques employed by the IPDR NDM-U specification. The wide availability of XML aware tools and libraries for client devices is a primary consideration for this choice. In particular libraries for parsing XML documents are readily available. Also mechanisms such as the Extensible Stylesheet Language (XSL) allow for transforming a source XML document into other documents. This draft was initially authored in XML and transformed according to RFC2629. It should be noted that the use of XML in exporters, collectors or other tools is not mandatory for the deployment of IPFIX. In particular exporting processes do not produce or consume XML as part of their operation. It is expected that IPFIX collectors MAY take advantage of the machine readability of the Information Model vs. hardcoding their behavior or inventing proprietary means for accomodating extensions. Calato, et al. Expires May 21, 2004 [Page 22] Internet-Draft IPFIX Information Model November 2003 8. Security Considerations The IPFIX information model itself does not directly introduce security issues. Rather it defines a set of attributes which may for privacy or business issues be considered sensitive information. The underlying protocol used to exchange the information described here must therefor apply appropriate procedures to guarantee the integrity and confidentiality of the exported information. Such protocols are defined in separate documents. Specifically the IPFIX Protocol document. Calato, et al. Expires May 21, 2004 [Page 23] Internet-Draft IPFIX Information Model November 2003 References [1] Quittek, J., "Requirements for IP Flow Information Export", IETF draft work in progress, August 2003, . [2] Sadasivan, G. and N. Brownlee, "Architecture Model for IP Flow Information Export", IETF draft work in progress, June 2003, . [3] Zseby, T., Penno, R., Claise, B. and N. Brownlee, "IPFIX Applicability", IETF draft work in progress, June 2003, . [4] Claise, B., Fullmer, M., Calato, P. and R. Penno, "IPFIX Protocol Specification", IETF draft work in progress, June 2003, . [5] Claise, B., "Cisco Systems NetFlow Services Export Version 9", IETF draft work in progress, June 2003, . [6] World Wide Web Consortium, "Extensible Markup Language (XML) 1.0", W3C XML, February 1998, . [7] World Wide Web Consortium, "XML Schema Part 1: Structures", W3C XML, May 2001, . [8] World Wide Web Consortium, "XML Schema Part 2: Datatypes", W3C XML, May 2001, . [9] Internet Protocol Detail Record Organization, "Network Data Management - Usage (NDM-U) For IP-Based Services Version 3.1.1", October 2002, . [10] Brownlee, N. and A. Blount, "Accounting Attributes and Record Formats", RFC 2924, Sept. 2000, . [11] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, June 1999, . Calato, et al. Expires May 21, 2004 [Page 24] Internet-Draft IPFIX Information Model November 2003 [12] Hollenbeck, S., Rose, M. and L. Masinter, "Guidelines for the Use of Extensible Markup Language (XML) within IETF Protocols", RFC 3470, January 2003, . [13] Pras, A. and J. Schoenwaelder, "On the Difference between Information Models and Data Models", RFC 3444, January 2003, . Authors' Addresses Paul Calato Riverstone Networks Inc 5200 Great America Parkway Santa Clara, CA 95054 US Phone: +1 603 557-6913 EMail: calato@riverstonenet.com URI: http://www.riverstonenet.com Jeff Meyer Hewlett-Packard 19420 Homestead Rd. Cupertino, CA 95014 US Phone: +1 408 447-3477 EMail: jeff.meyer2@hp.com URI: http://www.hp.com Juergen Quittek NEC Europe Ltd. Adenauerplatz 6 Heidelberg 69115 Germany Phone: +49 6221 90511-15 EMail: quittek@ccrle.nec.de URI: http://www.neceurope.com/ Calato, et al. Expires May 21, 2004 [Page 25] Internet-Draft IPFIX Information Model November 2003 Appendix A. IPFIX IPDR Service Definition This proposal does not currently address possible IANA implications associated with XML Namespace URIs. The use of Namespaces as an extension mechanism implies that an IANA registered Namespace URI should be available and that directory names below this base URI be assigned for relevant IETF specifications. The author is not aware of this mechanism today. Alternatively IPDR.org could fulfill this role. The sample uses the IPDR.org namespace. The normative status of this appendix versus the section "Flow Attributes" is a point of discussion. The "Flow Attributes" section is simply machine generated from the formal XML document below. As such using the formal XML document would seem preferable. However historical conventions and IETF's overall level of XML adoption may lead to selection of the human readable text in the "Flow Attributes" section as being preferable as normative. This document defines a subset of the identified IPFIX data model as XML Schema elements and complexTypes. This schema definition is compatable with the IPDR Service Definition format, enabling flow information to be represented as XML or binary documents. And defines the format used when streaming flow information to a recording system. IPv4 source address taken from the packet header. 8 Calato, et al. Expires May 21, 2004 [Page 26] Internet-Draft IPFIX Information Model November 2003 IPv6 source address taken from the packet header. 27 IPv4 destination address taken from the packet header. 12 IPv6 destination address taken from the packet header. 28 Calato, et al. Expires May 21, 2004 [Page 27] Internet-Draft IPFIX Information Model November 2003 Protocol number identified in the IP packet. In the Internet Protocol version 4 (IPv4) [RFC791] there is a field, called "Protocol", to identify the next level protocol. This is an 8 bit field. In Internet Protocol version 6 (IPv6) [RFC1883] this field is called the "Next Header" field. These numbers are administered by IANA. identifier http://www.iana.org/assignments/protocol-numbers 4 This information element is used to report UDP source port [see RFC 768] or TCP source port [see RFC 793] as taken from the IP header. identifier 7 This information element is used to report UDP destination port [see RFC 768] or TCP destination port [see RFC 793] as taken from the IP header. Calato, et al. Expires May 21, 2004 [Page 28] Internet-Draft IPFIX Information Model November 2003 identifier 11 The ifIndex where the packets for the flow are being received. ifIndex is defined by RFC 2863. identifier 10 The ifIndex where the packets for the flow are exiting. ifIndex is defined by RFC 2863. identifier 14 Contains the number of packets in the flow, in the "downstream" Calato, et al. Expires May 21, 2004 [Page 29] Internet-Draft IPFIX Information Model November 2003 (source-to-destination) direction. packets 2 Contains the number of bytes in the flow, in the "downstream" (source-to-destination) direction. bytes 1 The class of service associated with a flow. Class of Service Received Class of Service Transmitted 1. IPv4, CoS value is defined by ToS in RFC 791 2. IPv6, CoS value is defined by Traffic Class in RFC 2460 3. MPLS, CoS value is defined by Exp in RFC 3032 4. VLAN, CoS value is defined by user_priority in IEEE802.1q[802.1q] and IEEE 802.1p[802.1p] 5 Calato, et al. Expires May 21, 2004 [Page 30] Internet-Draft IPFIX Information Model November 2003 The Flow Label information element contains the IPV6 Flow Label information as defined by RFC 2460. Note that a flow label only occupies 20 bits in the IPv6 header. 31 0..1048575 The timestamp of the first packet of the flow. (Ed. note: current NFv9 protocol uses sysuptime vs. direct time. Not interesting from an info model perspective, an artifact (and an annoying one from a consumer perspective) of the protocol implementation details. How to address?) 22 The timestamp of the last packet of the flow. (Ed. note: current NFv9 protocol uses sysuptime vs. direct time. Not interesting from an info model perspective, an artifact (and an annoying one from a consumer perspective) of the protocol implementation details. How to address?) 21 Calato, et al. Expires May 21, 2004 [Page 31] Internet-Draft IPFIX Information Model November 2003 The Autonomous System (AS) numbers for the source address associated with a flow. Autonomous System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4): identifier 16 The Autonomous System (AS) numbers for the destination address associated wit a flow. Autonomous System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4). identifier 17 The Autonomous System (AS) numbers for the next hop IP. Autonomous System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4). identifier -1 Calato, et al. Expires May 21, 2004 [Page 32] Internet-Draft IPFIX Information Model November 2003 The TCP control bits seen for this flow. Note a 0 value for each bit only indicates that the flag was not detected (i.e. it may have occurred but was not detected by the reporting CCE). TCP Control Bits are defined by RFC 793. flags 6 The IPV4 address of the Exporter reporting the flow. This information is used by applications to later correlate the ingress/egress port with a specific Exporter. It is also used to maintain the source Exporter information when there is an intermediate proxy. For example, given the picture below:
SW1 -------- P1 ------ Collector ^ | SW2---------- |
Flows coming from SW1 and SW2 through proxy P1 would look to the Collector like the same Exporter connection. With the Source Exporter in the message the original Exporter address is maintained.
-1
Calato, et al. Expires May 21, 2004 [Page 33] Internet-Draft IPFIX Information Model November 2003
The IPv4 address of the Exporter reporting the flow. This information is used by applications to later correlate the ingress/egress port with a specific Exporter. It is also used to maintain the source Exporter information when there is an intermediate proxy. For example, given the picture below:
SW1 -------- P1 ------ Collector ^ | SW2---------- |
Flows coming from SW1 and SW2 through proxy P1 would look to the Collector like the same Exporter connection. With the Source Exporter in the message the original Exporter address is maintained.
-1
Contains the count of packets dropped at the observation point associated with the identified flow since the last report for this flow. packets -1 Calato, et al. Expires May 21, 2004 [Page 34] Internet-Draft IPFIX Information Model November 2003 When using Sampling, the rate at which packets is sampled. For example, a value of 100 indicates that one of every hundred packets is sampled. 34 The type of algorithm used for sampling data. Currently, the only sampling algorithm defined is: 0x02 packet-sampling 35 The reason the flow has ended. 1. Inactivity timeout 2. End of flow detected (e.g. TCP FIN) 3. Forced end ???? 4. Cache full [enumerations in IPDR service def schemas are recommended to be of form string, w/ integer values (for Compact format) defined via annotation] -1 Calato, et al. Expires May 21, 2004 [Page 35] Internet-Draft IPFIX Information Model November 2003 Contains the count of octets dropped at the observation point associated with the identified flow since the last report for this flow. bytes -1
Calato, et al. Expires May 21, 2004 [Page 36] Internet-Draft IPFIX Information Model November 2003 Appendix B. Change History This document was originally based on a submission of the IETF draft document "IPFIX Data Model, Data Model for IP Flow Information Export", draft-ietf-ipfix-data-00.txt. Written by Paul Calato and K.C. Norseth. That document expired in August 2002. There was significant restructuring of the document to create teh first draft-ietf-ipfix-info-00.txt and a switch to using a more formal information model. B.1 Changes Since -01 Revision Issue: INFO-17 Variant Field Types Description: - allow in encoding, make explicit in information model. Motivation is if known integer quantities for a given exporter are in a smaller range, fewer bytes can be used to send across the wire. A consumer should be made aware of the largest size any compliant should export - (information model) http://ipfix.doit.wisc.edu/archive/1935.html Issue: INFO-18 Field semantics (counters vs. quantities) Description: Counters vs. Identifiers (OVMS) vs. Discrete Quantities (Gauge?) - explicitly distinguish between the two. However, when it comes to storing results, e.g. to do reporting, a counter is pretty much useless (i.e. a collector will likely turn a counter into an integer). - Counters may NOT have a variable length, as this makes wrap calculation problematic. http://ipfix.doit.wisc.edu/archive/1981.html Issue: INFO-19 Timestamps Description: http://ipfix.doit.wisc.edu/archive/2056.html - various resolutions and encoding formats can be specified: o seconds - 32 bit since EPOCH Calato, et al. Expires May 21, 2004 [Page 37] Internet-Draft IPFIX Information Model November 2003 o milliseconds - 64 bit since EPOCH (no rollover) o microseconds - ? o nanoseconds - ? ** Use for NTP ** o NTP format - 232 picosecond resolution (1/2**32) Encoding options include: - simple 32-bit time (sec) - simple 64-bit time (msec) - high res 32-bit time (m or usec) relative to header time - very high res 2*32-bit time in NTP format sec.fraction Issue: INFO-24 Use of signed versus unsigned Description: Many of the items in the information model specify signed quantities, when these values will never go below zero. Call them out as unsigned: - Sign bit on AS numbers x - Sign and direction on counters x - ProtocolIdentifier should be of type unsignedByte x - ClassOfService should be unsigned byte v. byte x - TcpControlBits should be an unsigned byte x - SamplingInterval should be unsignedInt x - SamplingAlgorithm as unsignedShort x - FlowEndState as unsigned (int or short) x - IfIndex (egress and ingress) can be 2**32 x Issue: INFO-21 FlowCreationTime/EndTime ids should be swithced 22,21 Description: transcription error from NFv9. http:// ipfix.doit.wisc.edu/archive/1919.html Issue: INFO-22 Flow Label is 3 bytes long? x - address through doc... Calato, et al. Expires May 21, 2004 [Page 38] Internet-Draft IPFIX Information Model November 2003 Description: the flow label field only has 20 bytes allocated to it. Hence it doesn't need a full int. Is this an issue, or simply indicate this fact in info model. http://ipfix.doit.wisc.edu/archive/ 1919.html B.2 Changes Since -00 Revision Issue: INFO-1 reference section updates Issue: INFO-2 change the XSL style sheet to reduce verbosity when creating human friendly information element section. Issue: INFO-3 add discussion around use of the "ipdr:" namespace qualifier in some of the type fields. Issue: INFO-4 change reference from "flow id" to "field id" Issue: INFO-5 qualify the sourceExporterAddress with ipV4 and ipV6 variants (still need field ID assignment) Issue: INFO-6 correct the fieldId value for packetCount to 2 Issue: INFO-7 remove excessive whitespace in between words "IP flows" Issue: INFO-8 rewording in introduction section around Issue: INFO-9 remove "Definition of a Flow" section and merge wording with "Scope" section Issue: INFO-10 modifications to section "Properties of an IPFIX Flow Attribute" Issue: INFO-11 modify wording and capitalization in "Type Space" introduction Issue: INFO-12 change wording of IPv6 type descriptrion Issue: INFO-13 change section title to be "Flow Attributes" from "Information Elements" Issue: INFO-14 move and combine the sections on "Use of XML-Schema" and "Benefits of a Formal Information Model" to the back. Leaving the readability of the major front matter fairly "XML free". Issue: INFO-15 add "Security Considerations" section which is mandatory. Calato, et al. Expires May 21, 2004 [Page 39] Internet-Draft IPFIX Information Model November 2003 Issue: INFO-16 modify abstract following additional comments from Juergen. Calato, et al. Expires May 21, 2004 [Page 40] Internet-Draft IPFIX Information Model November 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Calato, et al. Expires May 21, 2004 [Page 41] Internet-Draft IPFIX Information Model November 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Calato, et al. Expires May 21, 2004 [Page 42]