Network Working Group Ganesh Sadasivan Internet Draft Benoit Claise Expiration Date: August 2002 cisco Systems, Inc. February 2002 Proposal for IPFIX Flow Export Architecture and Data Model draft-gsadasiv-ipfix-proposal-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This memo defines the architecture, the data model and the information model for the export of measured IP flow information out of an exporter to a collector, per the requirements defined in [1]. While this document discusses the key concepts of flow export, some of the topics discussed are far from complete. The intention of this revision is to provide a starting framework for the flow export architecture. Sadasivan & Claise [Page 1] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Sadasivan & Claise [Page 2] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 Table of Contents 1 Introduction ........................................... 4 1.1 Overview ............................................... 4 1.2 Terminologies Used ..................................... 4 2 Flow Type and Flow Definition .......................... 5 2.1 Observation Point ...................................... 7 2.2 Unidirectional and Bidirectional Flows ................. 7 3 Metering Process Functions ............................. 7 3.1 Flow Classification .................................... 7 3.2 Selection Criteria Of Packets .......................... 8 3.2.1 Funtion on properties that determines a flow type (Fi) . 8 3.2.2 Sampling packets on a flow type (Si) ................... 8 3.3 Selection Criteria of flows for export ................. 8 3.4 Flow Expiration ........................................ 9 4 Flow Export Protocol ................................... 9 4.1 Simple Export Model .................................... 10 4.1.1 Collector Crash ........................................ 10 4.1.2 Collector Redundancy ................................... 10 4.1.3 Collector Failover ..................................... 10 4.2 Export Model with Reliable Control Connection .......... 10 4.2.1 Collector Crash ........................................ 11 4.2.2 Collector Redundancy ................................... 11 4.2.3 Collector Failover ..................................... 12 5 Flow Export Structure .................................. 12 5.1 Flow Header ............................................ 12 5.1.1 Fields ................................................. 13 5.2 Template and Template Flowsets ......................... 14 5.3 Categories of Template Flowset ......................... 14 5.3.1 Case A, Template for Flow Record Definition ............ 14 5.3.2 Case B, Template for Method Description ................ 16 5.3.3 Case C, Template for Dependancies ...................... 18 5.4 Template Export Management ............................. 21 5.5 Exporting exporter statistics and errors ............... 21 5.6 Exporting Flow Records ................................. 22 5.6.1 Field Descriptions ..................................... 22 6 Specific Technology Considerations ..................... 23 6.1 Network Address and Port Translation ................... 23 7 Information Model ...................................... 23 8 The Collector Side ..................................... 27 Acknowledgements ....................................... 28 References ............................................. 28 Authors' Addresses ..................................... 28 Sadasivan & Claise [Page 3] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 1. Introduction 1.1. Overview The IP Flow data can be used for a variety of purposes. Usage-based Accounting, Traffic Profiling, Traffic Engineering, Attack/Intrusion Detection, Network Surveillance, QoS Monitoring are some of the applications that use IP flows to list a few. Different applications require different set of fields, which are mostly subsets of all the fields that an IPFIX compliant exporter could possibly export. So mostly not all possible exportable fields need to be exported at all times. The intention of the chosen approach here is to make the flow export: 1. Flexible: There should be flexibility in choosing to export only the required fields. 2. Extensible: Addition of new fields or technologies should have no impact on the export framework. The transport protocol used for sending these flow records and templates is beyond the scope of this document. 1.2. Terminologies Used * Exporter: The entity on which flows are measured and are exported. The exporter can be a router, a middlebox [2], or a traffic measurement probe. * Collector: The entity that collects the exported information. * Observation Point: The observation point may be one or a set of interfaces (physical or logical) of the exporter. * Observation Domain: The set of observation points which is the largest aggregatable set of flow information at the exporter is termed as an observation domain. The observation domain presents itself a unique ID to the collector for identifying the export packets generated by it. Example: The observation domian could be a router linecard, composed of several interfaces with each interface being an observation point. Sadasivan & Claise [Page 4] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 * Metering Process: Measurement process that is carried out at the observation domain. * Flow Header: Every flow export packet includes a flow header which carries some general information about the exporter, packet identification etc. * Flowset: Following the Flow Header, an export packet contains information that would be parsed and interpreted by the collector device. A flowset is a generic term for a collection of records that follow the similar template format in an export packet. * Flow Record: A flow record provides information about a specific flow that was measured at an observation point using a certain set of methods within an exporter. 2. Flow Type and Flow Definition As defined in the requirement draft [1], "A flow is a set of packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties derived from the data contained in the packet and from the packet treatment at the observation point." In this draft we define the flow more specifically. A flow is defined as a set of packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties. Each property is defined as the result of applying a function to the values of: a. one or more of packet header fields (eg. destination IP address) b. one or more properties of the packet itself (eg. packet length) c. one or more of fields derived from packet treatment (eg. AS number) A packet is defined to belong to a flow if it matches all the defined properties of the flow. Flows can be defined in multiple ways. Some examples are: Example 1: To create flows, we define the different fields to distinguish flows. The different combination of the field values creates unique flows. If the keys are defined as {source IP address, destination IP address, TOS}, then all of Sadasivan & Claise [Page 5] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 1. {192.1.40.1, 171.6.23.5, 4} 2. {192.1.40.23, 171.6.23.67, 4} 3. {192.1.40.23, 171.6.23.67, 2} 4. {198.20.9.200, 171.6.23.67, 4} are different flows. Example 2: To create flows, we can apply a match function to all the packets that pass through an observation point, in order to aggregate some values. This could be done by defining the keys as {source IP address, destination IP address, TOS} like in the example 1, and applying the function which masks the least significant 8 bits of the source IP address and destination IP address (.i.e the resultant is a /24 address). The 4 flows from example 1 would now be aggregated into 3 flows, by merging the flows 1. ans 2. into a single flow. 1. {192.1.40.0/24, 171.6.23.0/24, 4} 2. {192.1.40.0/24, 171.6.23.0/24, 2} 3. {198.20.9.0/24, 171.6.23.0/24, 4} Example 3: To create flows, we can filter some field values on all packets that pass the observation point, in order to select only certain flows. The filter is defined by choosing fixed values for specific fields from the packet. All the packets that go from a customer network 192.1.40.0/24 to another customer network 171.6.23.0/24 with TOS value of 4 define a flow. All other combinations don't define a flow and are not taken into account. The 3 flows from example 2 would now be reduced to 1 flow, by filtering away the second and the third flow. {192.1.40.0/24, 171.6.23.0/24, 4}. The above example can be thought of as a function F takes as input {source IP address, destination IP address, TOS}. The function selects only the packets which satisfies all the 3 conditions which are: * mask the least significant 8 bits of source IP address, compare against 192.1.40.0. * mask the least significant 8 bits of destination IP address, compare against 171.6.23.0. * tos value equal to 4. Depending on the values of {source IP address, destination IP address, TOS} of the different observed packets, the metering process function F would choose/filter/aggregate different sets of packets, which would create different flows. In other words, based on various Sadasivan & Claise [Page 6] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 combination of values of {source IP address, destination IP address, TOS}, F(source IP address, destination IP address, TOS) would results in the definition of one or more flows. The function F is referred to as Flow Type. 2.1. Observation Point For definition refer to section 1.2. A flow is uniquely defined by the way it gets measured at an observation point. Example: The flow defined in the example 1 of section 2., can be used at 2 different observation points 'a' and 'b' in 2 different ways. Observation point 'a' measures the packets that match the flow {192.1.40.0/8, 171.6.23.0/8, 4} at a sampling rate of 1 in 10 and 'b' measures packets that match the same flow with a sampling rate of 1 in 100. 2.2. Unidirectional and Bidirectional Flows A flow defined by the above terms is unidirectional. In case the exporter has got the knowledge of the bi-directional flows and in case the bi-directional information make sense, the exporter MAY choose to export in the same Template/Flow Record, along with bidirectional accounting information. This would save some bandwidth as the exporter won't have to send two unidirectional flow records. 3. Metering Process Functions 3.1. Flow Classification The collector MUST be able to map the flow to the corresponding property types defined by the flow type. This can be done only if the collector has a mapping from flow type identifier (carried in each flow record) to its actual structure. More details of how this can be acheived is decribed in section 5. In addition the collecter, when it receives the flow records, MAY need the following interpreting the flow records furthur: a. Observation Point. b. Selection Criteria of Packets As mentioned in section 2.1, a flow record can be better analyzed if the Observation Point from which it is measured is known. As such it Sadasivan & Claise [Page 7] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 is recomended that the flow record carry the Observation Point information along with the flow records when exported. In cases where there is a single observation point or where the observation point information is not relevant, the exporter MAY choose not to add this to the flow records. 3.2. Selection Criteria Of Packets The measurement device MAY define rules so that only certain packets within a flow can be chosen for measurement at an observation point. This MAY be done by one of the two types of methods defined below or a combination of them. A combination of each of these ways can be adopted to select the packets .i.e one can define a set of methods {F1, S1, F2, S2, S3} executed in certain sequence at an observation point to collect flows of a particular type. 3.2.1. Funtion on properties that determines a flow type (Fi) Packets that satisfy a function on the fields defined by the packet header fields or fields obtained while doing the packet processing or the properties of the packet itself. Example: Mask/Match of the fields that define a filter. The filter may be defined as {Protocol == TCP, Destination Port between 80 and 120}. Multiple such filters could be used in any sequence to select packets. 3.2.2. Sampling packets on a flow type (Si) Packets that satisfy the sampling criteria for this flow type. Example: Sample every 100th packet that was received at an observation point and collect the flow information for a particular flow type. Choosing all the packets is a special case where sampling rate is 1:1. 3.3. Selection Criteria of flows for export The measurement device MAY define additional rules so that only certain flows records are picked up for export. This MAY be done by either one of the two types of methods defined in 2.3.1 and 2.3.2 or a combination of them. Example: The flow records which meets the following selection criteria are Sadasivan & Claise [Page 8] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 only exported. 1. All flow records whose destination IP address matches {20.3.1.5}. 2. Every other (.i.e sampling rate 1 in 2) flow record whose destination IP address matches {160.0.1.30}. 3.4. Flow Expiration A flow is considered to be inactive if no packets of this flow has been observed at the observation point for a given timeout interval. The flow can be exported under the following conditions: 1. If the exporter can deduce the end of a flow, the exporter SHOULD export the flow records when the end of the flow is detected. For example: flow generated by TCP type of traffic where the FIN or RST bits indicate the end of the flow 2. If the flow has been inactive for a certain period of time. This inactivity timeout SHOULD be configurable. For example: flow generated by UDP type of traffic. 3. For long aging flows, the exporter SHOULD export the flow records on regular basis, in order to: 1. report the flow records periodic accounting information to the collector 2. avoid counter wrapping This activity timeout SHOULD be configurable 4. If the exporter experiences resources constraints, a flow MAY be prematurely expired (example: memory) 4. Flow Export Protocol The information that needs to be exported from the exporter can be classified into the following categories: * Control Information - This includes the flow type definition, selection criteria for packets within the flow. This is also called as control stream. * Flow record - This includes data records corresponding to the various observed flows at each of the observation point. This is also called as data stream. Sadasivan & Claise [Page 9] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 4.1. Simple Export Model If the network in which the exporter and collector are located guarentees the security and reliability requirements, the transport of the export flow packets MAY done over a light-weight transport like UDP, for speed and simplicity. A single transport stream for control and data would suffice here. 4.1.1. Collector Crash In the simple export model, there is no way that a collector crash can be detected. 4.1.2. Collector Redundancy In case there are multiple collectors, the exporter MAY multicast the flow records to the set of collectors that joined the export multicast group, instead of sending several unicast streams towards the different collectors. Multicast would lower the bandwidth requirements on the export link in case that the collectors are on the same media, and could lower the internal bus utilization on the exporter. Using multicast session with more than one collector joining the flow export multicast group, redundancy of collectors can be easily achieved. 4.1.3. Collector Failover In the simple export model, there is no way that a collector crash can be detected. If the ability to detect collector crash is required, a framework with some reliability built into it SHOULD be chosen. This is explained below. 4.2. Export Model with Reliable Control Connection If the network in which the exporter and collector are located does not guarentee reliability, atleast the control information SHOULD be exported over a reliable transport. There could be network security concerns between exporter and collector. To avoid re-inventing the wheel, and to reduce the complexity of flow export protocol, one or a combination of the following methods MAY be adopted as a solution to achieve security : Sadasivan & Claise [Page 10] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 * IP Authentication Header MAY be used when the threat environment requires stronger integrity protections, but does not require confidentiality. * IP Encapsulating Security Payload (ESP) MAY be used to provide confidentiality and integrity. * If the transport protocol used is TCP, optionally TCP MD5 signature option MAY be used to protect against spoofed TCP segments. The flow records MAY be exported over an unreliable or reliable transport protocol. As explained above the transport connection (in the case of a connection oriented protocol) is pre-setup between the exporter and the collector and is out of the scope of this document. Once connected, the collector side receives the control information and uses this information to interpret the flow records. The exporter SHOULD set the keepalive (in the case of TCP) or the HEARTBEAT interval in the case of SCTP to a sufficiently low value so that it can quickly detect a collector crash. On detecting a Keepalive timeout, the exporter SHOULD stop sending the flow export data to the collector and try reconnecting the transport connection. This is valid for a single collector scenario. The case of multiple collector is discussed in section 4.2.2. 4.2.1. Collector Crash The collector crash is detected at the exporter by the break in control connection (depending on the transport protocol the connection timeout mechanisms differ). On detecting loss of connectivity, the exporter retries to open the control connection with the collector. 4.2.2. Collector Redundancy The control information is exported through a reliable transport and so cannot be multicast. The data stream MAY be multicast if it is over an unreliable transport like UDP. But the collector SHOULD join the multicast group only after the control stream is established and it has received the control information from the exporter. Using multicast session with more than one collector joining the flow export multicast group, redundancy of collectors can be easily achieved. Sadasivan & Claise [Page 11] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 4.2.3. Collector Failover This is an extension to section 4.2 to take care of collector crash. The exporter opens control connections to multiple collectors. But data gets sent only to one of the collectors which is chosen as the primary. There could be one or more collectors configured as secondary and a priority assigned to them. The primary collector crash is detected at the exporter by the break in control connection (depending on the transport protocol the connection timeout mechanisms differ). On detecting loss of connectivity, the exporter opens data stream with the secondary collector of the next highest priority. This collector now becomes the primary. The maximum export data loss would be the amount of data exported in the time between when the loss of connectivity to the collector happened, and the time at which this was detected by the exporter. 5. Flow Export Structure The general format of flow export packet is as follows: +-------+-------------------------------------------------------+ |Flow | +----------------+ +---------------+ +--------------+ | |Header | | Flow Set | | Flow Set | | Flow set | | | | +----------------+ +---------------+ +--------------+ | +-------+-------------------------------------------------------+ 5.1. Flow Header The flow header contains the export packet level information and some information pertaining to the exporter itself. The flow header format is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version Number | Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | sysUpTime | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Unix Secs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Sadasivan & Claise [Page 12] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 5.1.1. Fields Version Number: The version number of the flow export protocol on the exporter side. Both control and data records exported from the exporter are self- describing. The collector can thus selectively choose to collect the information from the export records regardless of the version. Note that the packet header format has been kept similar the one developed by the different versions of Netflow defined by Cisco Systems, for backward compatibility. This is also the reason why the version field is currently 0x0009. Flags: Flags indicate the packet type and other attributes associated with the packet. The format of flags is as follows: 15 1 0 +----------------------------------------------------+--------+ | |Control/| | Reserved |Data / | | |Both | +----------------------------------------------------+--------+ 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version Number | Flags |*|*| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ** Control/Data/Both Control/Data/Both: Flag to indicate whether the export packet is control/data/both. When separate data stream is used, the exporter MUST set the flags to indicate whether the stream type is control and data. SysUpTime: Time in milliseconds since this device was first booted. Unix Secs: Seconds since 0000 UTC 1970 Sequence Number: Exporter generated number which uniquely identifies a packet. The sequence number increments for subsequent export packets. A separate sequence number is manintained for control and export data packets if control and data are send in separate packets. Sadasivan & Claise [Page 13] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 Source ID: Unique identifier per observation domain. 5.2. Template and Template Flowsets Templates are used to completely identify the structure and semantics of a particular information that needs to be communicated from the exporter to the collector. Each template is uniquely identified by a Template ID. The Template ID is a 16 bit identifier. A block of templates from <0-255> are reserved for sending some of the control information. The rest of the template IDs can be used by the exporter to define the templates for the various flow types defined within the exporter. A Template within an export packet is called as a template record. A Template Flowset is a collection of one or more template records which have been grouped together in an export packet. The Flowset ID of a template flowset maps to a particular template format. 5.3. Categories of Template Flowset As explained in the terminology section, a flowset is a collection of records with a similar template format in an export packet. Flowset ID shares the same number space as a template ID. Template ID <0-255> are reserved as mentioned above and these are used by Flowset IDs to send Template Flowsets of different formats. All of the Templates are sent over the control stream. The different template formats being used are: 5.3.1. Case A, Template for Flow Record Definition The common examples of this case are templates for exported flow records, and templates for flow related statistics or errors in the exporter. Example of statistics information: total numbers of flows. The format of the Template Flowset for this case is described below: Sadasivan & Claise [Page 14] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 0 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 1 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type N | Field Length N | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 2 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type M | Field Length M | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ FlowSet ID: Template flowset of this structure has a FlowSet ID of 0. Length: Total length of this FlowSet including the Flowset ID and the Length field itself. Since an individual Template FlowSet MAY contain multiple Template IDs (as illustrated above), the Length value will be used to determine the position of the next FlowSet. Template ID: A unique ID per observation domain that defines the type of the record. Template IDs 0-255 are reserved for special uses.Templates that define Flow Record formats begin numbering at 256. Field Count: Number of fields in this Template Record. Since a Template Flowset MAY contain multiple Template Records, this field allows the collector to determine the end of the current Template Record and the start of the next. Field Type: A numeric value that represents the type of the field. The possible Sadasivan & Claise [Page 15] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 values of this field are described in the information model section. Field Length: The length of the above defined field, in bytes. See the information model section for the field length. Note that the reason why we need the Field Length, even if this one is fixed in the Information Model is because a collector MUST be able to decode flow records, even if it doesn't know the semantics of certain field types within the flow records. 5.3.2. Case B, Template for Method Description This case describes the format for exported configuration information. Each of the configuration templates within this flowset describes : * List of methods that define the Selection Criteria and the parameters for each method. * Observation Point and its description. One thing to note here is that instead of template ID, each configuration template is associated with an Observation ID. The observation ID uniquely identifies a {, Observation Point}. At the same Observation point, multiple methods could be adopted for packet measurement. As mentioned in section 3.1, for the full interpretation of the flows from a metering process, the associated {, Observation Point} associated with each flow record is required. This can be achieved by sending Observation ID as one of the fields in each of the flow records. In such a case the template for flow records (see section 5.3.1) would define Observation ID as one of the field types. Example: The exporter could be collecting the IPV4 packets that pass through an observation point at a sampling rate of 1 in 100 while the IPV6 packets that pass through the same observation point are collected at a sampling rate of 1 in 200. The number space of Observation ID is different from that of the Template ID. Each time a configuration change happens w.r.t {, Observation Point}, a new Observation ID is generated. The rate of configuration changes within an exporter could range from very infrequent to very frequent. To take care of the frequent cases, Observation ID is assigned a 32-bit quantity. The following diagram shows the Template format. Sadasivan & Claise [Page 16] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 1 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation ID 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Method Count | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Method Id 1 | Parameter Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Parameter Type 1 | Parameter Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Parameter Value (Padded to nearest 4 bytes) | ~ ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Parameter Type K | Parameter Length K | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Parameter Value (Padded to nearest 4 bytes) | ~ ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Method Id 2 | Parameter Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation Point ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sub-element Count | Sub-element Type 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sub-element Length 1 | Sub-element Value 1 (Padded | | | to nearest 2 bytes) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sub-element Length M | Sub-element Value M (Padded | | | to nearest 2 bytes) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation ID 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Method Count | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Sadasivan & Claise [Page 17] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 FlowSet ID: Template flowset of this structure has a FlowSet ID of 1. Length: Total length of this FlowSet including the Flowset ID and Length field. Observation ID: 32 bit identifier per {, Observation Point} tuple. Method Count: Number of methods used for selecting the packet. Parameter Count: The number of parameters for this method. Parameter Type: Type of the Parameter used by this method. Parameter Length: Length of the Parameter used by Parameter Type. Parameter Value: Configured value of the parameter Parameter Type. Observation Point ID: 32 bit identifier that defines an observation point. Sub-element Count: Number of sub-elements that form the observation point. Sub-element Type: Type of one sub-element. Sub-element Length: Length of one sub-element represented by Sub-element Type. Sub-element Value Value of the sub-element of type Sub-element Type. 5.3.3. Case C, Template for Dependancies If multiple methods are used in conjunction on the observation point, in order to analyze the flow records, the collector SHOULD know the dependancy between the multiple methods used. At the same observation point, multiple different sets of measurement could be carried out simultaneously or independently to the packets. Depending on how Sadasivan & Claise [Page 18] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 these sets of measurements are done, the interpretation of flow records vary. Example : Say packets passing through an observation point are filtered, using 2 different filters. case 1: Filters are completely non-overlapping. Say the filters are set on the destination IP prefix. The first filter F1 masks and matches the destination prefix {1.0.0.0/8} and the second filter masks and matches the destination prefix {2.0.0.0/8}. The result of applying these 2 filters generates 2 different sets of flows which are completly non-overlapping. The collector could sum up the counters of flows that result from F1 and F2 to provide more aggregation. case 2: Overlapping which are predictable. Say one filter is set on the source IP prefix and destination IP prefix and the other on the destination IP address. The first filter F1 masks and matches the {source prefix = 1.0.0.0/8, destination prefix = 2.0.0.0/8} and the second filter F2 masks and matches {destination prefix = 2.0.0.0/8}. Whenever a packet matches F1 it will necessarily match F2 also. The result is 2 different sets of flows where the flows created by applying F1 is a subset of that created by F2. case 3: Overlapping which are un-predictable. Say the filters are set on the source IP prefix and destination IP prefix. The first filter F1 masks and matches the {source prefix = 1.0.0.0/8, destination prefix = 2.2.0.0/16} and the second filter F2 masks and matches {source prefix = 1.1.0.0/16, destination prefix = 2.0.0.0/8}. A packet with {source IP address = 1.2.3.4, destination IP address = 2.2.2.2} matches F1 but not F2. A packet with {source IP address = 1.1.3.4, destination IP address = 2.3.1.1} matches F2 and not F1. The result of applying these 2 filters generates 2 different sets of flows, the dependancy between the two being unpredictable. Discovering the dependancies amongst the different methods at the observation point is non-trivial job for the exporter. The exporter will deduce the dependancies to the best of its capability and inform the collector. If unknown, the dependancy relationship will be set to unpredictible. The exporter MAY implement and export this dependancy template. The format of the dependency template template is as follows. Sadasivan & Claise [Page 19] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 2 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Number Of Dependency | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Dependency 1 | Number Of Observation IDs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Dependency ID 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation Id 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation Id 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Dependency 2 | Number Of Observation IDs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Dependency ID 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation Id 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation Id 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ FlowSet ID: Template flowset of this structure has a FlowSet ID of 2. Length: Total length of this FlowSet including the Flowset ID and Length field. Observation Point ID: 32 bit identifier that defines an observation point. Number Of Dependency: Number of dependency for this observation point. Dependency: Tells whether the set of templates that follow are dependent, independent or unpredictible. As mentioned in section 5.3.2, {, observation point} is identified by the Observation ID. Number Of Observation IDs: Number of Observation IDs in the dependency set. Sadasivan & Claise [Page 20] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 Dependency ID: Each dependency set is identified by a 32 bit ID. Observation ID: An element of an dependent or independent set. 5.4. Template Export Management The exporter takes the following steps for template export: * Periodically sends the entire template information and configuration information to the collector. This corresponds to the set of all Template IDs along with their descriptions and the set of all Observation IDs along with their descriptions. The periodic export frequency SHOULD be configurable in the exporter. * Changes in the control information happens due to change in configuration or change in the templates of export flow records. These changes can affect one or more of: 1. Template of flow records. See 4.3.1 2. Templates of configuration information. See 4.3.2 3. Dependency templates. See 4.3.3 When the control information changes, only the set of differences in the information at the granularity of a template need to be exported to the collector. This is valid for case 1. and case 2. For case 1., a new Template ID is generated and the changed template along with its description is sent to the collector. For case 2., a new Observation ID is generated and the changed Observation ID along with its description is sent to the collector. Case 2 may result in case 3 also. For case 3., the entire set of dependency list is regnerated at the exporter and sent to the collector. The set of differences MAY be sent a configurable number of consecutive times. This configurable parameter is recommended to be more than one in the case of the simple export model. 5.5. Exporting exporter statistics and errors The statistics and errors from the exporter MAY be periodically exported to the collector. The information model specifics for statistics and errors will be defined in future phase. Sadasivan & Claise [Page 21] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 5.6. Exporting Flow Records As mentioned above flow records can go through the same stream as a control information or through a different stream as per the description in section 4.1 and 4.2. If data stream is different, it would be identified by a different sequence number. In anycase the flags field of the flow export header MUST indicate that the packet has only data flowsets (flag in the flow export header is set to "data") or if it carries control and data flowsets (flag is set to "both"). Data packets contain flow records corresponding to one or more template IDs. Flow records that belong to the same template ID can be grouped together to utilize the bandwidth better. A collection of one or more flow records that have have the same template ID is termed as a Data Flowset. The Flowset ID of a data flowset is assigned the Template ID to which all the records in this data flowset belong to. The collector uses this Flowset ID to interpret the data contained within the flow records. A data flowset cannot be split across multiple flow record packets. The format of the Data Flowset is described below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = Template ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 1 | Record 1 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 1 | Record 2 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 3 - Field Value 1 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 5.6.1. Field Descriptions FlowSet ID = Template ID Template ID corresponding to the flow records in the flowset. Length: The length of the Data FlowSet including FlowSet ID and the Length . Record N - Field N The remainder of the Data FlowSet is a collection of field values. Sadasivan & Claise [Page 22] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 The Type and Length of the fields have been previously defined in the Template Record referenced by the FlowSet ID/Template ID. The flow records contains the values for the fields defined by flow type. 6. Specific Technology Considerations 6.1. Network Address and Port Translation In case the exporter is doing NAT [3] and in case the observation domain has got the knowledge of both inside and outside parameters, the exporter MAY choose to export both inside and outside parameters in the same Template/Flow Record (The NAT parameters being the IP address and/or the UDP/TCP ports). If not the exporter will only export the parameters observed at the observation point. This just implies that the information model MUST have inside and outside parameters defined. A NAT device can be installed between the exporter and the collector. This means that the flow records IP address could be meaningless for the data analyzis without a NAT translator on the collector's side. This case is out of the scope of this document as it concerns the analysis of the data and not the export. 7. Information Model The information model for a flow measurement report is the list of possible attributes of a flow, selection criteria (like parameters for sampling) and observation point. The table below described all the field type definition that an IPFIX compliant exporter will support: Field Type Value Length Description IN_BYTES_32 1 4 32-bit counter for bytes associated with an IP Flow IN_PKTS_32 2 4 32-bit counter for packets associated with an IP Flow FLOWS 3 4 Number of main cache flows that were aggregated PROT 4 1 IP protocol byte. Sadasivan & Claise [Page 23] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 TOS 5 1 Type of service byte TCP_FLAGS 6 1 TCP Flags (cumulative OR of TCP flags) TCP/UDP source port L4_SRC_PORT 7 2 number (.e.g, FTP, Telnet, etc...) IPV4_SRC_ADDR 8 4 Source IPv4 Address SRC_MASK 9 1 source route's mask bits INPUT_SNMP 10 2 Input interface index TCP/UDP destination port L4_DST_PORT 11 2 number (.e.g, FTP, Telnet, etc...) IPV4_DST_ADDR 12 4 Destination IPv4 Address DST_MASK 13 1 destination route's mask Bits OUTPUT_SNMP 14 2 Output interface index IPV4_NEXT_HOP 15 4 Next hop router's IPv4 Address SRC_AS 16 4 Source BGP Autonomous System Number DST_AS 17 4 Destination BGP Autonomous System Number BGP_NEXT_HOP 18 4 Next-hop router in the BGP domain IPM_DPKTS 19 4 Packet count for IP Multicast IPM_DOCTETS 20 4 Octet (byte) count for IP Multicast SysUptime at which the LAST_SWITCHED 21 4 last packet of this flow was switched Sadasivan & Claise [Page 24] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 SysUptime at which the FIRST_SWITCHED 22 4 first packet of this flow was switched IN_BYTES_64 23 8 64-bit counter for bytes associated with an IP Flow IN_PKTS_64 24 8 64-bit counter for packets associated with an IP Flow MAC_ADDR 25 6 Layer 2 Media Access Control address associated with a flow VLAN_ID 26 2 Virtual LAN identifier associated with a flow IPV6_SRC_ADDR 27 16 IPv6 Source Address IPV6_DST_ADDR 28 16 IPv6 Destination Address IPV6_SRC_MASK 29 1 IPv6 Source Mask IPV6_DST_MASK 30 1 IPv6 Destination Mask FLOW_LABEL 31 2 IPV6 Flow Label ICMP Packet Type. This is ICMP_TYPE 32 1 reported as (ICMP Type * 256) + ICMP code IGMP_TYPE 33 1 IGMP Packet Type When using Sampling, the rate at which packets are sampled. For SAMPLING_INTERVAL 34 1 example, a value of 100 indicates that one of every hundred packets is sampled. The type of algorithm used for sampling data. Currently, the only sampling algorithm defined SAMPLING_ALGO 35 1 is: 0x02 packet-sampling Sadasivan & Claise [Page 25] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 Timeout value for active FLOW_ACTIVE_TIMEOUT 36 2 flow entries in the cache. Timeout value for inactive FLOW_INACTIVE_TIMEOUT 37 2 flow entries in the cache. OUT_BYTES_32 38 4 32-bit counter for bytes associated with an IP Flow OUT_PKTS_32 39 4 32-bit counter for packets associated with an IP Flow OUT_BYTES_64 40 8 64-bit counter for bytes associated with an IP Flow OUT_PKTS_64 41 8 64-bit counter for packets associated with an IP Flow inside TCP/UDP destination INSIDE_L4_SRC 42 2 port number (.e.g, FTP, Telnet, etc...) only applicable with NAT INSIDE_IPV4_SRC_ADDR 43 4 Source IPv4 Address only applicable with NAT TCP/UDP destination port INSIDE_L4_DST_PORT 44 2 number (.e.g, FTP, telnet etc...) only applicable with NAT INSIDE_IPV4_DST_ADDR 45 4 Destination IPv4 Address only applicable with NAT INSIDE_IPV6_SRC_ADDR 46 16 IPv6 Source Address only applicable with NAT INSIDE_IPV6_DST_ADDR 47 16 IPv6 Destination Address only applicable with NAT TCP/UDP destination port OUTSIDE_L4_SRC_PORT 48 2 number (.e.g, FTP, Telnet, etc...) only applicable with NAT OUTSIDE_IPV4_SRC_ADDR 49 4 Source IPv4 Address Sadasivan & Claise [Page 26] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 only applicable with NAT TCP/UDP destination port OUTSIDE_L4_DST_PORT 50 2 number (.e.g, FTP, Telnet, etc...) only applicable with NAT OUTSIDE_IPV4_DST_ADDR 51 4 Destination IPv4 Address only applicable with NAT OUTSIDE_IPV6_SRC_ADDR 52 16 IPv6 Source Address only applicable with NAT OUTSIDE_IPV6_DST_ADDR 53 16 IPv6 Destination Address only applicable with NAT Flow Direction 54 1 The direction of the flow observed at the observation point. If the observation point is a set of interface(s) the direction is w.r.t the wire. The "Value" column in the above table is a numeric identifier for the field type. When extensibility is needed (when new technologies are defined or when some new field types are needed), the newly added field types MUST be added to the list. They MUST be defined by the exporter and understood by the collector. 8. The Collector Side The collector SHOULD receive template definitions from the exporter, before receiving flow records. The flow records can then be decoded and stored locally on the collector. In case the template definitions have not been received at the time a Flow Record is received, the collector SHOULD keep the Flow Record for later decoding when the template definitions will be received. The collector MUST decode and store the entire flow records, even if the semantics of one or more of the field types in the template associated with the flow record is not understood. The collector SHOULD maintain a table of active templates. The template information SHOULD be kept for templates of flow records, non-flow records, and dependency template. The table format is as defined below. Sadasivan & Claise [Page 27] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 +-----------+--------------+--------------+-----------------+ | Exporter | Template ID | Template Def |Time Since Last | | | | |Refresh (Tr) | +-----------+--------------+--------------+-----------------+ Note that the Template IDs are unique per exporter. A template can exist without being refreshed by an update for Tm period of time. Tr gets incremented periodically. Each time an update is received for a template, Tr is reset to 0. When Tr >= Tm, the entry is removed from the table. Collector devices SHOULD use the combination of the source IP address and the Source ID field to distinguish different export streams originating from the same exporting device. For example: different linecards on a router MAY generate different export streams to a single collector. Acknowledgements We like to thank Will Eatherton, Michelle Ma, Peram Marimuthu, Martin Djernaes and Vamsi Valluri for reviewing this document and providing valuable comments. References [1] Requirements for IP Flow Export, [2] B. Carpenter, "Middleboxes: taxonomy and issues", draft- carpenter-midtax-01.txt, work in progress. [3] RFC 3022, Traditional IP Network Address Translator (Traditional NAT) Authors' Addresses Ganesh Sadasivan Cisco Systems, Inc. 170 W. Tasman Dr. San Jose, CA 95134 USA Phone: +1 (408) 527-0251 Email: gsadasiv@cisco.com Sadasivan & Claise [Page 28] Internet Draft draft-gsadasiv-ipfix-proposal-00.txt February 2002 Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 Email: bclaise@cisco.com Sadasivan & Claise [Page 29]