IPFIX working group Internet Draft EDITORS: B. Claise draft-ietf-ipfix-protocol-01.txt Cisco Systems Expires: April 2004 Mark Fullmer OARnet Paul Calato Riverstone Networks Reinaldo Penno Nortel Networks October 2003 IPFIX Protocol Specifications Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsolete by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document discusses the IPFIX protocol that provides network administrators with access to IP flows information. This document focuses on how IPFIX flow record data, options record data and control information is carried (via a congestion-aware transport protocol) from IPFIX exporting process to IPFIX collecting process. Conventions used in this document Claise, et. al Standard Track [Page 1] IPFIX Protocol Specifications October 2003 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Table of Contents 1. Points of Discussion........................................3 1.1 Open Issues................................................3 1.2 Action Items...............................................4 2. Introduction................................................4 2.1 Overview...................................................4 3. Terminology.................................................4 3.1 Terminology Summary Table..................................9 4. The Metering Process........................................9 4.1 Flow Expiration............................................9 5. Transport Protocol.........................................10 5.1 TCP.......................................................10 5.2 STCP......................................................10 5.2.1 Congestion Avoidance..................................11 5.2.2 Reliability...........................................11 5.2.3 Exporting Process.....................................11 5.2.3.1 MTU size.............................................11 5.2.3.2 Source ID............................................11 5.2.3.3 Association..........................................12 5.2.3.4 Template.............................................12 5.2.3.5 Template.............................................12 5.2.4 Collecting Process....................................13 5.2.5 SCTP Partially Reliable...............................13 6. Failover...................................................14 6.1 Simple Failover based on the transport protocol...........14 6.2 Something else?...........................................14 7. Packet Layout..............................................14 8. Export Packet Format.......................................16 8.1 Header Format.............................................16 8.2 Template FlowSet Format...................................17 8.3 Data FlowSet Format.......................................19 9. Options....................................................20 9.1 Options Template FlowSet Format...........................20 9.2 Options Data Record Format................................22 10. Variable Length Data Type.................................24 11. Template Management.......................................25 12. The Collecting Process's Side.............................26 13. Examples..................................................27 13.1 Packet Header Example....................................28 13.2 Template FlowSet Example.................................28 13.3 Data FlowSet Example.....................................29 13.4 Options Template FlowSet Example.........................30 13.5 Data FlowSet with Options Data Records Example...........31 14. References................................................31 Claise, et. al Standard Track [Page 2] IPFIX Protocol Specifications October 2003 15. Acknowledgments...........................................32 1. Points of Discussion 1.1 Open Issues This section covers the open issues, still to be resolved/updated in this draft: - The proposal on the table is to send a IPFIX Sync (this would be an Options Data Records) message periodically (periodicity is configurable), with the following information (aside the standard IPFix header) * Number of flow records sent (for each template?) * Packets and bytes sent (for each template?) Question: Per observation domain? Question: Do we need a specific FlowSet ID? - Template don't need lifetimes with connection oriented protocol. We guess this is the consensus from the Working Group. - No periodic retransmission of templates is needed, with a reliable transport protocol. Remark: the template management will vary with TCP, SCTP, etcà Must have both sections updated: transport updated and template management sections (BTW, this is the same for the failover section). - There seems to be a consensus that having a length field in the Export Packet header. Related question: what about the count field then in the NetFlow version 9 header? But no consensus yet. So no consensus whether the current count should simply be replaced by the length or the length field be added. - Sub-second timestamps - Export ID (ie IP address of exporter) sent to the collector. This can be done once with an options template. - Metering process stats: for example packets / flows dropped at the metering process due to resource exhaustion, etcà This can be done with an options template. - Error recovery, for example what to do if a collector receives a message it can't decode. Per protocol issue, ie TCP reset the session because it's a stream protocol and can't recover. - If we tackle reliablilty a state diagram is needed. - Talk a little bit about extensibility, for example make some Claise, et. al Standard Track [Page 3] IPFIX Protocol Specifications October 2003 decisions now about the reserved template ID's 2-254 for future work. 1.2 Action Items This section covers the action items for this draft - change the abstract: done, to be reviewed -> Reinaldo - introduction to be completed: introduction + references to other drafts + applications for the protocol. -> Reinaldo - ôtransport protocolö section to be updated. -> TCP, Reinaldo Note: each transport protocol (TCP/SCTP) should have at least the following subsections: Reliability, Failover, Template Management - Review the requirements draft to see what we miss! -> Benoit - variable length data types -> Text and packet format to be worked on. Benoit - field types information -> should we have basic type in the protocol draft, like Jeff suggested. -> Mark: will send an abstract - add references to the IPFIX drafts, when published 2. Introduction 2.1 Overview Will be completed. It will cover an introduction, pointers/explanation towards/of the other drafts, and the applications for the IPFIX protocol 3. Terminology The definitions of IP Traffic Flow, Metering Process, Exporting Process and Collecting Process are the definitions as found in the IPFIX requirement document [IPFIX-REQ]. Note nevertheless that, even if the Observation Point definition has been also been completely copied over from the IPFIX requirement document [IPFIX- REQ], this definition has been expanded. The terminology summary table in Section 3.1 gives a quick overview of the relationships between some of the different terms defined. Claise, et. al Standard Track [Page 4] IPFIX Protocol Specifications October 2003 Observation Point The Observation Point is a location in the network where IP packets can be observed. Examples are a line to which a probe is attached, a shared medium such as an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) of a router. An Observation Domain is associated with every Observation Point. Note that one Observation Point may be a superset of several other Observation Points. For example one Observation Point can be an entire line card. This would be the superset of the individual Observation Points at the line card's interfaces. Observation Domain The set of Observation Points, which is the largest aggregatable set of Flow information at the Metering Process is termed an Observation Domain. Each Observation Domain presents itself as a unique ID to the Collecting Process for identifying the Export Packets it generates. For example, a router line card composed of several interfaces with each interface being an Observation Point. IP Traffic Flow or Flow There are several definitions of the term 'flow' being used by the Internet community. Within this document we use the following one: A Flow is defined as a set of IP packets passing an Observation Point in the network during a certain time interval. All packets belonging to a particular Flow have a set of common properties. Each property is defined as the result of applying a function to the values of: 1. one or more packet header field (e.g. destination IP address), transport header field (e.g. destination port number), or application header field (e.g. RTP header fields [RFC1889]) 2. one or more characteristics of the packet itself (e.g. number of MPLS labels, etc...) 3. one or more of fields derived from packet treatment (e.g. next hop IP address, the output interface, etc...) Claise, et. al Standard Track [Page 5] IPFIX Protocol Specifications October 2003 A packet is defined to belong to a Flow if it completely satisfies all the defined properties of the Flow. This definition covers the range from a Flow containing all packets observed at a network interface to a Flow consisting of just a single packet between two applications with a specific sequence number. Please note that the Flow definition does not necessarily match a general application-level end-to-end stream. However, an application may derive properties of application-level streams by processing measured Flow data. Also, please note that although packet properties may depend on application headers, there is no requirement defined in this document related to application headers. Flow Record A Flow Record provides information about an IP Traffic Flow observed at an Observation Point. Metering Process The Metering Process generates Flow Records. Input to the process are packet headers observed at an Observation Point and packet treatment at the Observation Point, for example the selected output interface. The Metering Process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining Flow Records. The maintenance of Flow Records may include creating new records, updating existing ones, computing Flow statistics, deriving further Flow properties, detecting Flow expiration, passing Flow Records to the Exporting Process, and deleting Flow Records. The sampling function and the classifying function may be applied more than once with different parameters. Figure 1 shows the sequence in which the functions are applied. Sampling is not illustrated in the figure, it may be applied before any other function. packet header capturing | timestamping Claise, et. al Standard Track [Page 6] IPFIX Protocol Specifications October 2003 | v +----->+ | | | classifying | | +------+ | maintaining Flow Records | v Figure 1: Functions of the Metering Process Exporting Process The Exporting Process sends Flow Records to one or more Collecting Processes. The Flow Records are generated by one or more Metering Processes. Collecting Process The Collecting Process receives Flow Records from one or more Exporting Processes. The Collecting Process might store received Flow Records or further process them, but these actions are out of the scope of this document. Export Packet An Export Packet is a packet originating at the Exporting Process that carries the Flow Records of this Exporting Process and whose destination is the Collecting Process. Packet Header The Packet Header is the first part of an Export Packet, which provides basic information about the packet such as the IPFIX version, number of records contained within the packet, sequence numbering, etc. Template Record Claise, et. al Standard Track [Page 7] IPFIX Protocol Specifications October 2003 A Template Record defines the structure and interpretation of fields in a Flow Data Record. Flow Data Record A Flow Data Record is a data record that contains values of the Flow parameters corresponding to a Template Record. In this document, the Flow Data Record is also referred to as Flow Record. Options Template Record An Options Template Record defines the structure and interpretation of fields in an Options Data Record, including defining how to scope the applicability of the Options Data Record. Options Data Record The Options Data Record is a data record that contains values and scope information of the Flow measurement parameters, corresponding to an Options Template Record. FlowSet FlowSet is a generic term for a collection of records that have a similar structure. In an Export Packet, one or more FlowSets follow the Packet Header. There are three different types of FlowSets: Template FlowSet, Options Template FlowSet, and Data FlowSet. Template FlowSet A Template FlowSet is a collection of one or more Template Records that have been grouped together in an Export Packet. Options Template FlowSet An Options Template FlowSet is a collection of one or more Options Template Records that have been grouped together in an Export Packet. Data FlowSet Claise, et. al Standard Track [Page 8] IPFIX Protocol Specifications October 2003 A Data FlowSet is one or more records, of the same type, that are grouped together in an Export Packet. Each record is either a Flow Data Record or an Options Data Record previously defined by a Template Record or an Options Template Record. 3.1 Terminology Summary Table FlowSet Template Record Data Record +----------------------------------------------------------------+ | | | Flow Data Record(s) | | Data FlowSet | / | or | | | | Options Data Record(s) | +----------------------------------------------------------------+ | Template FlowSet | Template Record(s) | / | +----------------------------------------------------------------+ | Options Template | Options Template | / | | FlowSet | Record(s) | | +----------------------------------------------------------------+ A Data FlowSet is composed of an Options Data Record(s) or Flow Data Record(s). No Template Record is included. A Template Record defines the Flow Data Record, and an Options Template Record defines the Options Data Record. A Template FlowSet is composed of Template Record(s). No Flow or Options Data Record is included. An Options Template FlowSet is composed of Options Template Record(s). No Flow or Options Data Record is included. 4. The Metering Process The description of the Metering Process (for example, sampled or not, aggregation or not), which is the way in which Flows are created from the observed IP packets, is beyond the scope of this document. Nevertheless, some aspects of the Metering Process have some influences on the IPFIX protocol 4.1 Flow Expiration A Flow is considered to be inactive if no packets belonging to the Flow have been observed at the Observation Point for a given Claise, et. al Standard Track [Page 9] IPFIX Protocol Specifications October 2003 timeout. If any packet is seen within the timeout, the flow is considered an active Flow. A Flow can be exported under the following conditions: 1. If the Metering Process can detect the end of a Flow. For example, if the FIN or RST bit is detected in a TCP [3] connection, the Flow Record is exported. 2. If the Flow has been inactive for a certain period of time. This inactivity timeout SHOULD be configurable at the Metering Process, with a minimum value of 0 for an immediate expiration. 3. For long-lasting Flows, the Exporting Process SHOULD export the Flow Records on a regular basis. This timeout SHOULD be configurable at the Metering Process. 4. If the Metering Process experiences internal constraints, a Flow MAY be forced to expire prematurely; for example, counters wrapping or low memory. 5. Transport Protocol The IPFIX Protocol Specifications have been designed to be transport protocol independent. It can operate over congestion-aware protocols such as TCP [3] or SCTP [RFC2960]. Note that the Exporter can export to multiple Collecting Processes, using independent transport protocols. 5.1 TCP To be completed. TCP [3] 5.2 STCP This section describes how IPFIX can be transported over SCTP [RFC2960] using traditional reliable mode. IPFIX can also be transported over the partial reliable or unreliable mode [PR-SCTP]. These last 2 modes will be briefly discussed, while waiting for [PR-SCTP] to become a standard. Claise, et. al Standard Track [Page 10] IPFIX Protocol Specifications October 2003 5.2.1 Congestion Avoidance The SCTP transport protocol provides the required level of congestion avoidance by design. 5.2.2 Reliability The SCTP transport protocol is by default reliable, but has the capability to operate in unreliable and partially reliable modes [PR-SCTP]. Using reliable SCTP streams for the IPFIX export is not in itself a guarantee that all records are delivered. If there is congestion on the link from the exporter to the collector, or if a significant amount of retransmissions are needed, the send queues on the Exporting Process may fill up. In that case it's up to the Exporting Process to decide what to do. It may either halt export (buffer the data until there is space in the send queues again) or just throw Export Packets away instead of inserting them into the send queue. If any data is not inserted into the send queues, the sequence numbers used for export must reflect the loss of data. 5.2.3 Exporting Process 5.2.3.1 MTU size Each IPFIX Export Packet should be equal to or less than the local MTU in size. When an IPFIX Export Packet is transmitted over a network with an MTU smaller than the local MTU, IP fragmentation may be used. 5.2.3.2 Source ID The Export Packet must contain a Packet Header, which includes a source id (SID). The SID indicates from which Observation Domain the data is being exported, and should be kept unique for each such domain. If a Metering Process consists of a single Observation Domain, a single SID value must be used for all Export Packets. The Exporting Process will typically open one association to the collector, but more are possible, in which one or more streams can be used. The Claise, et. al Standard Track [Page 11] IPFIX Protocol Specifications October 2003 Exporting Process has the choice of transmitting parts of the export data in separate streams or all data in one stream. If a Metering Process consists of multiple Observation Domains, one SID value for each Observation Domain must be used. The Exporting Process will typically open one association, but more are possible, in which at least one stream per Observation Domain is used. The Exporting Process has the choice of using more than one stream per Observation Domain, but data from multiple Observation Domains should not be transmitted over the same stream. 5.2.3.3 Association The Exporting Process may create one or more associations (connection "bundle" in SCTP terminology) to the Collecting Process. The Collecting Process may not initiate the connection. Inside each association one or more streams may be requested by the Exporting Process. If the Collecting Process can not support the requested number of streams, it may choose to refuse the connection and the Exporting Process should try to reduce, if possible, the number of streams needed to perform the export. 5.2.3.4 Template An Observation Domain must use at least one stream, but may use multiple streams, to export data records. The Observation Domain must use the same SID value for all streams used. An Exporting Process must not transmit packets with different SID values in one stream, the Collecting Process should however verify that the SID values are the expected values. 5.2.3.5 Template Since the SCTP association is connection oriented the available Template Records must be transmitted from each Observation Domain to the Collecting Process immediately after the association is established. As a minimum the Template Records must be transmitted immediately after they start to exist on the Metering Process and should Claise, et. al Standard Track [Page 12] IPFIX Protocol Specifications October 2003 preferably be transmitted before any data, using the new Template Record, have been transmitted. The Collecting Process should however accept data without a Template Record. When using a reliable mode for Template Record export, or if the exporter knows that the Export Packet containing the templates was positively acknowledged by the SCTP layer, it is not necessary to periodically export the Template Records. 5.2.4 Collecting Process The Collecting Process should listen for a new association request from the Exporting Process. The Collecting Process will request a number of streams to use for export. If the Collecting Process doesn't support the number of streams inside the association, the Collecting Process must refuse the connection and continue listen for a new request. When data is received from an association, the Collecting Process must correlate data, with the same SID value, from multiple streams into one export Flow from an Observation Domain. This allows the Observation Domain to use separate streams for different types of information. The Collecting Process should verify that the received Export Packets inside one stream does not have diverting SID values. The Exporting Process must not export packets inside one stream with multiple SID values. The correlated Flow Records are then treated like a normal export Flow. 5.2.5 SCTP Partially Reliable This mode will not be discussed any further until [PR-SCTP] becomes a standard, even if this mode offers a few advantages: freedom to use SCTP as a reliable, single stream transport, as well as multiple streams with different properties, for example in terms of reliability, carrying different data types dependant on their importance for the system. Unreliable or partial reliability may be chosen for one or more streams inside an association. Unreliable transport may be preferred where large amount of data is to be exported and keeping send queues is either an unnecessary overhead or impractical. Partial reliability may be chosen where a small amount of queuing is Claise, et. al Standard Track [Page 13] IPFIX Protocol Specifications October 2003 possible. Naturally it is better to send templates over a reliable stream and send the data on an unreliable (or partial reliable) stream. When an exporter handles data with different properties it might even be preferable to send them over different streams according to those properties. Example: an Exporting Process can use two streams per Observation Domain. A reliable stream could be used for exporting templates, to reduce the likelihood of loss and to remove the need for blind retransmissions, and a partial or unreliable stream for data, to avoid buffering of large amounts of data. 6. Failover When to fail over? How to fail back? How to ensure stability of the failover mechanism (prevent oscillations)? Does the exporter open connections to all the potential collectors and keep them primed with template info? 6.1 Simple Failover based on the transport protocol In case the transport protocol is connection oriented. So in case of TCP [3] or SCTP [RFC2960]. To be completed. 6.2 Something else? Potentially based on some application level ACK from the exporter? 7. Packet Layout An Export Packet consists of a Packet Header followed by one or more FlowSets. The FlowSets can be any of the possible three types: Template, Data, or Options Template. Export Packet: +--------+-------------------------------------------+ | | +----------+ +---------+ +----------+ | | Packet | | Template | | Data | | Options | | Claise, et. al Standard Track [Page 14] IPFIX Protocol Specifications October 2003 | Header | | FlowSet | | FlowSet | | Template | ... | | | | | | | | FlowSet | | | | +----------+ +---------+ +----------+ | +--------+-------------------------------------------+ A FlowSet ID is used to distinguish the different types of FlowSets. FlowSet IDs lower than 256 are reserved for special FlowSets, such as the Template FlowSet (ID 0) and the Options Template FlowSet (ID 1). The Data FlowSets have a FlowSet ID greater than 255. The format of the Template, Data, and Options Template FlowSets will be discussed later in this document. The Exporter MUST code all fields of the different FlowSets in network byte order (big-endian). Following are some examples of export packets: 1. An Export Packet consisting of interleaved Template, Data, and Options Template FlowSets-A newly created Template is exported as soon as possible. So if there is already an Export Packet with a Data FlowSet that is being prepared for export, the Template and Option FlowSets are also interleaved with this information, subject to availability of space. Export Packet: +--------+--------------------------------------------------------+ | | +----------+ +---------+ +-----------+ +---------+ | | Packet | | Template | | Data | | Options | | Data | | | Header | | FlowSet | | FlowSet | ... | Template | | FlowSet | | | | | | | | | FlowSet | | | | | | +----------+ +---------+ +-----------+ +---------+ | +--------+--------------------------------------------------------+ 2. An Export Packet consisting entirely of Data FlowSets-After the appropriate Template Records have been defined and transmitted to the NetFlow Collector device, the majority of Export Packets consists solely of Data FlowSets. Export Packet: +--------+----------------------------------------------+ | | +---------+ +---------+ +---------+ | | Packet | | Data | ... | Data | ... | Data | | | Header | | FlowSet | ... | FlowSet | ... | FlowSet | | | | +---------+ +---------+ +---------+ | Claise, et. al Standard Track [Page 15] IPFIX Protocol Specifications October 2003 +--------+----------------------------------------------+ 3. An Export Packet consisting entirely of Template and Options Template FlowSets-The Exporter MAY transmit a packet containing Template and Options Template FlowSets periodically to help ensure that the NetFlow Collector has the correct Template Records and Options Template Records when the corresponding Flow Data records are received. Export Packet: +--------+-------------------------------------------------+ | | +----------+ +----------+ +----------+ | | Packet | | Template | | Template | | Options | | | Header | | FlowSet | ... | FlowSet | ... | Template | | | | | | | | | FlowSet | | | | +----------+ +----------+ +----------+ | +--------+-------------------------------------------------+ 8. Export Packet Format 8.1 Header Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version Number | Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | sysUpTime | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UNIX Secs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Packet Header Field Descriptions Version Version of Flow Record format exported in this packet. The value of this field is 0x0009 for the current version. Count Claise, et. al Standard Track [Page 16] IPFIX Protocol Specifications October 2003 The total number of records in the Export Packet, which is the sum of Options FlowSet records, Template FlowSet records, and Data FlowSet records. SysUpTime Time in milliseconds since this device was first booted. Refer to [1]. UNIX Secs Seconds since 0000 UTC 1970. Sequence Number Incremental sequence counter of all Export Packets sent from the current Observation Domain by the Exporting Process. This value MUST be cumulative, and SHOULD be used by the Collector to identify whether any Export Packets have been missed. Source ID A 32-bit value that identifies the Exporter Process Observation Domain. Collecting Process SHOULD use the combination of the source IP address and the Source ID field to separate different export streams originating from the same Exporting Process. 8.2 Template FlowSet Format One of the essential elements in the IPFIX format is the Template FlowSet. Templates greatly enhance the flexibility of the Flow Record format because they allow the Collecting Process to process Flow Records without necessarily knowing the interpretation of all the data in the Flow Record. The format of the Template FlowSet is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 0 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 1 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | Claise, et. al Standard Track [Page 17] IPFIX Protocol Specifications October 2003 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type N | Field Length N | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 2 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type M | Field Length M | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID K | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Template FlowSet Field Descriptions FlowSet ID FlowSet ID value of 0 is reserved for the Template FlowSet. Length Total length of this FlowSet. Because an individual Template FlowSet MAY contain multiple Template Records, the Length value MUST be used to determine the position of the next FlowSet record, which could be any type of FlowSet. Length is the sum of the lengths of the FlowSet ID, the Length itself, and all Template Records within this FlowSet. Template ID Each of the newly generated Template Records is given a unique Template ID. This uniqueness is local to the Observation Domain that generated the Template ID. Template IDs 0-255 are reserved for Template FlowSets, Options FlowSets, and other reserved FlowSets yet to be Claise, et. al Standard Track [Page 18] IPFIX Protocol Specifications October 2003 created. Template IDs of Data FlowSets are numbered from 256 to 65535. Field Count Number of fields in this Template Record. Because a Template FlowSet usually contains multiple Template Records, this field allows the Collecting Process to determine the end of the current Template Record and the start of the next. Field Type A numeric value that represents the type of the field. Refer to the "Field Type Definitions" section. Field Length The length of the corresponding Field Type, in bytes. Refer to the "Field Type Definitions" section. 8.3 Data FlowSet Format The format of the Data FlowSet is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = Template ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 1 | Record 1 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 1 | Record 2 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 3 - Field Value 1 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Data FlowSet Field Descriptions FlowSet ID = Template ID Claise, et. al Standard Track [Page 19] IPFIX Protocol Specifications October 2003 Each Data FlowSet is associated with a FlowSet ID. The FlowSet ID maps to a (previously generated) Template ID. The Collecting Process MUST use the FlowSet ID to find the corresponding Template Record and decode the Flow Records from the FlowSet. Length The length of this FlowSet. Length is the sum total of lengths of FlowSet ID, Length itself, all Flow Records within this FlowSet, and the padding bytes, if any. Record N - Field Value M The remainder of the Data FlowSet is a collection of Flow Data Record(s), each containing a set of field types and values. The Type and Length of the fields have been previously defined in the Template Record referenced by the FlowSet ID or Template ID. Padding The Exporting Process SHOULD insert some padding bytes so that the subsequent FlowSet starts at a 4-byte aligned boundary. It is important to note that the Length field includes the padding bits. Interpretation of the Data FlowSet format can be done only if the Template FlowSet corresponding to the Template ID is available at the Collecting Process. 9. Options 9.1 Options Template FlowSet Format The Options Template Record (and its corresponding Options Data Record) is used to supply information about the Metering Process configuration or Metering Process specific data, rather than supplying information about IP Flows. For example, the Options Template FlowSet can report the sample rate of a specific interface, if sampling is supported, along with the sampling method used. The format of the Options Template FlowSet follows. Claise, et. al Standard Track [Page 20] IPFIX Protocol Specifications October 2003 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 1 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID | Option Scope Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option Length | Scope 1 Field Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope 1 Field Length | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope N Field Length | Option 1 Field Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option 1 Field Length | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option M Field Length | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Options Template FlowSet Field Definitions FlowSet ID = 1 A FlowSet ID value of 1 is reserved for the Options Template. Length Total length of this FlowSet. Each Options Template FlowSet MAY contain multiple Options Template Records. Thus, the Length value MUST be used to determine the position of the next FlowSet record, which could be either a Template FlowSet or Data FlowSet. Length is the sum total of lengths of FlowSet ID, the Length itself, and all Options Template Records within this FlowSet Template ID. Template ID Template ID of this Options Template. This value is greater than 255. Option Scope Length The length in bytes of any Scope fields definition contained in the Options Template Record (The use of "Scope" is described below). Claise, et. al Standard Track [Page 21] IPFIX Protocol Specifications October 2003 Option Length The length (in bytes) of any options field definitions contained in this Options Template Record. Scope 1 Field Type The relevant portion of the Exporting Process/Metering Process to which the Options Template Record refers. Currently defined values are: 1 System 2 Interface 3 Line Card 4 Cache 5 Template For example, the Metering Process can be implemented on a per-interface basis, so if the Options Template Record were reporting on how the NetFlow process is configured, the Scope for the report would be 2 (interface). The associated interface ID would then be carried in the associated Options Data FlowSet. The Scope can be limited further by listing multiple scopes that all must match at the same time. Note that the Scope fields always precede the Option fields. Scope 1 Field Length The length (in bytes) of the Scope field, as it would appear in an Options Data Record. Option 1 Field Type A numeric value that represents the type of field that would appear in the Options Template Record. Refer to the Field Type Definitions section. Option 1 Field Length The length (in bytes) of the Option Field. Padding The Exporting Process SHOULD insert some padding bytes so that the subsequent FlowSet starts at a 4-byte aligned boundary. It is important to note that the Length field includes the padding bits. 9.2 Options Data Record Format Claise, et. al Standard Track [Page 22] IPFIX Protocol Specifications October 2003 The Options Data Records are sent in Data FlowSets, on a regular basis, but not with every Flow Data Record. How frequently these Options Data Records are exported is configurable. See the Templates Management" section for more details. The format of the Data FlowSet containing Options Data Records follows. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = Template ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Scope 1 Value |Record 1 - Option Field 1 Value| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Record 1 - Option Field 2 Value| ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Scope 1 Value |Record 2 - Option Field 1 Value| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Record 2 - Option Field 2 Value| ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 3 - Scope 1 Value |Record 3 - Option Field 1 Value| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Record 3 - Option Field 2 Value| ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Options Data Records of the Data FlowSet Field Descriptions FlowSet ID = Template ID A FlowSet ID precedes each group of Options Data Records within a Data FlowSet. The FlowSet ID maps to a previously generated Template ID corresponding to this Options Template Record. The Collecting Process MUST use the FlowSet ID to map the appropriate type and length to any field values that follow. Length The length of this FlowSet. Length is the sum of the lengths of the FlowSet ID, Length itself, all the Options Data Records within this FlowSet, and the padding bytes, if any. Claise, et. al Standard Track [Page 23] IPFIX Protocol Specifications October 2003 Record N - Option Field M Value The remainder of the Data FlowSet is a collection of Flow Records, each containing a set of scope and field values. The type and length of the fields were previously defined in the Options Template Record referenced by the FlowSet ID or Template ID. Padding The Exporting Process SHOULD insert some padding bytes so that the subsequent FlowSet starts at a 4-byte aligned boundary. It is important to note that the Length field includes the padding bits. The Data FlowSet format can be interpreted only if the Options Template FlowSet corresponding to the Template ID is available at the Collecting Process. 10. Variable Length Data Type Note: this section reflects the mailing list ideas but the wording should be worked on. In the Template Flowset (template definition), let's put FFFF for the length of the variable length data types. Then, the actual length is encoded in data Flowset (Flow Records) as follows: - if length is < 255 bytes, the length is stored in 1 byte - if length is >=255 bytes, 255 is stored in the first byte, and the actual 16 bits length is stored in the next 2 bytes. The cases of length of >= 255 will be very rear. And even in these cases, one extra byte will represent <= 1/255-th part of data record. Why FFFF and not 0? The reason for this is that FFFF is a truly invalid value which should be caught in collectors, by good implementations, today. For 0 this isn't necessary the case as it's meaning is "just" pointless, but not a course of concern for the collectors. Basically FFFF should make the Collecting Process (current implementations) discard the template as being invalid. Note also that some option templates data types could potentially have a length of 0. Claise, et. al Standard Track [Page 24] IPFIX Protocol Specifications October 2003 So the advantages of this proposal are: - code the length on 1 byte for most of the cases; in that case, we use the 8 bits to code the length, so with a max of 254 bytes (because 255 is reserved) - if the length is above 254, we use 3 bytes, the last 2 bytes could transport a length of max 64 k - 7 bytes (in the Data FlowSet, 2 for the FlowSet ID + 2 for the FlowSet Length + 1 byte for "0xff" + 2 for the variable length) 11. Template Management Flow Data records that correspond to a Template Record MAY appear in the same and/or subsequent Export Packets. The Template Record is not necessarily carried in every Export Packet. As such, the Collecting Process MUST store the Template Record to interpret the corresponding Flow Data Records that are received in subsequent data packets. A Collecting Process that receives Export Packets from several Observation Domains from the same Exporter MUST be aware that the uniqueness of the Template ID is not guaranteed across Observation Domains. The Template IDs must remain constant for the life of the Metering Process and the Exporting Process. If the Exporting Process or the Metering Process restarts for any reason, all information about Templates will be lost and new Template IDs will be created. Template IDs are thus not guaranteed to be consistent across an Exporting Process or Metering Process restart. A newly created Template record is assigned an unused Template ID from the Exporter. If the template configuration is changed, the current Template ID is abandoned and SHOULD NOT be reused until the Metering Process. If a Collecting Process should receive a new definition for an already existing Template ID, it MUST discard the previous template definition and use the new one. If a configured Template Record on the Exporting Process is deleted, and re-configured with exactly the same parameters, the same Template ID COULD be reused. Claise, et. al Standard Track [Page 25] IPFIX Protocol Specifications October 2003 The Exporting Process sends the Template FlowSet and Options Template FlowSet under the following conditions: 1. After a NetFlow process restarts, the Exporting Process MUST NOT send any Data FlowSet without sending the corresponding Template FlowSet and the required Options Template FlowSet in a previous packet or including it in the same Export Packet. It MAY transmit the Template FlowSet and Options Template FlowSet, without any Data FlowSets, in advance to help ensure that the Collector will have the correct Template Record before receiving the first Flow or Options Data Record. 2. In the event of configuration changes, the Exporting Process SHOULD send the new template definitions at an accelerated rate. In such a case, it MAY transmit the changed Template Record(s) and Options Template Record(s), without any data, in advance to help ensure that the Collector will have the correct template information before receiving the first data. 3. On a regular basis, the Exporting Process MUST send all the Template Records and Options Template Records to refresh the Collecting Process. Template IDs have a limited lifetime at the Collecting Process and MUST be periodically refreshed. Two approaches are taken to make sure that Templates get refreshed at the Collecting Process: * Every N number of Export Packets. * On a time basis, so every N number of minutes. Both options MUST be configurable by the user on the Exporting Porcess. When one of these expiry conditions is met, the Exporting Process MUST send the Template FlowSet and Options Template. 12. The Collecting Process's Side The Collecting Process receives Template Records from the Exporting Process, normally before receiving Flow Data Records (or Options Data Records). The Flow Data Records (or Options Data Records) can then be decoded and stored locally on the devices. If the Template Records have not been received at the time Flow Data Records (or Options Data Records) are received, the Collecting Process SHOULD store the Flow Data Records (or Options Data Records) and decode them after the Template Records are received. A Collecting Process device MUST NOT assume that the Data FlowSet and the associated Claise, et. al Standard Track [Page 26] IPFIX Protocol Specifications October 2003 Template FlowSet (or Options Template FlowSet) are exported in the same Export Packet. The Collecting Process MUST NOT assume that one and only one Template FlowSet is present in an Export Packet. The life of a template at the Collecting Process is limited to a fixed refresh timeout. Templates not refreshed from the Exporting Process within the timeout are expired at the Collecting Process. The Collecting Process MUST NOT attempt to decode the Flow or Options Data Records with an expired Template. At any given time the Collecting Process SHOULD maintain the following for all the current Template Records and Options Template Records: Note that the Observation Domain is identified by the Source ID field from the Export Packet. Template IDs are unique per Exporting Process and per Observation Domain. If the Collecting Process receives a new Template Record (for example, in the case of an Exporter restart) it MUST immediately override the existing Template Record. Finally, note that the Collector MUST accept padding in the Data FlowSet and Options Template FlowSet, which means for the Flow Data Records, the Options Data Records and the Template Records. Refer to the terminology summary table in Section 3.1. 13. Examples Let's consider the example of an Export Packet composed of a Template FlowSet, a Data FlowSet (which contains three Flow Data Records), an Options Template FlowSet and a Data FlowSet (which contains 2 Options Data Records). Export Packet: +--------+---------------------------------------------. . . | | +--------------+ +-----------------------+ | Packet | | Template | | Data | | Header | | FlowSet | | FlowSet | . . . | | | (1 Template) | | (3 Flow Data Records) | | | +--------------+ +-----------------------+ Claise, et. al Standard Track [Page 27] IPFIX Protocol Specifications October 2003 +--------+---------------------------------------------. . . . . .+-------------------------------------------------+ +------------------+ +--------------------------+ | | Options | | Data | | . . .| Template FlowSet | | FlowSet | | | (1 Template) | | (2 Options Data Records) | | +------------------+ +--------------------------+ | . . .--------------------------------------------------+ 13.1 Packet Header Example The Packet Header is composed of: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version = 0x0009 | Count = 7 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | sysUpTime | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UNIX Secs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 13.2 Template FlowSet Example We want to report the following Field Types: - The source IP address (IPv4), so the length is 4 - The destination IP address (IPv4), so the length is 4 - The next-hop IP address (IPv4), so the length is 4 - The number of bytes of the Flow - The number of packets of the Flow Therefore, the Template FlowSet will be composed of the following: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 0 | Length = 28 bytes | Claise, et. al Standard Track [Page 28] IPFIX Protocol Specifications October 2003 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 256 | Field Count = 5 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP_SRC_ADDR = 0x0008 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP_DST_ADDR = 0x000C | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP_NEXT_HOP = 0x000F | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IN_PKTS = 0x0002 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IN_BYTES = 0x0001 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 13.3 Data FlowSet Example In this example, we report the following three Flow records: Src IP addr. | Dst IP addr. | Next Hop addr. | Packet | Bytes | | | Number | Number --------------------------------------------------------------- 198.168.1.12 | 10.5.12.254 | 192.168.1.1 | 5009 | 5344385 192.168.1.27 | 10.5.12.23 | 192.168.1.1 | 748 | 388934 192.168.1.56 | 10.5.12.65 | 192.168.1.1 | 5 | 6534 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 256 | Length = 64 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 198.168.1.12 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10.5.12.254 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 5009 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 5344385 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.27 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10.5.12.23 | Claise, et. al Standard Track [Page 29] IPFIX Protocol Specifications October 2003 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 748 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 388934 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.56 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10.5.12.65 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 5 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 6534 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Note that padding was not necessary in this example. 13.4 Options Template FlowSet Example Per line card (the router being composed of two line cards), we want to report the following Field Types: - Total number of Export Packets - Total number of exported Flows The format of the Options Template FlowSet is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 1 | Length = 24 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 257 | Option Scope Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option Length = 8 | Scope 1 Field Type = 0x0003 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope 1 Field Length = 2 | TOTAL_EXP_PKTS_SENT = 41 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Length = 4 | TOTAL_FLOWS_EXP = 42 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Length = 4 | Padding | Claise, et. al Standard Track [Page 30] IPFIX Protocol Specifications October 2003 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 13.5 Data FlowSet with Options Data Records Example In this example, we report the following two records: Line Card ID | Export Packet| Export Flow ------------------------------------------ Line Card 1 | 345 | 10201 Line Card 2 | 690 | 20402 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 257 | Length = 14 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 1 | 345 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10201 | 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 2 | 690 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 20402 | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 14. References [1] R. Presuhn et al, "Management Information Base (MIB) for the Simple Network Management Protocol (SNMP)" RFC 3418, December 2002 [2] J. Postel, "User Datagram Protocol" RFC 768, August 1980 [3] "TRANSMISSION CONTROL PROTOCOL DARPA INTERNET PROGRAM PROTOCOL SPECIFICATION" RFC 793, September 1981 [IPFIX-REQ] J. Quittek, T. Zseby, B. Claise, S. Zander, "Requirements for IP Flow Information Export" draft-ietf-ipfix-reqs- 10.txt, June 2003 [IPFIX-ARCH] Sadasivan, G, Brownlee, N. "Architecture Model for IP Flow Information Export" draft-ietf-ipfix-arch-01.txt", June 2003 Claise, et. al Standard Track [Page 31] IPFIX Protocol Specifications October 2003 [IPFIX-INFO] Calato, P, Meyer, J, Quittek, J, "Information Model for IP Flow Information Export" draft-ietf-ipfix-info-01, August 2003 [IPFIX-PROTO] Claise, B, Fullmer, M, Calato, P, Penno, R, "IPFIX Protocol Specifications", draft-ietf-ipfix-protocol-00.txt, June 2003 [RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol", RFC 2960, October 2000 [PR-SCTP] Stewart, R, "SCTP Partial Reliability Extension", draft- stewart-tsvwg-prsctp-04.txt [NETFLOW9] Claise, B, et al "Cisco Systems NetFlow Services Export Version 9", draft-claise-netflow-9-06.txt, October 2003 5. Acknowledgments To be completed. uthors Addresses Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 E-mail: bclaise@cisco.com Mark Fullmer OARnet 2455 North Star Rd. Columbus, Ohio 43221 Phone: +1 (614) 728-8100 Email: maf@eng.oar.net Reinaldo Penno Nortel Networks 2305 Mission College Blvd Santa Clara, CA 95054 Phone: +1 408.565.3023 Email: rpenno@nortelnetworks.com Claise, et. al Standard Track [Page 32] IPFIX Protocol Specifications October 2003 Paul Calato Riverstone Networks, Inc. 5200 Great America Parkway Santa Clara, CA 95054 USA Phone: +1 (603) 557-6913 Email: calato@riverstonenet.com Claise, et. al Standard Track [Page 33]