IP Flow Information Export WG G. Sadasivan (ipfix) Cisco Systems, Inc. Internet-Draft N. Brownlee Expires: November 30, 2004 CAIDA | The University of Auckland B. Claise Cisco Systems, Inc. J. Quittek NEC Europe Ltd. June 2004 Architecture for IP Flow Information Export draft-ietf-ipfix-architecture-03 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on November 30, 2004. Copyright Notice Copyright (C) The Internet Society (2004). All Rights Reserved. Abstract This memo defines the IPFIX architecture for the selective monitoring of network traffic flows, and for the export of measured IP flow information from an IPFIX device to a Collector, as per the requirements set out in the IPFIX Requirements document. Sadasivan, et al. Expires November 30, 2004 [Page 1] Internet-Draft IPFIX Architecture June 2004 Table of Contents 1. Changes/Issues from the -02 Draft . . . . . . . . . . . . . 4 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 5. Examples of Flows . . . . . . . . . . . . . . . . . . . . . 9 6. IPFIX reference Model . . . . . . . . . . . . . . . . . . . 11 7. IPFIX Functional and Logical blocks . . . . . . . . . . . . 13 7.1 Metering Process . . . . . . . . . . . . . . . . . . . . . 13 7.1.1 Observation Point . . . . . . . . . . . . . . . . . . 13 7.1.2 Selection Criteria for Packets . . . . . . . . . . . . 13 7.2 Observation Domain . . . . . . . . . . . . . . . . . . . . 15 7.3 Flow Recording Process . . . . . . . . . . . . . . . . . . 15 7.4 Exporting Process . . . . . . . . . . . . . . . . . . . . 15 8. Overview of the IPFIX Protocol . . . . . . . . . . . . . . . 16 8.1 Encoding Flow Data Information . . . . . . . . . . . . . . 17 8.2 Encoding Control Information . . . . . . . . . . . . . . . 17 8.3 Exporting Control Information . . . . . . . . . . . . . . 17 8.4 Export Error Handling . . . . . . . . . . . . . . . . . . 18 8.5 Flow Expiration and Export . . . . . . . . . . . . . . . . 19 8.6 Selection Criteria of flows for export . . . . . . . . . . 19 9. IPFIX Protocol Details . . . . . . . . . . . . . . . . . . . 20 9.1 The IPFIX basis protocol . . . . . . . . . . . . . . . . . 20 9.2 The Collecting Process . . . . . . . . . . . . . . . . . . 22 9.3 IPFIX Protocol on the Collecting Process . . . . . . . . . 23 9.4 Support for Applications . . . . . . . . . . . . . . . . . 23 10. Export Models . . . . . . . . . . . . . . . . . . . . . . . 23 10.1 Export with Reliable Control Connection . . . . . . . . 23 10.2 Collector Failure Detection and Recovery . . . . . . . . 24 10.3 Collector Redundancy . . . . . . . . . . . . . . . . . . 25 11. IPFIX flow collection for Special Traffic . . . . . . . . . 25 12. IPFIX flow collection from Special Devices . . . . . . . . . 25 13. Security Considerations . . . . . . . . . . . . . . . . . . 26 13.1 Data security . . . . . . . . . . . . . . . . . . . . . 26 13.1.1 No security . . . . . . . . . . . . . . . . . . . . 27 13.1.2 Authentication-only . . . . . . . . . . . . . . . . 27 13.1.3 Encryption . . . . . . . . . . . . . . . . . . . . . 27 13.2 IPFIX end point authentication . . . . . . . . . . . . . 28 14. IPFIX overload . . . . . . . . . . . . . . . . . . . . . . . 28 14.1 Denial of service (DoS) attack prevention . . . . . . . 28 14.1.1 Network under attack . . . . . . . . . . . . . . . . 28 14.1.2 Generic DoS attack on the IPFIX system . . . . . . . 28 14.1.3 IPFIX specific DoS attack . . . . . . . . . . . . . 29 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . 29 16. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 29 17. References . . . . . . . . . . . . . . . . . . . . . . . . . 30 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 30 Sadasivan, et al. Expires November 30, 2004 [Page 2] Internet-Draft IPFIX Architecture June 2004 Intellectual Property and Copyright Statements . . . . . . . 32 Sadasivan, et al. Expires November 30, 2004 [Page 3] Internet-Draft IPFIX Architecture June 2004 1. Changes/Issues from the -02 Draft Flow Aggregates: Are an optional part of the Flow Recording Process. That means that aggregate flows will need to be exported, e.g. an Exporting Process could send aggregate flows to one Collector, and the individual flows which comprise the aggregate to another. I've added text to point this out. Packet Selection functions: I've rewritten these sections so as to make it clearer what they actually do, and to point out that the selected packets may be subdivided further into actual Flows, each of which gets exported. Selective Export: I've taken this out of the draft because it doesn't seem useful, and it adds unnecessary complexity to the IPFIX system. Given that one configures an IPFIX device to select and/or sample only those packets which are of interest, surely one would always want to export them? Flow Expiration: The text says "inactivity timeout 0 means immediate expiration." That would break long-running flows into a sequence of single-packet flows - do we really want that? I've added text to explain this. MUST vs SHOULD: I've changed SHOULD to MUST in some places, where that seems more sensible to me. Comments to the IPFIX list please! Editorial: I've changed lots of the wording to improve readability, and added explanations to make descriptions clearer. 2. Introduction There are several applications e.g., usage-based accounting, traffic profiling, traffic engineering, attack/intrusion detection, QoS monitoring, that require flow-based IP traffic measurements. It is Sadasivan, et al. Expires November 30, 2004 [Page 4] Internet-Draft IPFIX Architecture June 2004 therefore important to have a standard way of exporting information related to IP flows. This document defines an architecture for IP traffic flow monitoring, measuring and exporting. It provides a high-level description of the key components and their functions. 3. Scope This document defines the architecture for IPFIX. Its main objectives are to: o Describe the key architectural components of IPFIX. o Define the architectural requirements, e.g., Recovery, Security, etc., for the IPFIX system. o Describe the characteristics of the IPFIX (flow export) protocol. Note that the IPFIX system does not provide for remote configuration of an IPFIX device. Instead, IPFIX devices are configured locally by Network Operations Staff. 4. Terminology The definition of the basic terms like IP Traffic Flow, Exporting Process, Collecting Process, Observation Point, etc. are semantically identical with those found in the IPFIX requirements document IPFIX-REQS [1]. Some of the terms have been expanded for more clarity when defining the protocol. Additional terms required for the architecture have also been defined. For the same terms defined in both this document and IPFIX-PROTO [4] the definitions are identical with IPFIX-PROTO [4]. * Observation Point An Observation Point is a location in the network where IP packets can be observed. Examples include: a line to which a probe is attached, a shared medium, such as an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) of a router. Note that one Observation Point may be a superset of several other Observation Points. For example one Observation Point can be an entire line card. That would be the superset of the individual Observation Points at the line card's interfaces. Sadasivan, et al. Expires November 30, 2004 [Page 5] Internet-Draft IPFIX Architecture June 2004 * Observation Domain The set of Observation Points which is the largest aggregatable set of flow information at the Metering Process is termed an Observation Domain. Each Observation Domain presents itself using a unique ID to the Exporting Process to identify the IPFIX messages it generates. For example, a router line card may be composed of several interfaces with each interface being an Observation Point. Every Observation Point is associated with an Observation Domain. * IP Traffic Flow or Flow There are several definitions of the term 'flow' being used by the Internet community. Within the context of IPFIX we use the following definition: A flow is defined as a set of IP packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties. Each property is defined as the result of applying a function to the values of: 1. One or more packet header field (e.g. destination IP address), transport header field (e.g. destination port number), or application header field (e.g. RTP header fields [RFC1889]) 2. One or more characteristics of the packet itself (e.g. number of MPLS labels) 3. One or more fields derived from packet treatment (e.g. next hop IP address, output interface) A packet is said to belong to a flow if it completely satisfies all the defined properties of the flow. This definition covers the range from a flow containing all packets observed at a network interface to a flow consisting of just a single packet between two applications with a specific sequence number. * Flow Key Each of the fields which 1. Belong to the packet header (e.g. destination IP address) Sadasivan, et al. Expires November 30, 2004 [Page 6] Internet-Draft IPFIX Architecture June 2004 2. Are a property of the packet itself (e.g. packet length) 3. Are derived from packet treatment (e.g. AS number) and which are used to define a Flow are termed Flow Keys. * Flow Type A function F() which would take as input a set of Flow Keys, and produce as output one or more Flows depending on the combination of values for the set of Flow Keys, is termed a Flow Type. In other words, a Flow Type F() maps sets of Flow Key values to Flows. * Flow Record A Flow Record contains information about a specific Flow that was observed at an Observation Point. A Flow Record contains measured properties of the Flow (e.g. the total number of bytes for all the Flow's packets) and usually characteristic properties of the Flow (e.g. source IP address). * Metering Process A Metering Process generates Flow Records. Input to the process are packets observed at an Observation Point, and packet treatment at the Observation Point. The Metering Process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining Flow Records. The maintenance of Flow Records may include creating new records, updating existing ones, computing Flow statistics, deriving further Flow properties, detecting Flow expiration, passing Flow Records to the Exporting Process, and deleting Flow Records. * Exporting Process An Exporting Process sends Flow Records to one or more Collecting Processes. The Flow Records are generated by one or more Metering Processes. * IPFIX Device An IPFIX device hosts at least one Observation Point, a Metering Process and an Exporting Process. Typically, corresponding Observation Point(s), Metering Process(es) and Exporting Process(es) are co-located at such a device, for example at a router. Sadasivan, et al. Expires November 30, 2004 [Page 7] Internet-Draft IPFIX Architecture June 2004 * IPFIX Node An IPFIX node is a host that implements the IPFIX protocol, i.e. it contains an Exporting Process or a Collecting Process or both. * Collecting Process A Collecting Process receives Flow Records from one or more Exporting Processes. The Collecting Process might process or store received Flow Records, but such actions are out of scope for this document. * Collector A device which hosts one or more Collecting Processes is termed a Collector. * Flow Recording Process The Flows generated from the metering device(s) in an Observation Domain MAY be collected into one or more databases before exporting. This functional block, in addition to maintaining the Flow database(s) MAY do Flow aggregation, maintain aggregate statistics etc. This block is optional for an IPFIX device. * Template A Template is an ordered list (e.g. of pairs), used to completely identify the structure and semantics of a particular set of information that needs to be communicated from an IPFIX Device to a Collector. Each template is uniquely identifiable by some means, e.g. by using a Template ID. * Control Information, Data Stream The information that needs to be exported from the IPFIX device can be classified into the following categories: Control Information This includes the Flow type definition, selection criteria for packets within the Flow sent by the Exporting Process, and any IPFIX protocol messages (e.g. keepalives). The 'control' stream carries all the information needed for the end-points to understand the IPFIX protocol, and specifically for the receiver (collector) to understand and interpret the data sent by the sender (exporter). Sadasivan, et al. Expires November 30, 2004 [Page 8] Internet-Draft IPFIX Architecture June 2004 Data Stream This includes data records carrying the field values for the various observed Flows at each of the Observation Points. IPFIX Message An IPFIX Message is a message originating at the Exporting Process that carries the IPFIX records of this Exporting Process and whose destination is the Collecting Process. An IPFIX Message is encapsulated within a transport layer header. 5. Examples of Flows Some examples of flows are listed below: Example 1: To create flows, the different fields to distinguish flows are defined. The different combination of the field values creates unique flows. If the keys are defined as {source IP address, destination IP address, DSCP}, then all of these are different flows. 1. {192.1.40.1, 171.6.23.5, 4} 2. {192.1.40.23, 171.6.23.67, 4} 3. {192.1.40.23, 171.6.23.67, 2} 4. {198.20.9.200, 171.6.23.67, 4} Example 2: To create flows, a match function can be applied to all the packets that pass through an Observation Point, in order to aggregate some values. This could be done by defining the keys as {source IP address, destination IP address, DSCP} as in example 1 above, and applying the function which masks the least significant 8 bits of the source IP address and destination IP address (i.e. the result is a /24 address). The four flows from example 1 would now be aggregated into three flows by merging the flows 1. and 2. into a single flow. 1. {192.1.40.0/24, 171.6.23.0/24, 4} 2. {192.1.40.0/24, 171.6.23.0/24, 2} 3. {198.20.9.0/24, 171.6.23.0/24, 4} Example 3: To create flows, a filter defined by some field values can be applied on all packets that pass the Observation Point, in order to select only certain flows. The filter is defined by choosing fixed values for specific fields from the packet. All the packets that go from a customer network 192.1.40.0/24 to another customer network 171.6.23.0/24 with DSCP value of 4 define a Sadasivan, et al. Expires November 30, 2004 [Page 9] Internet-Draft IPFIX Architecture June 2004 flow. All other combinations don't define a flow and are not taken into account. The three flows from example 2 would now be reduced to one flow, by filtering away the second and the third flow. {192.1.40.0/24, 171.6.23.0/24, 4}. The above example can be thought of as a function F() taking as input {source IP address, destination IP address, DSCP}. The function selects only the packets which satisfy all three of the following conditions: 1. Mask out the least significant 8 bits of source IP address, match against 192.1.40.0. 2. Mask out the least significant 8 bits of destination IP address, match against 171.6.23.0. 3. DSCP value equal to 4. Depending on the values of {source IP address, destination IP address, DSCP} of the different observed packets, the metering process function F() would choose/filter/aggregate different sets of packets, which would create different flows. For example, various combination of values of {source IP address, destination IP address, DSCP}, F(source IP address, destination IP address, DSCP) would result in the definition of one or more flows. The function F() is referred to as a Flow Type. Sadasivan, et al. Expires November 30, 2004 [Page 10] Internet-Draft IPFIX Architecture June 2004 6. IPFIX reference Model The figure below shows the reference model for IPFIX. This figure covers the various possible scenarios that can exist in an IPFIX system. +----------------+ +----------------+ |[*Application 1]| ..|[*Application n]| +--------+-------+ +-------+--------+ ^ ^ ~ ~ +~~~~~~~~~~+~~~~~~~~+ ^ ~ +------------------------+ +-------+------------------+ |IPFIX Device(1) | | Collector(1) | |[Exporting Process(es)] |<----------->| [Collecting Process(es)] | +------------------------+ +--------------------------+ .... .... +------------------------+ +---------------------------+ |IPFIX Device(i) | | Collector(j) | |[Obsv Point(s)] |<---------->| [Collecting Process(es)] | |[Metering Process(es)] | +---->| [*Application(s)] | |[Exporting Process(es)] | | +---------------------------+ +------------------------+ . .... . .... +------------------------+ | +--------------------------+ |IPFIX Device(m) | | | Collector(n) | |[Obsv Point(s)] |<-----+---->| [Collecting Process(es)] | |[Metering Process(es)] | | [*Application(s)] | |[Exporting Process(es)] | +--------------------------+ +------------------------+ The various functional components are indicated within brackets []. The functional components within [*] are not part of the IPFIX framework. The interfaces shown by "<-->" are defined by the IPFIX framework but those shown by "<~~>" are not. The figure below shows a typical IPFIX device. +--------------------------------------------------+ | IPFIX Device | | +------------------------+ (*) +-----+ | | | Flow Recording Process +----+---------> | | | | | | | E | | | +------------------------+ | | | | | ^ ^ | | x | | | |(*) |(*) | | | | Sadasivan, et al. Expires November 30, 2004 [Page 11] Internet-Draft IPFIX Architecture June 2004 | +---......--+------------+ | p | | | | | | | | | +----+----+ +----+----+ | o | | | |Metering | |Metering | | | | | |Process 1| |Process N| | r | | | |(Packet | |(Packet | | | | | | Level) | | Level) | | t | | | +---------+ +---------+ | | | | ^ ^ | i | | |+-------+-----------------------+-------+ | | | || | Observation Domain 1 | | | n | | || +-----+------+ +-----+------+| | | | || |Obsv Point 1| ... |Obsv Point M|| | g | | || +------------+ +------------+| | | | Packets|+-------^-------------------------^-----+ | | |Export --->---+--------+----------.....----------+ | | |Packet to In | | +-------> | ........ | | |Collector(s) | +------------------------+ (*) | P | | | | Flow Recording Process +----+---------> | | | | | | | r | | | +------------------------+ | | | | | ^ ^ | | o | | | |(*) | (*) | | | | | +--- ... ---+------------+ | c | | | | | | | | | +----+----+ +----+----+ | e | | | |Metering | |Metering | | | | | |Process 1| |Process N| | s | | | +---------+ +---------+ | | | | ^ ^ | s | | |+-------+-----------------------+-------+ | | | || | Observation Domain K | | | | | || +-----+------+ +-----+------+| | | | || |Obsv Point 1| ... |Obsv Point M|| | | | || +------------+ +------------+| | | | Packets|+-------^-------------------------^-----+ +-----+ | --->---+--------+---------- ... ----------+ | In | | +--------------------------------------------------+ In the above figure the IPFIX components are shown in rectangular boxes. The interface shown by (*) is applicable only if the optional flow recording process is present. Otherwise the metering process(es) at the packet level interface directly with the exporting function. Note that in case of multiple Observation Domains, a unique ID per Observation Domain must be transmitted as a parameter to the exporting function. The exporting process includes IPFIX Sadasivan, et al. Expires November 30, 2004 [Page 12] Internet-Draft IPFIX Architecture June 2004 protocol and underlying transport layer. 7. IPFIX Functional and Logical blocks 7.1 Metering Process Every observation point in an IPFIX device, participating in flow measurements, MUST be associated with at least one metering process. Every packet coming into an observation point goes into each of the metering processes associated with the observation point. Broadly, each metering process extracts the packet headers that come into an observation point, does timestamping and classifies the packet into flow(s) based on the selection criteria. Collecting Processes use a Flow Record's Template ID to interpret that record's Flow Keys. To allow this, an IPFIX exporter MUST ensure that an IPFIX collector knows the Template ID for each incoming Flow Record. To interpret incoming Flow Records, an IPFIX collector MAY also need to know the Flow Type, i.e. the function F() that was used by the Metering Process for each Flow. In addition to Flow Type, an IPFIX collector MAY also use the following supplementary information to interpret the flow records further: o Observation Point o Selection Criteria for Packets 7.1.1 Observation Point A Flow Record can be better analyzed if the Observation Point from which it was measured is known. As such it is RECOMMENDED that exporters send this information to collectors as well as Flow Records. In cases where there is a single observation point or where the observation point information is not relevant, the metering process MAY choose not to add this to the flow records. 7.1.2 Selection Criteria for Packets A measurement device MAY define rules so that only certain packets within an incoming stream of packets are chosen for measurement at an observation point. This MAY be done by one of the two methods defined below or a combination of them. A combination of each of these methods can be adopted to select the packets, i.e. one can define a set of methods {F1, S1, F2, S2, S3} executed in a specified Sadasivan, et al. Expires November 30, 2004 [Page 13] Internet-Draft IPFIX Architecture June 2004 sequence at an observation point to select flows of a particular type. 7.1.2.1 Filter Functions, Fi A Filter Function selects only those incoming packets that satisfy a function on fields defined by the packet header fields, fields obtained while doing the packet processing, or properties of the packet itself. Example: Mask/Match of the fields that define a filter. A filter might be defined as {Protocol == TCP, Destination Port between 80 and 120}. Several such filters could be used in any sequence to select packets. Note that packets selected by a (sequence of) filter functions may be further classified by their Flow Type, i.e. the selected packets may belong to several Flows, all of which are exported. 7.1.2.2 Sampling Functions, Si A Sampling Function determines which packets within a stream of incoming packets is selected for measurement, i.e. packets that satisfy the sampling criteria for this Metering Process. Example: Sample every 100th packet that was received at an observation point and collect the flow information for a particular flow type. Choosing all the packets is a special case where the sampling rate is 1:1. The figure below shows the operations which MAY be applied as part of a typical metering process. packet header capturing | timestamping | v +----->+ | | | sampling Si (1:1 in case of no sampling) | | | filtering Fi (select all when no criteria) | | +------+ | v Flows Sadasivan, et al. Expires November 30, 2004 [Page 14] Internet-Draft IPFIX Architecture June 2004 Note that filtering and sampling functions may also be used in an exporting process to select flows to be exported. 7.2 Observation Domain The Observation Domain is a logical block that presents a single identity for a group of Observation Points within an IPFIX device. Each {Observation point, Metering Process} MUST belong to a single Observation Domain. An IPFIX device could have multiple Observation Domains each of which has a subset of the total set of Observation Points in it. Each Observation Domain MUST carry a unique ID within the context of an IPFIX device. One exporting process MAY serve multiple Observation Domains. In such a case the exporting process uses this unique ID to distinguish export packets among the different Observation Domains. The same concept is also used at the collecting process to identify packets from different Observation Domains within the same IPFIX device. 7.3 Flow Recording Process The Flow Recording Process is a functional block which manages all the flows generated from an Observation Domain. The typical functions of a Flow Recording Process MAY include: o Maintain database(s) of all the flows from an Observation Domain. This includes creating new records, updating existing ones, computing flow statistics, deriving further flow properties, adding non-flow-specific information (in some cases fields like AS numbers, router state, etc.) o Maintain aggregate statistics like flows generated, flows exported etc. Aggregate flows could be exported to one collector while the aggregate's component flows were being exported to a different collector. It is not mandatory that every IPFIX device use a Flow Recording Process. Instead the flows generated by the metering process can be sent directly to the exporting process. 7.4 Exporting Process The Exporting Process is the functional block that contains one or more instances of the IPFIX protocol. On one side it interfaces with Metering/Flow Recording Process to get flow records, while on the other side the Exporting Process talks to a collecting process on the collector(s). Sadasivan, et al. Expires November 30, 2004 [Page 15] Internet-Draft IPFIX Architecture June 2004 8. Overview of the IPFIX Protocol An IPFIX Device consists of a set of co-operating processes that implement the functional blocks described above, together with a supervisor process that provides management and oversight of the IPFIX system and its operation. Alternatively, an IPFIX Device can be viewed simply as a network entity which implements the IPFIX protocol. At the IPFIX device, the protocol functionality resides in the exporting process. The IPFIX protocol gets flows from a flow recording process or directly from the metering process, and carries them to the collector(s). At a high level, the IPFIX protocol at an IPFIX device does the following .. Maintain rules for: 1. Selection and sampling of the packets arriving at each observation point 2. Expiring Flows which have become inactive. 3. Picking and sending control information and flow records. 4. Encoding control and flow record information based on the IPFIX Information Model IPFIX-INFO [3]. 5. IPFIX device overload handling. Perform the following tasks: 1. Encode control information into templates. 2. Encode Flows observed at the observation points into flow records. 3. Packetize the selected templates and flow records into IPFIX export packets. 4. Use the underlying transport layer to send control and data packets to the collector. 5. Handle export errors and timeouts. 6. Handle IPFIX device overload. For details of the IPFIX protocol please refer to IPFIX-PROTO [4]. Sadasivan, et al. Expires November 30, 2004 [Page 16] Internet-Draft IPFIX Architecture June 2004 8.1 Encoding Flow Data Information The following rules provide guidelines to be followed while encoding the flow data information: o A flow data record MUST contain enough information so that the collecting process can identify the corresponding . o All fields MUST be encoded in network byte order. o The exporter MUST encode a given field based on the encoding standards prescribed by IPFIX-PROTO [4]. 8.2 Encoding Control Information The following rules provide guidelines to be followed while encoding the control information: o Per-flow control information SHOULD be encoded such that it can capture the structure and semantics of the corresponding flow data for each of the flows exported by the IPFIX device. o Configuration control information SHOULD be encoded such that it can capture the structure and semantics of the corresponding configuration data. The configuration data which is also control information SHOULD carry additional information on the Observation Domain within which the configuration takes effect. For example, sampling using the same sampling algorithm, say 1 in 100 packets, is configured on two observation points O1 and O2. The configuration in this case MAY be encoded as , where ID uniquely identifies this configuration. o There SHOULD be provisions to encode fixed length and variable length fields o All fields MUST be encoded in network byte order. o The exporter MUST encode a given field based on the encoding standards prescribed by IPFIX-PROTO [4]. 8.3 Exporting Control Information The Control Information is used by the collecting process to: Sadasivan, et al. Expires November 30, 2004 [Page 17] Internet-Draft IPFIX Architecture June 2004 o Decode and interpret flow records. o Understand the state of the exporting process. Sending control information from the exporting process in a timely and reliable manner is critical to the proper functioning of the IPFIX collecting process. The following approaches MAY be taken for the export of control information. 1. Send all the control information pertaining to flow records prior to sending the flow records themselves. This includes any incremental changes to the definition of the flow records. 2. Notify on a near real time basis the state of the IPFIX device to the collecting process. This includes all changes such as a configuration change that affects the flow behavior, changes to exporting process resources that alter export rates, etc., which the collector needs to be aware of. 3. Since it is vital that a collecting process maintains accurate knowledge of the exporter's state, the export of the control information SHOULD be done such that that it reaches the collector reliably. One way to achieve this would be to send the control information over a reliable transport. 8.4 Export Error Handling This section describes some of the errors that may be encountered by an IPFIX Export Process because of problems in exporting data, or because feedback is received by an IPFIX device from other entities in the export path towards the collector. Such errors include: o Unavailability of resources, e.g. packet buffers for IPFIX export packets. o Errors detected in the underlying transport layer. The protocol MAY choose to take one or more of the following actions: o Buffer the flow records until the error condition gets corrected. o Drop flow records for one or more flows based on some rules, i.e. decrease the flow's sampling rate. In such a case a record of what action is taken MUST be maintained, e.g. n flow records of a flow were dropped. Sadasivan, et al. Expires November 30, 2004 [Page 18] Internet-Draft IPFIX Architecture June 2004 8.5 Flow Expiration and Export A flow is considered to be inactive if no packets of this flow have been observed at the observation point for a given timeout interval. The flow MAY be expired and exported under the following conditions: 1. If the Metering Process can deduce the end of a Flow. The Flow SHOULD be exported when the end of the Flow is detected. For example: flow generated by TCP type of traffic where the FIN or RST bits indicate the end of the flow. 2. If the Flow has been inactive for a certain period of time. This inactivity timeout SHOULD be configurable at the Metering Process, with a minimum value of 0 seconds for immediate expiration. (However, note that a zero timeout would break a long-running flow into a sequence of single-packet flows.) 3. For long-running flows, the Exporting Process MAY export the flow records on regular basis. Some of the reasons for doing this could be: 1. Reporting for periodic accounting information. 2. Avoiding counter wrapping. 4. When a long-running flow is exported, the flow MAY still be maintained in the IPFIX device so that for the incoming packets that continue to come on the same flow, a new flow does not get created in the flow recording data base. 5. In some cases flows MAY be exported as they are generated. This can be useful when real time processing of flow records is required. 6. If the IPFIX device experiences resource constraints, a flow MAY be prematurely expired (e.g. lack of memory to store flow data) 7. In some cases flows the exporting process MAY choose not to export the generated flow as is. For example, this happens if a set of flows are aggregated into coarser flows. 8.6 Selection Criteria of flows for export There MAY be additional rules defined within the context observation domain so that only certain flows records are picked up for export. This MAY be done by either one or a combination of Si, Fi, as described in the section on "Selection Criteria for Packets". Sadasivan, et al. Expires November 30, 2004 [Page 19] Internet-Draft IPFIX Architecture June 2004 Example: Only the flow records which meet the following selection criteria are exported. 1. All flow records whose destination IP address matches {20.3.1.5}. 2. Every other (.i.e. sampling rate 1 in 2) flow record whose destination IP address matches {160.0.1.30}. 9. IPFIX Protocol Details When the IPFIX Working Group was chartered there were existing common practices in the area of flow export, for example NetFlow, CRANE, LFAP, RTFM, etc. IPFIX's charter required the Working Group to consider those existing practices, and select the one that was the closest fit to the IPFIX requirements IPFIX-REQS [1]. Additions or modifications would then be made to the selected protocol to fit it exactly into the IPFIX architecture. 9.1 The IPFIX basis protocol The working group went through an extensive evaluation of the various existing protocols that were available, weighing the level of compliance with the requirements, and finally selected NetFlow V9 with minor modification as the basis for the IPFIX protocol IPFIX-EVAL [2]. The following is a brief description of the selected IPFIX basis protocol; details of the IPFIX protocol proper are given in IPFIX-PROTO [4]. This protocol is template based. A template in terms of NetFlow V9 is a collection of fields with corresponding descriptions of their structures and their semantics which is in strict conformance with IPFIX architecture. This approach provides the following advantages: o Using the template mechanism, new fields can be added to IPFIX flow records without changing the structure of the export record format. o Templates that are sent to the collecting process carry structural information about the exported flow record fields. Therefore, if the collector does not understand the semantics of new fields it can ignore them, but still interpret the flow record. o Because the template mechanism is flexible, it allows the export of only the required fields from the flows to the collecting process. This helps to reduce the exported flow data volume and Sadasivan, et al. Expires November 30, 2004 [Page 20] Internet-Draft IPFIX Architecture June 2004 possibly provide memory savings at the Exporting Process and Collecting Process. Sending only the required information can also reduce network load. The figure below shows the functions performed in sequence by the various functional blocks in an IPFIX device. Packet(s) coming into Observation Point(s) | | v v +----------------+-------------------------+ +-----+-------+ | Metering Process on an | | | | Observation Point | | | | packet header capturing | | | | | | | Metering | | timestamping | | Process | | | | | on an | | +----->+ | | Observation | | | | | | Point | | | sampling Si (1:1 in case of no | | | | | | sampling) | | | | | classifying Fi (select all when | | | | | | no criteria) | | | | +------+ | | | | | | | | +--------+---------------------------------+ +-----+-------+ | | Flows (identified by observation domain) Flows +----... +----------+... | v | +-------------------------------------+-------------+ | | Flow Recording Process(*) | | | +----------------------+ +------------------+ | | | | Flow data base |<----|Provide non-flow | | | | | (includes flows | | information (e.g.| | | | | from all obsv | | router state) | | | | | points in an obsv | +------------------+ |... | | | domain) | | | | +----------------------+<----+------------------+| | | +-------------------------+ |Maintain aggregate| | | | |Aggregate flow records(*)| | statistics | | | | +-------------------------+ +------------------+ | | +---------------------------------+-----------------+ | | | Flow Database (identified by observation domain) | +------------------... | | | | Sadasivan, et al. Expires November 30, 2004 [Page 21] Internet-Draft IPFIX Architecture June 2004 v v +----------+------------------------------------+--------------------+ | | Exporting Process | | | | +------------------->| |+---------+------------------------------------------------------+ | || v IPFIX Protocol | | ||+------------------------------+ +----------------------------+| | |||Rules for | |Functions || | |||-Picking & sending templates | |-Packetize selected control || | |||-Picking & sending data recrds|->| & data information into || | |||-Timing out flows | | IPFIX export packet. ||->| |||-Encoding template & data | |-Handle export errors || | |||-Selecting flows for export(*)| |-Handle timeouts & overloads|| | ||+------------------------------+ +----------------------------+| | || | | |+----------------------------+-----------------------------------+ | | | | | +------------------------------------->| | IPFIX exported packet | | | | |+----------------------------+------------------------------------+ | || Anonymize export packet(*) | | |+----------------------------+------------------------------------+ | | | | |+----------------------------+------------------------------------+ | || Transport Protocol | | |+----------------------------+------------------------------------+ | | | | +-----------------------------+--------------------------------------+ | v IPFIX export packet to collector. (*) indicates that the block is optional. 9.2 The Collecting Process A Collecting Process is a subsystem that interacts with one or more IPFIX devices. The functions of the collecting process MUST include: o Identifying, accepting and decoding export packets from different pairs. o Running the IPFIX protocol. o Storing the control information and flow records received from IPFIX device. Sadasivan, et al. Expires November 30, 2004 [Page 22] Internet-Draft IPFIX Architecture June 2004 o Notifying the IPFIX device of the Collector's status and problems. At a high level, the IPFIX protocol at the collecting process: 1. Receives and stores the control information. 2. Decodes and stores the flow records using the control information. 3. May optionally monitor the status of the collecting process and execute a failover should any problem arise. 9.3 IPFIX Protocol on the Collecting Process The Collecting process is responsible for: 1. Receiving and decoding flow records from the IPFIX devices. 2. Indicating flow record losses to the exporting IPFIX device and/ or IPFIX users. 3. Optionally notifying status and overload conditions to the IPFIX device. Complete details of the IPFIX protocol are given in IPFIX-PROTO [4]. 9.4 Support for Applications Applications that use the information collected by IPFIX may be Billing or Intrusion Detection sub-systems, etc. These applications may be an integral part of the collecting process or they may be co-located with the collecting process. The way by which these applications interface with IPFIX system to get the desired information is out of scope for this document. 10. Export Models 10.1 Export with Reliable Control Connection As mentioned in the IPFIX-REQS [1] document, an IPFIX device MUST be able to transport its control information and data stream over a congestion-aware transport protocol. If the network in which the IPFIX device and collecting process are located does not guarantee reliability, at least the control information SHOULD be exported over a reliable transport. There could be network security concerns between IPFIX device and collecting process. To avoid re-inventing the wheel, and to reduce the complexity of the flow export protocol, Sadasivan, et al. Expires November 30, 2004 [Page 23] Internet-Draft IPFIX Architecture June 2004 one or a combination of the following methods MAY be adopted so as to achieve security: o IP Authentication Header MAY be used when the threat environment requires stronger integrity protection, but does not require confidentiality. o IP Encapsulating Security Payload (ESP) MAY be used to provide confidentiality and integrity. o If the transport protocol used is TCP, the TCP MD5 signature option MAY be used to protect against spoofed TCP segments. o If the transport protocol used is TCP, TLS MAY be used to provide integrity, authenticity and confidentiality. The data stream MAY be exported over a reliable or unreliable transport protocol. 10.2 Collector Failure Detection and Recovery The transport connection (in the case of a connection oriented protocol) is pre-configured between the IPFIX device and the collector. The IPFIX protocol does not provide any mechanism for configuring the Metering or Exporting processes. Once connected, an IPFIX Collector receives control information and uses that information to interpret flow records. The IPFIX device SHOULD set a keepalive (e.g. the keepalive timeout in the case of TCP, the HEARTBEAT interval in the case of SCTP, or an IPFIX protocol level keepalive if any) to a sufficiently low value so that it can quickly detect a collector failure. Collector failure refers to the crash or restart of the Collecting Process, or of the collector itself. A collector failure is detected at the IPFIX device by the break in control connection (depending on the transport protocol - the connection timeout mechanisms differ). On detecting a keepalive timeout, the IPFIX device SHOULD stop sending the flow export data to the collector and try to reestablish the transport connection. This is valid for a single collector scenario. If there are multiple collectors for the same IPFIX device, the IPFIX device opens control connections to each of the collectors. However, data gets sent only to one of the collectors which is chosen as the primary. There could be one or more collectors configured as secondary and a priority assigned to them. The primary collector crash is detected at the IPFIX device by the break in control connection (depending on Sadasivan, et al. Expires November 30, 2004 [Page 24] Internet-Draft IPFIX Architecture June 2004 the transport protocol - the connection timeout mechanisms differ). On detecting loss of connectivity, the IPFIX device opens a data stream with the secondary collector of the next highest priority. That collector now becomes the primary. The maximum export data loss would be the amount of data exported in the time between when the loss of connectivity to the collector happened, and the time at which this was detected by the IPFIX device. 10.3 Collector Redundancy Since the IPFIX protocol requires a congestion-aware transport, achieving redundancy using multicast is not an option. Multiple pairs could be set up, each to a different collector from the same IPFIX device. The control and data information would then be replicated on each of the control information and data streams. 11. IPFIX flow collection for Special Traffic An IPFIX device could be doing one or more of generating, receiving, altering special types of traffic which are listed below. Tunnel traffic: The IPFIX device could be the head, midpoint or endpoint of a tunnel. In such cases the IPFIX could be handling GRE, IPinIP or UTI traffic. VPN traffic: The IPFIX device could be a Provider Edge Device which receives traffic from customer sites belonging to different Virtual Private Networks. In the cases above, there should be clear guidelines as to: o How and when to classify the packets as flows in the IPFIX device. o If multiple encapsulations are used to define flows, how to convey the same fields (e.g. IP address) in different layers. o How to differentiate flows based on different private domains. For example, overlapping IP addresses in Layer-3 VPNs 12. IPFIX flow collection from Special Devices IPFIX could be implemented on devices which perform one or more of Sadasivan, et al. Expires November 30, 2004 [Page 25] Internet-Draft IPFIX Architecture June 2004 the following special services: o Explicitly drop packets. For example a device which provides firewall service drops packets based on some administrative policy. o Alter the values of fields used as IPFIX flow keys of interest. For example a device which provides NAT service can change source or(and) destination IP address. In the cases above, there should be clear guidelines as to: o How and when to classify the packets as flows in the IPFIX device. o What extra information be exported so that the collector can make a clear interpretation of the received flow records. 13. Security Considerations IP flow information can be used for various purposes, such as usage accounting, traffic profiling, traffic engineering, and intrusion detection. The security requirement may differ significantly for such applications. To be able to satisfy the security needs of various IPFIX users, an IPFIX system MUST provide different levels of security protection. 13.1 Data security IPFIX data comprises control information and data stream generated by the IPFIX device. The IPFIX data may exist in both the IPFIX device and the collector. In addition, the data is also transferred on the wire from the IPFIX device to the collector when it is exported. To provide security, the data SHOULD be protected from common network attacks. The protection of IPFIX data within the end system (IPFIX device and collector) is out of scope for this document. It is assumed that the end system operator will provide adequate security for the IPFIX data. The IPFIX architecture MUST allow different levels of protection to the IPFIX data on the wire. Wherever security functions are required it is recommended that users should leverage lower layers using either IPSEC or TLS, if these can successfully satisfy the security requirement of IPFIX data protection. Sadasivan, et al. Expires November 30, 2004 [Page 26] Internet-Draft IPFIX Architecture June 2004 To protect the data on the wire, three levels of granularity SHOULD be supported .. 13.1.1 No security Security may not be required when the transport between the IPFIX device and the collector is perceived as safe. This option allows the protocol to run most efficiently without extra overhead and an IPFIX system MUST support it. 13.1.2 Authentication-only Authentication-only protection provides IPFIX users with the assurance of data integrity and authenticity. The data exchanged between the IPFIX device and the collector is protected by an authentication signature. Any modification of the IPFIX data will be detected by the recipient, resulting in discarding of the received data. However, the authentication-only option doesn't offer data confidentiality. The IPFIX user SHOULD avoid use authentication-only when sensitive or confidential information is being exchanged. An IPFIX solution SHOULD support this option. The authentication-only option SHOULD provide replay attack protection. Some means to achieve this level of security are: o TCP with MD5 options. o IP Authentication Header 13.1.3 Encryption Data encryption provides the best protection for IPFIX data. The IPFIX data is encrypted at the sender and only the intended recipient can decrypt and have access to the data. This option MUST be used when the transport between the IPFIX device and the collector are unsafe and the IPFIX data needs to be protected. It is recommended that the underlying transport layer's security functions be used for this purpose. Some means to achieve this level of security are: o Encapsulating Security Payload. o Transport Layer Security Protocol The data encryption option adds overhead to the IPFIX data transfer. It may limit the rate that an exporter can report its flow to the collector due to the resource requirement for running encryption. Sadasivan, et al. Expires November 30, 2004 [Page 27] Internet-Draft IPFIX Architecture June 2004 13.2 IPFIX end point authentication It is important to make sure that the IPFIX device is talking to the "right" collector rather than to a masquerading collector. The same logic also holds true from the collector point of view, i.e. it may want to make sure it is collecting the flow information from the "right" IPFIX device. An IPFIX system SHOULD allow the end point authentication capability so that either one-way or mutual authentication can be performed between the IPFIX device and collector. The IPFIX architecture SHOULD use any existing transport protection protocols such as TLS or IPSEC to fulfill the authentication requirement. 14. IPFIX overload An IPFIX device could become overloaded under various conditions. This may be because of exhaustion of internal resources used for flow generation and/or export. Such overloading may cause loss of data from the exporting process, either from lack of export bandwidth (possibly caused by an unusually high number of observed flows) or from network congestion in the path from exporter to collector. IPFIX collectors SHOULD be able to detect the loss of exported flow data, and SHOULD at least record the number of lost flow data records. 14.1 Denial of service (DoS) attack prevention Since one of the potential usages for IPFIX is for intrusion detection, it is important for the IPFIX architecture to support some kind of DoS resistance. 14.1.1 Network under attack The Network itself may be under attack, resulting in an overwhelming number of IPFIX messages. An IPFIX system SHOULD try to capture as much information as possible. However, when a large number of IPFIX messages are generated in a short period of time, the IPFIX system may become overloaded. 14.1.2 Generic DoS attack on the IPFIX system The IPFIX system may subject to generic DoS attacks, just as any system on any open network. These types of attacks are not IPFIX specific. Preventing and responding to such types of attacks are out of the scope of this document. Sadasivan, et al. Expires November 30, 2004 [Page 28] Internet-Draft IPFIX Architecture June 2004 14.1.3 IPFIX specific DoS attack There are some specific attacks on the IPFIX portion of the IPFIX device or Collector. o The attacker could pound the Collector with spoofed IPFIX export packets. One way to solve this problem is to periodically synchronize the sequence numbers of the flow records between the exporting and collecting processes. o The attacker could provide false reports to the IPFIX device by sending spoofed control packets. The problems mentioned above can be solved to a large extent if the control packets are encrypted both ways. 15. IANA Considerations The IPFIX protocol will need an IP port number assigned by IANA, and the various fields described in the IPFIX Information Model will need a set of identifying numbers. Full IANA considerations are given in the IPFIX Information Model IPFIX-INFO [3] and IPFIX Protocol IPFIX-PROTO [4] documents. 16. Acknowledgements The document editors wish to thank all the people contributing to the discussion of this document on the mailing list, and the design teams for many valuable comments. In particular, the following made significant contributions: Tanja Zseby Paul Calato Dave Plonka Jeffrey Meyer Benoit Claise Ganesh Sadasivan K.C.Norseth Vamsi Valluri Cliff Wang Ram Gopal Jc Martin Carter Bullard Juergen Quittek Reinaldo Penno Nevil Brownlee Sadasivan, et al. Expires November 30, 2004 [Page 29] Internet-Draft IPFIX Architecture June 2004 Simon Leinen Kevin Zhang 17 References [1] Quittek, J., Zseby, T. and B. Claise, "Requirements for IP Flow Information Export", (work in progress), Internet Draft, draft-ietf-ipfix-reqs-16.txt, June 2004. [2] Leinen, S., "Evaluation of Candidate Protocols for IP Flow Information Export", (work in progress), Internet Draft, draft-leinen-ipfix-eval-contrib-03.txt, May 2004. [3] Quittek, J., Meyer, J. and P. Calato, "IPFIX: Information Model", (work in progress), Internet Draft, draft-ietf-ipfix-info-03.txt, February 2004. [4] Fulmer, M., Claise, B., Calato, P. and R. Penno, "IPFIX: Protocol", (work in progress), Internet Draft, draft-ietf-ipfix-protocol-03.txt, January 2004. Authors' Addresses Ganesh Sadasivan Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134 USA Phone: +1 408 527-0251 EMail: gsadasiv@cisco.com Nevil Brownlee CAIDA | The University of Auckland Private Bag 92019 Auckland New Zealand Phone: +64 9 373 7599 x8941 EMail: n.brownlee@auckland.ac.nz Sadasivan, et al. Expires November 30, 2004 [Page 30] Internet-Draft IPFIX Architecture June 2004 Benoit Claise Cisco Systems, Inc. De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 EMail: bclaise@cisco.com Juergen Quittek NEC Europe Ltd. Adenauerplatz 6 69225 Heidelberg Germany Phone: +49 6221 90511-15 EMail: quittek@ccrle.nec.de URI: Sadasivan, et al. Expires November 30, 2004 [Page 31] Internet-Draft IPFIX Architecture June 2004 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. The IETF has been notified of intellectual property rights claimed in regard to some or all of the specification contained in this document. For more information consult the online list of claimed rights. Full Copyright Statement Copyright (C) The Internet Society (2004). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. Sadasivan, et al. Expires November 30, 2004 [Page 32] Internet-Draft IPFIX Architecture June 2004 This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Sadasivan, et al. Expires November 30, 2004 [Page 33]