IP Measurement Protocol March 2003 Internet Draft J. C. R. Bennett Document: draft-bennett-ippm-ipmp-01.txt Harvard Expires: September 2003 March 2003 IP Measurement Protocol (IPMP) Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract The practice and need for active network measurement is well established, however current tools are not well suited to this task, mostly because the protocols which they employ have not been designed for measurement of the modern Internet. The IP Measurement Protocol (IPMP) is based on packet-probes, and is designed to allow routers to participate in measurements by the insertion of path information as the probe passes between a pair of hosts. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 Bennett Expires - September 2003 [Page 1] IP Measurement Protocol March 2003 v Table of Contents 1. Introduction...................................................2 1.1 IPMP Redirection..............................................4 2. Packet Formats.................................................4 2.1 IPMP Echo Request and Echo Reply..............................4 2.1.1 Echo Request Header.........................................4 2.1.2 IPMP Options................................................5 2.1.3 Header Fields...............................................6 2.1.4 Path Record formats.........................................7 2.1.5 Optional Data Section Formats..............................14 2.1.6 Defined Data Section Types.................................14 2.2 IPMP Information Request and Information Reply...............17 2.2.1 Information Request Header.................................17 2.2.2 Information Reply Header...................................18 3. IP Protocol Header Values.....................................22 4. Processing of IPMP packets....................................22 4.1.1 Measurement host processing................................22 4.2 Echoing System Processing....................................23 4.3 Forwarding System Processing.................................24 4.3.1 Path Record................................................24 5. Discussion....................................................24 5.1 Checksums....................................................24 5.1.1 Checksum update at a forwarding router.....................25 5.2 Timestamps...................................................25 5.2.1 NTP Timestamp format.......................................25 5.2.2 IPMP Timestamp Format......................................26 5.2.3 Inferred Real Time.........................................26 5.3 Minimum Implementations......................................27 5.3.1 Echoing System.............................................27 5.3.2 Forwarding System..........................................27 6. Security Considerations.......................................27 6.1 Denial of Service Attacks....................................28 7. References....................................................28 Acknowledgments..................................................29 Author's Addresses...............................................29 1. Introduction The practice and need for active network measurement is well established, however current tools are not well suited to this task, mostly because the protocols which they employ have not been designed for measurement of the modern Internet. ICMP, for example, is widely used for measurement despite its well known limitations for this task. These limitations include it being Bennett Expires - September 2003 [Page 2] IP Measurement Protocol March 2003 treated different to other IP protocols at routers and hosts. ICMP has also received bad press from denial of service attacks and because of the number of sites generating monitoring traffic. As a consequence some ISPs disable ICMP even though this potentially causes poor performance and does not comply with RFC1009 [1]. The protocol operates as an echo protocol allowing packet loss, path length, route, RTT and in some cases, one-way delay measurements to be taken. Packets are generated by a measurement host and returned by an echoing router or host, known as an echoing system in this memo. The translation of router time stamps to real time, time stamps is supported through a separate information request and reply exchange between the measurement system and systems that insert time stamps into the echo request or reply. Current packet probing techniques are not suited to measuring packet delay at the router level. Routers often make bad measurement targets because they are optimized for the relatively simple task of forwarding packets. Routers may process tasks that are resource intensive and therefore an opportunity for a denial of service attack at low priority or not at all. Some measurement techniques construct measurement traffic that can be difficult to efficiently detect and respond to amongst other network traffic. This type of measurement traffic precludes measuring to a router and makes the task of identifying where delay occurs in the network difficult. IPMP addresses these issues by providing a measurement protocol that is tightly constrained, efficient and easy to implement. The protocol has been designed so measurement packets can be processed with about the same level of computation as IP packet forwarding. The protocol is intended to allow for easy implementation in hardware/firmware so as to provide for highly accurate measurements. It should be noted that only the TTL and Timestamp fields in the Path Records are dynamic, all the other fields are likely to be the same for all the Path Records inserted at a particular stamping location. As a result it will be possible to precompute and/or cache the static portions of the Path Record as well as partial checksum update computations. The IPMP protocol has a number of options to allow it to measure a number of different properties of the links and devices on a path between two endpoints. It is intended that it should be practical to process IPMP packets in the same forwarding path as normal(non-IPMP) packets without any (significant) performance impact. To achieve this it is anticipated that a device will pre-compute all but one or two Bennett Expires - September 2003 [Page 3] IP Measurement Protocol March 2003 of the components of the path record and insert some combination of these components based on the options field of the packet. The option fields of the IPMP request packet are defined with the objective of allowing a forwarding device to determine what if any path records should be inserted with the minimum amount of logic complexity. S 1.1 IPMP Redirection To allow for measurement of paths that do not originate at the measurement host IPMP there are hooks defined to allow the addition of a redirection mode of operation. 2. Packet Formats 2.1 IPMP Echo Request and Echo Reply 2.1.1 Echo Request Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Faux Source Port | Faux Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPMP Options | Start TTL | Faux P-Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version |Info Req Flags | Path Pointer | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet ID | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (optional) Data Section | : .... : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (optional) Path Record(s) | : .... : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Padding (if required) | : .... : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version = 0 Bennett Expires - September 2003 [Page 4] IP Measurement Protocol March 2003 2.1.2 IPMP Options 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | P |R|F|T|D|A| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ P = Packet Type : 4 bits Value Packet Type 0000 Echo Request 0001 Echo Reply 0010 Information Request 0011 Information Reply 0100- Reserved for Redirection Packet Types 0111 1000- Reserved for Future use 1111 R = Record Path Record : 1 bit F = Fastpath ONLY Records Requested : 1 bit T = Timestamps Requested : 1 bit D = Data Section Present : 1 bit A = turnAround Options : 1 bit Record Path Record If this field is 1 then a Path Record SHOULD be inserted by any device that forwards this packet. If the field is 0 then a device forwarding the packet MUST NOT insert a Path Record into the packet. The echoing host MAY insert a Path Record on reception or transmission of the packet even if doing so would otherwise be prohibited by this option. Fastpath ONLY Records Requested If this field is 1 then a Path Record SHOULD NOT be inserted unless it can be done in the same processing path that would be taken by a regular data packet. The objective of this option is to allow measurement to be restricted to those points where inserting the path Bennett Expires - September 2003 [Page 5] IP Measurement Protocol March 2003 record will not affect the forwarding of the IPMP packet with respect to non-IPMP packets. This may be of use if the IPMP packet is being inserted periodically into a stream of packets so that the position of the IPMP packet in the data stream is not disturbed. Timestamp Requested If this field is 1 then any Path Records inserted into the packet SHOULD include a timestamp in the Path Record. If this field is 0 then any Path Records inserted into the packet SHOULD NOT contain a timestamp. This field allows the IPMP packet to be used to perform a function similar to traceroute, without the need to collect timing data. Data Section Present If this field is 1 then it indicates that the optional data section is present and contains at least one data element. All data elements are specified as TLVs. The data section if present MUST end on a word alignment and MUST be terminated with the æEndÆ TLV. Turnaround Options If this field is 0 then there are no elements in the Data section which require processing by the Echo Host. If this field is 1 and and the Data Section Present field is 1 then there MAY be elements in the Data section which if present MUST be processed by the Echo Host. 2.1.3 Header Fields Faux Source Port, Faux Destination Port In order for IPMP to be used to monitor the characteristics of the path being taken by the data packets of an application it will be necessary for the IPMP packets to be filtered and queued in a manner identical to that of user data packets. In order to allow such filtering to be performed, In the location where the source and destination port fields of a TCP/UDP packet are located an IPMP packet contains two "faux" port fields which allow it to be matched any per flow filters that might be in use. The real source and destination port fields used by the IPMP protocol are found further down in the IPMP header. Faux P-Type (Packet Type) Bennett Expires - September 2003 [Page 6] IP Measurement Protocol March 2003 For those devices with packet filters that include the packet type field of the IP header in determining how to process a packet, and are IPMP aware, this field SHOULD be used by the filter in place of the actual packet type field which will always contain the IPMP packet type. Checksum The checksum is the 16-bit one's complement of the one's complement sum of the IPMP message starting with the Faux Source Port. For computing the checksum, the checksum is initialized to zero. Path Pointer The position, in 32 bit words from the beginning of the IP packet, of the next available path record location. Info Request Flags These flags allow for hosts to request additional information (if available, and at the option of the stamping device) to be added to the Path Records. Data The data field is used to contain additional information about the IPMP packet either for use by the echoing host and/or by the measurement host. 2.1.4 Path Record formats Without Timestamp 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|1|1| Flags | Reserved | TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Proxy Address | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | Extensions | Bennett Expires - September 2003 [Page 7] IP Measurement Protocol March 2003 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - With Timestamp 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flags | Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Timestamp | TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Proxy Address | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | Extensions | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Extensions Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+ | 0 (Pad) | +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ | 255 (End) | +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - | Type | Value +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - | Type | Length | Value +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - Base Extension Types Type# Length Field Name 0 No (L=1) Pad 1 No (L=13)IPv6 Proxy Address (partial) 255 No (L=1) End Bennett Expires - September 2003 [Page 8] IP Measurement Protocol March 2003 Flags 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TT |F|AT | L |Accuracy |T|E|R| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Flag Fields TT = Timestamp Type 0 = Undefined 1 = Local source 2 = External (Stratum ?) source 3 = Reserved 4 = Reserved 5 = ~UTC 6 = UTC 7 = No Timestamp F = Processed in "Fastpath" 0 No 1 Yes AT = Address Type 0 = IP address of interface 1 = IP address of interface (non-responsive) 2 = IP address of interface + IPMP proxy Address 3 = Non-IP address L = Stamping Location 0 = Packet RX 1 = Packet TX 2 = Internal processing 3 = Unspecified A = Accuracy 0-31, most accurate bit position in timestamp, compared to 'real time clock' on the device, this will be the worst of the number of bits of resolution of the device's clock and the maximum time difference between the real time clock at the time the packet is stamped and the value of the timestamp placed in the packet. T = Tunnel [De/En]capsulation Point 0 = No 1 = Yes Bennett Expires - September 2003 [Page 9] IP Measurement Protocol March 2003 E = Extensions 0 = No 1 = Yes R = Reserved TTL The value of the TTL field in the Path Record is set to the value of the TTL in the packet when transmitted by the device which inserted the Path Record. By comparing the TTL field of consecutive Path Records it can be determined if the time delta between two consecutive Path Records measures the time between two directly connected devices or if there are intermediate (non-IPMP aware) devices between the those that inserted the Path Records. Stamping Location The Stamping Location field indications where in the device the Path Record was inserted into the Echo Request packet. If the field value is 0 then the Path Record was inserted at the interface where the packet was received. If the field value is 1 then the Path Record was inserted at the interface where the packet was transmitted. If the field value is 2 then the Path Record was inserted at an internal location in the device. If the field value is 3 then the Path Record was inserted at an unknown/unspecified location in the device. Usage Example: +--------+ +---------+ +----------+ +----------+ | | | | | | | | | A |---|rx B tx|---|rx C tx|---| D | | | | | | | | | +--------+ +---------+ +----------+ +----------+ If host A sends an Echo Request to D with a TTL of 255 and both B and C insert Path Records, with TTLs of 254 and 253, then when the Echo Bennett Expires - September 2003 [Page 10] IP Measurement Protocol March 2003 Request returns to A it can determine that there are no devices between B and C. But without knowing the stamping location it does not know what was measured. Suppose that B stamped at the 'rx' location and C stamped at the 'rx' location then the difference in the timestamps would be due to the processing at B and the propagation delay of the B->C link, and any variation in the difference seen over repeated measurements would be due entirely to variations in delay within B. However if C stamped at 'tx' instead of 'rx' than any variation between measurement may be caused by either B or C or both. By marking the stamping location it is possible to determine what was or was not measured by the difference in two consecutive timestamps. Address If the value of the Address Type field is 0, the Address field MUST contain the address of the interface at which the packet was processed by the system that inserted the Path Record into the Echo Request or Echo Reply packet, as well as the address to which any Information Request packets should be sent to. If the value of the Address Type field is 1, the Address field MUST contain the address of the interface at which the packet was processed by the system that inserted the Path Record into the Echo Request or Echo Reply packet. Unlike the case above, if the field value is 1 the system will NOT respond to Information Request packets at this address. If the value of the Address Type field is 2, the Address field MUST contain the address of the interface at which the packet was processed by the system that inserted the Path Record into the Echo Request or Echo Reply packet. As in the case of Type 1 the system will not respond to Information Request packets at this address. However in this case the optional Proxy Address field is present and MUST contain the address of an IPMP proxy that will respond in place of the stamping interface. The proxy address option is designed to allow a system operator to support the IPMP protocol without requiring the processing of Information Request packets by the router interfaces that inserted the Path Records. The proxy address option also allows for information about stamping interfaces located behind firewalls/NATs/ALGs etc to be made Bennett Expires - September 2003 [Page 11] IP Measurement Protocol March 2003 available to hosts which could not otherwise reach them with an Information Request packet. If the value of the Address Type field is 3, the Address field contains a non-IP address value, more accurately it contains a value which is not the IP address of the interface which inserted the Path Record, since it may be any 32 bit value it MAY by chance be an IP address of some other system. The value of the Address field SHOULD be static while a system is up. The value SHOULD be static across system restarts. The value is not guaranteed to be globally unique, but SHOULD be unique at least amongst systems belonging to the same AS. This value is designed to allow a system operator to support the IPMP protocol without being required to expose the addresses of their systems to the protocol while still providing a unique identifier to be associated with the interface which inserted the Path Record. If the value in the Address field is not the IP address of the receiving interface or of an IPMP 'proxy' this value MUST be used. This value SHOULD also be used if the IP address of the interface is a 'private' address, e.g. 10.0.0.1 . Timestamp The time at which the Path Record is inserted into the packet is recorded as a reduced size form (see section 4.3) of an RFC1305 NTP format [2] timestamp. The relationship between this timestamp and real time, if there is one, may be derived using information from an IPMP Information Reply (see section 4.3), or it may be inferred from a combination of the Fastpath, Timestamp Type and Accuracy fields. This reduced size timestamp is designed to allow the entire Path Record to fit into 3 words, or 4 words for Path Records with a Proxy Address. Timestamp Type The Timestamp Type field gives an indication of the clock source that the timestamp was derived from. If the Timestamp Type is 0 then the clock source is of undefined quality and/or may be subject to arbitrary amounts of drift. An example case would be a PC router using the OS clock for the timestamp, in this case the software performing the stamping may have no knowledge of the quality of the clock source and the value of the OS clock may be subject to large adjustments. Bennett Expires - September 2003 [Page 12] IP Measurement Protocol March 2003 If the Timestamp Type is 1 then the clock source is a local oscillator which may drift but which is not subject to adjustments. Different stamping points on a device with a Type 1 source are neither guaranteed to have the same rate of drift nor to have identical values. If the Timestamp Type is 2 then the clock source is an external clock signal of (NOTE what level accuracy is required here?) quality and is not subject to adjustments. Different stamping points on a device with a Type 2 source are not guaranteed to have identical values, but they are guaranteed to have the same drift rate. If the Timestamp Type is 5 then the clock source is an external clock signal of (Stratum 1/GPS?) quality (i.e. it doesn't drift because it IS the reference time) which is not subject to adjustments. Further the second's value in at least the 3 highest order bits of the timestamp, corresponding to bits 16-18 of an NTP timestamp, MUST have a value which differs by no more than +/- 2 from the value of bits 16-18 of an NTP timestamp from a station at (+0 GMT). This allows the 'missing' upper bits to be determined if the receiver knows the correct value of real time (+0 GMT). Different stamping points on a device with a Type 5 source are not guaranteed to have identical values, but they are guaranteed to have the same (~0) drift rate. If the Timestamp Type is 6 then the device inserting the Path Record asserts that the value in the Timestamp IS the UTC time value for +0 GMT, within the accuracy as defined by the Accuracy flag. Different stamping points on a device with a Type 6 source are guaranteed to have the same time value (+/- Accuracy) and the same (~0) drift rate. Fastpath If the Fastpath field value is 1, aside from the alterations to contents of the IPMP packet the IPMP packet was processed and routed identically to a normal data packet containing the same IP header fields (addresses,ports,DSCP). This field allows the measurement host to determine how close the reported times reflect the delays seen by regular data packets. Tunnel If the Tunnel Encapsulation field is 1 then the device inserting the Path Record is an de/en-capsulation point for the packet. This may be any form of encapsulation that will prevent the insertion of Path Records and results in transmission over equipment with variable delays. For example IPSEC tunnels, MPLS encapsulation, IP-over-ATM, Bennett Expires - September 2003 [Page 13] IP Measurement Protocol March 2003 etc. The flag deliberately does not indicate if the stamping device is the source or end point of a tunnel since unless all devices along the path are IPMP aware it might be possible to have two records the first indicating an encapsulation followed by one indicating a decapsulation making it appear that there was a tunnel between the two points, when in fact there was more than one tunnel between the two points. 2.1.5 Optional Data Section Formats 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+ | 0 (Pad) | +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ | 255 (End) | +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - | Type | Value +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - | Type | Length | Value +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - 2.1.6 Defined Data Section Types Type# Length Field Name 0 Fixed (1)Pad 1 Variable Error Return 240- Reserved for Redirect Options 251 252 Variable Echo Host Options (future use) 253 Fixed (3) Echo Host Std Options 254 Fixed (1)End Echo Host Options 255 Fixed (1)End Bennett Expires - September 2003 [Page 14] IP Measurement Protocol March 2003 Echo Host Standard Options 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 11111101 | Flags | TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Echo Host Standard Options Flags 0 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ | P |F|T|R| Res | +-+-+-+-+-+-+-+-+ P = Path Records Requested F = Swap Faux Ports T = TTL Reset R = Record Route Set Path Records Requested Specifies the number (0-2) of Path Records that the echo host SHOULD insert into the packet. If 0, the host SHOULD NOT insert any Path Records. If 1, the host SHOULD insert one (1) Path Record. The record SHOULD be inserted when the packet is received if the Record Route bit is set when the packet is received. If the Record Route bit is not set when the packet is received the record SHOULD be inserted when the packet is transmitted. If 2, the host SHOULD insert a first record when the packet is received and a second record when the packet is transmitted. If the host implementation can not insert a record on either reception or transmission then only one (1) record SHOULD be inserted. The value 3 is invalid and SHOULD be treated as 0. Swap Faux Ports Bennett Expires - September 2003 [Page 15] IP Measurement Protocol March 2003 If this field is 1 then the echoing host MUST swap the values in the Faux Src/Dst Port fields when returning the packet. This allows both forward and backward paths of a flow to be measured by one packet. The swapping of the ports is needed to insure that any packet filters act on the packet as if it were part of the flow being measured. Record Route Set The echoing host MUST set the value of the Record Path Record field to the value of this field before echoing the packet. This allows for path records to be recorded in only one direction by turning on or off the stamping process when the packet reaches the echoing host. TTL Reset If this field is 1 then the echoing host MUST set the TTL field of the IP header in the echoed packet to be equal to the TTL field of the Echo Host Options data element. End Echo Host Options This option tells the echo host that there are no more TLVs in the data section for it to process. This allows the echo host to avoid having to process the entire data section once all of itÆs options have been reached. Error Return Option 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 00000001 | Length | Error Type | Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- - - - - - - - | Value - - - - - - - - - - - - - - - - Error Types Value Error 1 Invalid Field/Option 2 Missing Data Element 3 Security Option Required 4 Security Option Rejected Bennett Expires - September 2003 [Page 16] IP Measurement Protocol March 2003 2.2 IPMP Information Request and Information Reply 2.2.1 Information Request Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 0000000000000000 | 0000000000000000 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPMP Options | 0000000000000000 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 0000000000000000 | 0000000000000000 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Info Flags | 0000000000000000 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet ID | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Forwarding IP Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Proxy Address | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | Extensions | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | 0000000000000000 | Timestamp | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | Timestamp | Last | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Info Flags 0 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ |P|T|I|Reserved | +-+-+-+-+-+-+-+-+ Bennett Expires - September 2003 [Page 17] IP Measurement Protocol March 2003 P = Proxy Address Present If this flag is 1 then the information request packet contains a Proxy Address. T = Timestamps Present If this flag is 1 than the information request packet contains one or more Timestamps. I = Information Request Extensions Present The Extensions section exists and contains Information Request TLVs requesting additional information about the forwarding address specified. 2.2.2 Information Reply Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 0000000000000000 | 0000000000000000 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPMP Options | Precision Hi | Precision Lo | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Info Flags | Performance Data Pointer | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet ID | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Forwarding IP Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Proxy Address | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | Accuracy | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPMP Processing Overhead | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (optional) Real Time Reference Points | | | | | | | Bennett Expires - September 2003 [Page 18] IP Measurement Protocol March 2003 : .... : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (optional) Performance Data | : .... : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Real Time Reference Point 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Real Time | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reported Time | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version = 0. Flags The Flags field indicates the presence of optional elements in the information request. The information reply MUST contain the same value in the Flags field as the request packet contained. Last If there is a timestamp included in the information request the Last field indicates which is the last timestamp. If the Last field is 0 then another timestamp follows, if the Last field is any non-zero value then there are no more timestamps. Checksum The checksum is the 16-bit one's complement of the one's complement sum of the IPMP message starting with the version number. For computing the checksum, the checksum and returned checksum fields are zero. Bennett Expires - September 2003 [Page 19] IP Measurement Protocol March 2003 Precision Hi The number of bits in the whole seconds part of timestamps that are valid. A Precision Hi of N, means that the timestamps wrap around every 2^N seconds. Precision Lo The number of bits in the fractional part of timestamps that are valid. Length The total length, in bytes, of the IPMP packet. The length should not normally exceed 556 bytes. That is the Real Time Reference Points and Performance Data fields should not exceed 524 bytes. Longer packets risk being discarded if they need to be forwarded of a path with a MTU less that 576. If no performance data is included 32 Real Time Reference Points may be included in the packet. The IPMP packet MAY be the MTU size supported by the path, so equipment MUST be prepared to process IPMP packets as large as their supported MTU size. Performance Data Pointer The position, in bytes from the beginning of the IPMP packet, of the performance data field if it exists. If there is no performance data this field is 0. Forwarding IP Address The address of the interface to which the Information Request was sent. Accuracy The maximum difference between actual real time and the inferred real time (see section 4.3.2) of any timestamp generated by this interface. If there is no relationship between real time and the timestamps recorded by the interface or the extent of the difference from real time is unknown accuracy is set to 0. Bennett Expires - September 2003 [Page 20] IP Measurement Protocol March 2003 IPMP Processing Overhead The maximum difference between the time taken to process and forward an IPMP packet and the time taken to forward an IP packet with the same characteristics. If the overhead is unknown this is reported as MAX_TIME, i.e. all bits to 1. Real Time Reference Point A real time reference point gives the relationship between real time and the timestamp that would have been placed in a Path Record by the interface at that time. If there is no relationship between real time and timestamps no reference points are included in the Information Reply. If there were any timestamps present in the information request packet then the reply packet SHOULD contain a real time reference point corresponding to each timestamp in the request packet. If the host does not believe that a valid reference point can be returned, for example if the host recently restarted and believes the timestamp was taken before the restart, it MAY return a reference point with a real time of 0, for the reported timestamp. Performance Data The Performance Data field allows arbitrary information from the MIB of the system or the interface to optionally be included in the Information Reply. It is formatted as a VarBindList from the SNMPv2- PDU defined in [6]. In this context ObjectSyntax is the only valid choice within VarBind. I.E. IPMP-PERFORMANCE-DATA DEFINITIONS ::= BEGIN IMPORTS ObjectName, ObjectSyntax, FROM SNMPv2-SMI; -- IPMP simplified list element IPMPVarBind ::= SEQUENCE { name ObjectName, value Bennett Expires - September 2003 [Page 21] IP Measurement Protocol March 2003 ObjectSyntax } -- variable-binding list VarBindList ::= SEQUENCE (SIZE (0..max-bindings)) OF IPMPVarBind END 3. IP Protocol Header Values Version = 4 IHL = 5 Identification = unspecified Flags = DF Fragment offset = 0 IP Protocol type = TO BE ASSIGNED. IP options are forbidden. 4. Processing of IPMP packets 4.1.1 Measurement host processing A measurement host constructs an IPMP request, encapsulates it in an IP datagram and the appropriate link layer protocol and sends it into the network. Performance information is gleaned from the presence or absence of a reply, the delay between the request and the reply the value of TTL and the path record(s) if present and possibly the presence of errors. The measurement host, when constructing the echo request, MUST set all words from the end of the data field to the end of the echo request (the space allocated for path records) to zero. When an IPMP echo reply arrives the checksum is recomputed. Unless further protection is provided in the data field an IPMP echo reply Bennett Expires - September 2003 [Page 22] IP Measurement Protocol March 2003 with an incorrect checksum SHOULD be discarded because of the risk of data corruption causing incorrect matching with the echo request, or the reporting of invalid measurement data. 4.2 Echoing System Processing The IPMP Echo Request and Echo Reply packet formats are designed to make processing at the stamping points simple and efficient, at the echoing point however there are a number of more complex functions that an echoing system may have to perform. On receipt of the IPMP Echo Request (IPMP Option Packet Type = 0 or 4) the echoing system constructs the echo reply from the echo request by: 1. Exchanging the IP source and destination addresses 2. Check the IPMP checksum, if checksum invalid, either add Error Return data section with value 1 (Bad Checksum), may require moving any path records already present or drop the packet. 3. Optionally inserting a path record (as described in section 4.3.1). 4. If the IPMP option Toggle Record Path is set to 1, then toggle the value of the Record Path Record field of the IPMP options. 5. If the IPMP option Swap Faux Ports is set to 1, then swap the values of the Fax Src/Dst Port fields. 6. If the Packet Type field is 0, set it to 1, if 4, set it to 5. 7. Set TTL according to Reverse Path TTL option. 8. Calculate the IPMP checksum. 9. Schedule the packet for forwarding taking account of the Faux P- type field if appropriate. Processing IPMP Information Request packets requires more resources than an Echo Request. Direct measurements are not made from Information Request packets. Consequently an implementer may choose to processes Information Request packets off the interface card and/or at low priority. Bennett Expires - September 2003 [Page 23] IP Measurement Protocol March 2003 4.3 Forwarding System Processing A forwarding router does not need to be IPMP aware. In the simplest case IPMP is forwarded like any other IP protocol. If the router forwards schedules packets based (perhaps in part) on the value of the IP protocol field, from the IP header, then the forwarding router SHOULD use the Faux P-type field of the IPMP header for scheduling instead of the IP protocol field. A forwarding router may include a path record as described below. 4.3.1 Path Record Inclusion of path records is optional. A path record MAY be inserted by forwarding routers (on the forward or reverse path) and the echoing system. A system which adds a path record MUST increase the Path Pointer by the size of the inserted path record, which MUST be a multiple of 4 bytes in length and MUST update the Checksum field. The length of the packet is not changed. Before inserting the path record the path pointer plus the size of the path record MUST be compared with the packet length to ensure that sufficient space is available for the new path record. A system that adds a path record MAY include a timestamp in the path record using the IPMP timestamp format described in section 5.2.2. If it does not include a timestamp the timestamp field in the path record is left unaltered. 5. Discussion 5.1 Checksums The IPMP checksum is calculated by the measurement host when it creates the echo request packet. It is updated (as described in section 5.1.1) by forwarding routers that insert a route record into the IPMP packet. The checksum MUST be checked by the echoing host, and by the redirecting host (if any). Packets with bad IPMP checksum SHOULD be discarded, but hosts MAY choose to return a Bad Checksum Error Return instead. Bennett Expires - September 2003 [Page 24] IP Measurement Protocol March 2003 5.1.1 Checksum update at a forwarding router. A forwarding router that does not include a path record does not check or modify the IPMP checksum. (Normal IP forwarding occurs including decrementing TTL and updating the IP header checksum.) A forwarding router that includes a path pointer must update the IPMP checksum. This MAY be done in two ways: 1. Absolute. Check that the checksum matches for the received packet. If it does not set the checksum in the forwarded packet to 0. If the checksum in the received packet does match add the Route Record to the packet and recalculate the checksum. 2. Relative. Form the checksum of the path record (this will be a constant for a particular interface if timestamps are not used) and the previous checksum. Option #1 or #2 is selected on the basis of efficiency. If possible option #2 SHOULD be used as option #1 allows for errors to be introduced and then covered up. 5.2 Timestamps The timestamp field is coded following the conventions described in RFC1305 NTP [4] but using reduced number of bits. 5.2.1 NTP Timestamp format Summarizing from RFC1305: In conformance with standard Internet practice, timestamps are specified as integer or fixed-point quantities, with bits numbered in big-endian fashion from 0 starting at the left, or high-order, position. All quantities are unsigned and may occupy the full field width with an implied 0 preceding bit 0. Timestamps are represented as a 64-bit unsigned fixed-point number, in seconds relative to 0h on 1 January 1900. The integer part is in the first 32 bits and the fraction part in the last 32 bits. In the fraction part, the non-significant low order can be set to 0. Bennett Expires - September 2003 [Page 25] IP Measurement Protocol March 2003 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seconds | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seconds Fraction (0-padded) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ This format allows convenient multiple-precision arithmetic and conversion to UDP/TIME representation (seconds), but does complicate the conversion to ICMP Timestamp message representation, which is in milliseconds. The most future time that can be represented is 4,294,967 (some time in the year 2036) with a precision of about 200 picoseconds, which should be adequate for even the most demanding measurements. RFC 2030 [5] contains a proposal for extending timestamps beyond the year 2036. 5.2.2 IPMP Timestamp Format The IPMP Timestamp Format includes bits [16:31] of the 'seconds' word of an NTP timestamp and bits [0:23] of the 'seconds fraction' of an NTP timestamp. This format reduces the overall size of the Path Records. This reduced timestamp should be sufficient for all purposes of the IPMP protocol being accurate to about 50 nanoseconds, and taking 18 hours for the seconds field to wrap around. If archival time stamps are needed then the sending host can use IPMP info request messages, if needed, to determine the values of the high order seconds bits. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seconds | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seconds Fraction (0-padded) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 5.2.3 Inferred Real Time Bennett Expires - September 2003 [Page 26] IP Measurement Protocol March 2003 The real time of the timestamp may be inferred when a system provides an IPMP Information Reply with at least one Real Time Reference Point earlier and one later than the timestamp. For the purpose of this inference the clock drift of the interfaces clock is assumed to be linear and linear interpolation is used between the two nearest Real Time Reference Points where one is greater than and one is less than, the timestamp. The accuracy field of an Information Reply reports the greatest difference between an inferred real time, calculated using linear interpolation, and true real time. 5.3 Minimum Implementations. 5.3.1 Echoing System The simplest echoing system implementation returns the IPMP echo request without a path record. As described in section 4.2 this only requires that the IP source and destination addresses be exchanged, the type field changed and the packet scheduled for forwarding. Because of the format of the IPMP echo request and echo reply packets this can be implemented with a very small number of instructions. A system that does not insert path records does not need to processes IPMP Echo Requests. Systems which just provide this level of implementation allow a number of measurements to be made that are not currently possible, particularly if they are routers that processes ICMP at a low priority to avoid DOS attacks. 5.3.2 Forwarding System Forwarding systems do not need to be IPMP aware. A forwarding system that is IPMP aware MAY include path records with only the Forwarding IP Address set. This requires writing the address to the packet and updating the checksum and Path Pointer in the packet as described in section 4.3.1 and 5.1.1. In this case the forwarding system does not need to process IPMP Information Request packets. 6. Security Considerations Info request amplification and CPU use Bennett Expires - September 2003 [Page 27] IP Measurement Protocol March 2003 6.1 Denial of Service Attacks Because IPMP echo request packets are processed with about the same effort as forwarding an IP packet they do not introduce any new denial of service opportunities. IPMP Information Request packets require more processing and may be used as the basis of a denial of service attack in the same way as any information request on a router or host. Because Information Request packets are not used to make measurements an designer may implement protection against denial of service attacks made with these packets in the same way as other information requests. This might involve processing IPMP Information Requests at a low priority or regulating the maximum flow of packets. Since the IPMP Information Request packet is echoed by the responding interface it could be used as a reflector in a DOS attack by a host which sent packets with false source addresses. The use of a proxy host to respond to Information Request packets should make it possible to detect and stop such attacks. Also since the volume of legitimate IPMP Information Request traffic should be low, the proxy may simply limit how many requests it processes. The use of the redirection function for a denial of service attack is addressed in the Security Considerations section. 7. References [1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 [3] Braden, R., and J. Postel, "Requirements for Internet Gateways", STD 4, RFC 1009, USC/Information Sciences Institute, June 1987. [4] Mills, D., "Network Time Protocol (Version 3) Specification, Implementation and analysis", RFC 1305, March 1992. [5] Mills, D. "Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI", RFC 2030, October 1996. [6] Case, J., et al. "Protocol Operations for Version 2 of the Simple Network Management Protocol (SNMPv2)", RFC 1905, October 1996. Bennett Expires - September 2003 [Page 28] IP Measurement Protocol March 2003 Acknowledgments This draft draws very heavily, and in some cases draws verbatim from the text of the IPMP draft by Anthony J. McGregor as well as drawing from the experiences presents by the implementation and use of the protocol. This draft would not have been produced if not for those efforts. Author's Addresses Jon C. R. Bennett Harvard University Maxwell Dworkin 33 Oxford Street Cambridge, MA 02138 jcrb@yahoo.com Full Copyright Statement Copyright (c) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Bennett Expires - September 2003 [Page 29]