Internet Draft C. Bullard Document: draft-bullard-pcap-01 QoSient, LLC Category: Informational March, 2001 Remote Packet Capture Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1 Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. 2 Abstract This memo describes a set of recommendations for remote packet capture. The approach is designed to address deployment, performance and privacy issues that limit the usefulness of existing packet capture technology. Bullard Informational-Expires September 2001 1 Remote Packet Capture March, 2001 3 Table of Contents 1 Copyright Notice..............................................1 2 Abstract......................................................1 3 Table of Contents.............................................2 4 Overview......................................................2 5 Remote Packet Capture Requirements/Goals......................3 5.1 High Performance Packet Capture...............................3 5.2 Distributed Monitoring Applications...........................3 5.3 Support Remote Captured Packet Storage........................4 5.4 Wireline Packet Timestamping..................................4 5.5 Address Privacy Threat Issues.................................4 5.5.1 Content Separation............................................4 5.5.2 Captured Content Protection...................................4 5.6 Partial Packet Capture........................................5 5.7 Captured Packet Transport.....................................5 6 Recommendations...............................................5 6.1 Packet Header Capture.........................................5 6.2 Captured Packet Encapsulation Header..........................6 6.2.1 Source Identifier.............................................6 6.2.2 ifIndex.......................................................6 6.2.3 Status........................................................6 6.2.4 Sequence Number...............................................7 6.2.5 Time Stamp....................................................7 6.2.6 CapLength.....................................................7 6.2.7 PktLength.....................................................7 6.2.8 Captured Packet Data..........................................7 6.2.9 Pad...........................................................7 6.3 Multiple Captured Packet Encapsulation Header.................8 6.4 In-Band Captured Packet Transport.............................8 7 Security Considerations.......................................9 8 References....................................................9 9 Acknowledgments...............................................9 10 Author's Addresses............................................9 11 Full Copyright Statement.....................................10 4 Overview Packet capture is a fundamental mechanism in Internet network management and is used in support of a wide range of network operational functions, such as fault detection, protocol functional assurance, performance analysis, and security assessment. The IETF has specified technology for remote packet capture in RMON, STD-59[2], and working groups, such as RMON and IPPM, continue to describe various aspects of packet capture technology in RMON2, RFC-2021[3], SMON, RFC- 2613[4], and IPPM, RFC-2330[5]. Bullard Informational-Expires September 2001 2 Remote Packet Capture March, 2001 With new network paradigms and new network analytic requirements, existing packet capture technology is finding its limits, and vendors are generating proprietary mechanisms to overcome these hurdles. To address the issue of proprietary remote packet capture mechanisms in the switched network environment, the SMON MIB introduced the concept of an extended data source mechanism. This facility is designed to support vendors supplied "Copy Port" functions, and does not address the general issue of proprietary remote packet capture. This proposal recommends specification of a remote packet capture source facility that can act as a reliable external packet capture point in Internet networks. Designed to support a number of existing applications, such as RMON and RTFM, a remote packet capture facility should also enable new applications, such as those emerging from the IPPM and TE WGs. This proposal recommends that in order to overcome the perceived limitations of existing technology, new remote packet capture mechanisms should be required to support full wire speed packet header capture with wireline timestamping and in-band transport of the captured data. 5 Remote Packet Capture Requirements/Goals 5.1 High Performance Packet Capture A remote packet capture source facility should perform at a level that assures comprehensive packet capture at full duplex line rate, without packet loss. If packet loss does occur, the condition should be detectable and possibly quantifiable. Existing RMON packet capture technology cannot support sustained packet capture at existing link speeds, primarily due to local storage requirements. Port mirroring, or port copy, based packet capture technology, can perform full packet capture at wireline rates, but these mechanisms have inherent congestion problems when mirroring full duplex interfaces, and congestion related packet loss is not detectable or reportable. 5.2 Distributed Monitoring Applications A remote packet capture facility should provide reliable packet capture in asymmetric network architectures. Remote capture facilities should also support distributed monitoring applications, such as those where the same packet must be monitored at multiple points along its path. Both of these situations require multiple remote capture points with packet event correlation handled by a mediation device. Remote packet strategies should support this type of application. Bullard Informational-Expires September 2001 3 Remote Packet Capture March, 2001 5.3 Support Remote Captured Packet Storage One of the primary performance limitations in existing remote packet capture technology is the architectural requirement for local storage of captured packets. Because of the need to support very high-speed packet capture, any new remote packet strategy should support remote capture storage. 5.4 Wireline Packet Timestamping In order to perform time dependant analysis of network events, accurate time reporting is critical. All packet capture technology should support/conform to the wireline timestamp guidelines as described in the IPPM WG Framework document, RFC-2330. 5.5 Address Privacy Threat Issues Packet capture can pose a threat to individual privacy as it can be used to support unauthorized disclosure. In order to address this important issue, new packet capture technology must adopt strategies that help to minimize these threats when packet capture is used to support authorized tasks. 5.5.1 Content Separation Network management functions, such as RMON, must inspect some aspect of network datagram content in order to perform the designed function. If a particular network management function is an authorized function, then the minimum set of packet data required to support the authorized function will represent authorized data. All other packet contents can be referred to as unauthorized data content. Current packet capture technology does not support the separation of authorized data from unauthorized data, and as a result, current technology can pose a threat to privacy even when used to support authorized functions. A primary requirement of future packet capture technology should be that the facility support selective capture of authorized information. 5.5.2 Captured Content Protection Captured packet contents MUST be protected from unauthorized modification and/or disclosure. Integrity protection for captured packet contents is very important not only because network management decisions and actions will be made based on captured packet contents, but because fraudulent packet capture data can have significant impacts on individual privacy. Bullard Informational-Expires September 2001 4 Remote Packet Capture March, 2001 5.6 Partial Packet Capture Remote packet capture facilities should provide selective length packet capture. Applications may require full packet capture of a subset of network traffic, such as ICMP packets, but require only partial packet contents for other network traffic. A few examples would be Layer 2 headers capture to support the RMON Ethernet History function, or simple MPLS tags capture to support emerging TE performance measurements. 5.7 Captured Packet Transport One of the primary limitations of RMON packet capture is the architectural requirement that captured packets be stored locally on the probe. This single requirement limits the capacity for an RMON probe to capture packets in quantity, which limits either the capture load or the capture duration. The ability to transport captured packets off the probe to a mediation device can overcome this basic architectural barrier. 6 Recommendations 6.1 Packet Header Capture In order to support performance, security and application specific issues in remote packet capture, this proposal recommends that remote packet capture technology support packet capture on protocol header boundaries. If a network management function has authority to access IP header content information from packets of interest, such as the information needed to compile a RMON HostTopN list, the remote packet data source that is supplying the information needed for this function should be designed to provide only the IP header from packets of interest. This approach will significantly reduce the captured packet load, and limit the capture to the minimum set of information required of the application. And because there are hardware packet classifiers that are currently available that can make header boundary determinations at wireline speeds, this approach can be implemented as an integrated mechanism in high performance equipment. Bullard Informational-Expires September 2001 5 Remote Packet Capture March, 2001 6.2 Captured Packet Encapsulation Header To support distributed packet capture, partial packet capture, and wireline timestamping, this proposal recommends pre-pending captured packet data with a captured packet header. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ifIndex | Status | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (sec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (usec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CapLength | PktLength | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | Captured Packet Data +-+-+-+-+-+-+-+-+ | | Pad | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Draft PCAP Header Format 6.2.1 Source Identifier This uniquely identifies the source of this packet capture to the mediation device. 6.2.2 ifIndex This uniquely identifies the interface on which the packet was captured. 6.2.3 Status This bit field contains indicators that are specific to this particular packet, or all the packets in a particular group. Including direction (ingress or egress), whether the packet was going to be forwarded or dropped, etc.... Bullard Informational-Expires September 2001 6 Remote Packet Capture March, 2001 6.2.4 Sequence Number This unsigned integer is used to indicate the sequence number of the captured packet. This is used help the mediation device realize if any packets have been dropped by the capture facility. 6.2.5 Time Stamp The time stamp is the wireline timestamp of this captured packet data. This value is generated through guidelines set by the IPPM Framework. 6.2.6 CapLength This 16-bit unsigned value is the length of the captured packet data buffer. 6.2.7 PktLength This 16-bit unsigned value is the actual length of the capture packet. 6.2.8 Captured Packet Data This is the actual packet data that was captured. If the capLength field is zero (0), then this field is omitted. 6.2.9 Pad The pad is a MUST be zero (MBZ) 8 bit value that is added so that the entire captured packet structure is 16-bit aligned. Bullard Informational-Expires September 2001 7 Remote Packet Capture March, 2001 6.3 Multiple Captured Packet Encapsulation Header To support collecting multiple packets together into a single captured packet buffer, a multiple captured packet header is defined. The values, such as Source Identifier, ifIndex, Status and Time Stamp apply to all the packets included in the buffer. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ifIndex | Group Status | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (sec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (usec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CapLength | PktLength | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Offset (usec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | Captured Packet Data +-+-+-+-+-+-+-+-+ | | Pad | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CapLength | PktLength | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Offset (usec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | Captured Packet Data +-+-+-+-+-+-+-+-+ | | Pad | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Draft PCAP Header Format 6.4 In-Band Captured Packet Transport In order to support high-speed integrated implementations, as well as to meet the requirement to support security protection for captured packets, this proposal recommends In-Band transport of captured packets. In-Band transport can be accomplished through the use of Layer 2 encapsulation, IP header encapsulation or UDP/TCP transport strategies. NSAP identifiers may need to be allocated to identify the packet as a Bullard Informational-Expires September 2001 8 Remote Packet Capture March, 2001 packet capture transport packet, when using Layer 2 and IP transport schemes. In-Band transport may involve fragmentation when supporting large packet capture sizes. 7 Security Considerations Protection against known security threats such as unauthorized access, modification or disclosure of remotely captured packet content can be accomplished using existing IETF specified technology, particularly SNMPv3 and IPSec. Threats against captured packet data while resident in a persistent store is beyond the scope of this document. 8 References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Waldbusser, S., "Remote Network Monitoring Management Information Base", STD-59, May 2000 3 Waldbusser, S., "Remote Network Monitoring Management Information Base Version 2 using SMIv2", RFC 2021, January 1997 4 Waterman, R., Lahaye, B., Romascanu, D. Waldbusser, S., "Remote Network Monitoring MIB Extensions for Switched Networks Version 1.0", RFC 2613, June 1999 5 Paxson, V., Almes, G., Mahdavi, J., Mathis, M., "Framework for IP Performance Metrics", RFC 2330, May 1998 9 Acknowledgments 10 Author's Addresses Carter Bullard QoSient, LLC. 300 E. 56th Street, Suite 18K New York, New York 10022-1439 Phone: +1 212 588 9133 Email: carter@qosient.com Bullard Informational-Expires September 2001 9 Remote Packet Capture March, 2001 11 Full Copyright Statement Copyright (C) The Internet Society (2000). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Bullard Informational-Expires September 2001 10