NMRG J. Schoenwaelder
Internet-Draft Jacobs University Bremen
Intended status: Informational February 12, 2008
Expires: August 15, 2008
SNMP Traffic Measurements and Trace Exchange Formats
draft-irtf-nmrg-snmp-measure-03.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on August 15, 2008.
Copyright Notice
Copyright (C) The IETF Trust (2008).
Schoenwaelder Expires August 15, 2008 [Page 1]
Internet-Draft SNMP Traffic Measurements February 2008
Abstract
The Simple Network Management Protocol (SNMP) is widely deployed to
monitor, control and (sometimes also) configure network elements.
Even though the SNMP technology is well documented, it remains
relatively unclear how SNMP is used in practice and what typical SNMP
usage patterns are.
This document proposes to carry out large scale SNMP traffic
measurements in order to develop a better understanding how SNMP is
used in real world production networks. It describes the motivation,
the measurement approach, and the tools and data formats needed to
carry out such a study.
This document was produced within the IRTF's Network Management
Research Group (NMRG) and represents the consensus of all of the
active contributors to this group.
Schoenwaelder Expires August 15, 2008 [Page 2]
Internet-Draft SNMP Traffic Measurements February 2008
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Measurement Approach . . . . . . . . . . . . . . . . . . . . . 6
2.1. Capturing Traffic Traces . . . . . . . . . . . . . . . . . 6
2.2. Converting Traffic Traces . . . . . . . . . . . . . . . . 7
2.3. Filtering Traffic Traces . . . . . . . . . . . . . . . . . 8
2.4. Storing Traffic Traces . . . . . . . . . . . . . . . . . . 9
2.5. Analyzing Traffic Traces . . . . . . . . . . . . . . . . . 9
3. Analysis of Traffic Traces . . . . . . . . . . . . . . . . . . 11
3.1. Basic Statistics . . . . . . . . . . . . . . . . . . . . . 11
3.2. Periodic vs. Aperiodic Traffic . . . . . . . . . . . . . . 11
3.3. Message Size and Latency Distributions . . . . . . . . . . 11
3.4. Concurrency Levels . . . . . . . . . . . . . . . . . . . . 12
3.5. Table Retrieval Approaches . . . . . . . . . . . . . . . . 12
3.6. Trap-Directed Polling - Myths or Reality? . . . . . . . . 12
3.7. Popular MIB Definitions . . . . . . . . . . . . . . . . . 13
3.8. Usage of Obsolete Objects . . . . . . . . . . . . . . . . 13
3.9. Encoding Length Distributions . . . . . . . . . . . . . . 13
3.10. Counters and Discontinuities . . . . . . . . . . . . . . . 13
3.11. Spin Locks . . . . . . . . . . . . . . . . . . . . . . . . 14
3.12. Row Creation . . . . . . . . . . . . . . . . . . . . . . . 14
4. Trace Exchange Formats . . . . . . . . . . . . . . . . . . . . 15
4.1. XML Representation . . . . . . . . . . . . . . . . . . . . 15
4.2. CSV Representation . . . . . . . . . . . . . . . . . . . . 20
5. Security Considerations . . . . . . . . . . . . . . . . . . . 22
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8.1. Normative References . . . . . . . . . . . . . . . . . . . 25
8.2. Informative References . . . . . . . . . . . . . . . . . . 25
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 29
Intellectual Property and Copyright Statements . . . . . . . . . . 30
Schoenwaelder Expires August 15, 2008 [Page 3]
Internet-Draft SNMP Traffic Measurements February 2008
1. Introduction
The Simple Network Management Protocol (SNMP) was introduced in the
late 1980s [RFC1052] and has since then evolved to what is known
today as the SNMP version 3 Framework (SNMPv3) [RFC3410]. While SNMP
is widely deployed, it is not clear what protocol versions are being
used, which protocol features are being used, how SNMP usage differs
in different types of networks or organizations, which information is
frequently queried, and what typical SNMP interactions patterns occur
in real world production networks.
There have been several publications in the recent past dealing with
the performance of SNMP in general [SM99][Mal02][Pat01], the impact
of SNMPv3 security [DSR01][CT04], or the relative performance of SNMP
compared to Web Services [PDMQ04][PFGL04]. While these papers are
generally useful to better understand the impact of various design
decisions and technologies, some of these papers lack a strong
foundation because authors typically assume certain SNMP interaction
patterns without having experimental evidence that the assumptions
are correct. In fact, there are many speculations how SNMP is being
used in real world production networks and performance comparisons
are based on limited test cases, but no systematic measurements have
been performed and published so far.
Many authors use the ifTable of the IF-MIB [RFC2863] or the
tcpConnTable of the TCP-MIB [RFC4022] as a starting point for their
analysis and comparison. Despite the fact that there is no evidence
that operations on these tables dominate SNMP traffic, it is even
more unclear how these tables are read and which optimizations are
done (or not done) by real world applications. It is also unclear
what the actual traffic trade-off between periodic polling and more
aperiodic bulk data retrieval is. Furthermore, we do not generally
understand how much traffic is devoted to standardized MIB objects
and how much traffic deals with proprietary MIB objects and whether
the operation mix between these object classes differs between
different operational environments (e.g., backbone networks, access
networks, enterprise networks).
This document describes an effort to collect SNMP traffic traces in
order to find answers to some of these questions. It describes the
tools that have been developed to allow network operators to collect
traffic traces and to share them with research groups interested in
analyzing and modeling network management interactions.
While the SNMP trace collection and analysis effort was initiated by
the research community, it should be noted that network operators can
benefit from the SNMP measurements too. Several new tools are being
developed as part of this effort that can be used to capture and
Schoenwaelder Expires August 15, 2008 [Page 4]
Internet-Draft SNMP Traffic Measurements February 2008
analyze the traffic generated by management stations. This resulting
information can then be used to improve the efficiency and
scalability of management systems.
This is an informational document, produced within the IRTF's Network
Management Research Group (NMRG) and represents the consensus of all
of the active contributors to this group.
Schoenwaelder Expires August 15, 2008 [Page 5]
Internet-Draft SNMP Traffic Measurements February 2008
2. Measurement Approach
This section outlines the process of doing SNMP traffic measurements
and analysis. The process consists of the following five basic
steps:
1. Capture raw SNMP traffic traces in pcap packet capture files [1].
2. Convert the raw traffic traces into a structured machine and
human readable format. A suitable XML schema has been developed
for this purpose which captures all SNMP message details. In
addition, another more compact comma separated values (CSV)
format has been developed which only keeps key information about
SNMP messages.
3. Filter the converted traffic traces to hide or anonymize
sensitive information. While the filtering is conceptually a
separate step, filtering may actually be implemented as part of
the previous data conversion step for efficiency reasons.
4. Submit the filtered traffic traces to a repository from where
they can be retrieved and analyzed. Such a repository may be
public, it may be under the control of a research group, or it
may be under the control of a network operator who commits to run
analysis scripts on the repository on behalf of researchers.
5. Analyze the traces by creating and executing analysis scripts
which extract and aggregate information.
Several of the steps listed above require the involvement of network
operators supporting the SNMP measurement projects. In many cases,
the filtered XML and CSV representation of the SNMP traces will be
the interface between the researchers writing analysis scripts and
the operators involved in the measurement activity. It is therefore
important to have a well defined specification of these interfaces.
This section provides some advice and concrete hints how the steps
listed above can be carried out efficiently. Some special tools have
been developed to assist network operators and researchers so that
the time spent on supporting SNMP traffic measurement projects is
limited. The following sections describe the five steps and some
tools in more detail.
2.1. Capturing Traffic Traces
Capturing SNMP traffic traces can be done using packet sniffers such
as tcpdump [2], wireshark [3], or similar applications. Some care
must be taken to specify pcap filter expressions that match the SNMP
Schoenwaelder Expires August 15, 2008 [Page 6]
Internet-Draft SNMP Traffic Measurements February 2008
transport endpoints used to carry SNMP traffic (typically 'udp and
(port 161 or port 162)'). Furthermore, it is necessary to ensure
that full packets are captured, that is packets are not truncated
(tcpdump option -s 0). Finally, it is necessary to carefully select
the placement of the capturing probe within the network. Especially
on bridged LANs, it is important to ensure that all management
traffic is captured and that the probe has access to all virtual LANs
carrying management traffic. This usually requires placing the
probe(s) close to the management system(s) and to configure dedicated
monitoring ports on bridged networks. Some bridges have restrictions
concerning their monitoring capabilities and this should be
investigated and documented where necessary.
It is recommended to capture at least a full week of data. Operators
are encouraged to capture traces over even longer periods of time.
Tools such as tcpslice [2] or pcapmerge [4] can be used to split or
merge pcap trace files as needed.
Several operating systems can offload some of the TCP/IP processing
such as the calculation of transport layer checksum to network
interface cards. Traces that include traffic to/from a capturing
interface which supports TCP/IP offloading can include incorrect
transport layer checksums. The simplest solution is of course to
turn checksum offloading off while capturing traces (if that is
feasible without loosing too many packets). The other solution is to
correct or ignore checksums during the subsequent conversion of the
raw pcap files.
It is important to note that the raw pcap files should ideally be
kept in permanent storage (e.g., compressed and encrypted on a CD ROM
or DVD). To verify measurements, it might be necessary to go back to
the original pcap files if, for example, bugs in the tools described
below have been detected and fixed.
For each captured trace, some meta data should be recorded and made
available. The meta data should include information such as where
the trace was collected, when it was collected, contact information,
the size of the trace, any known special events, equipment failures,
or major infrastructure changes during the data collection period and
so on. It is also extremely useful to provide a unique
identification. There are special online services such as DatCat [5]
where meta data can be stored and which provide unique identifiers.
2.2. Converting Traffic Traces
Raw traces in pcap format must be converted into a format that is
human readable while remaining also machine readable for efficient
post-processing. Human readability makes it easy for an operator to
Schoenwaelder Expires August 15, 2008 [Page 7]
Internet-Draft SNMP Traffic Measurements February 2008
verify that no sensitive data is left in a trace while machine
readability is needed to efficiently extract relevant information.
The natural choice here is to use an XML format since XML is human as
well as machine readable and there are many tools and high-level
scripting language application programming interfaces (APIs) that can
be used to process XML documents and to extract meaningful
information. However, it should be noted that XML is also pretty
verbose which increases processing overhead. In particular, the
usage of XML streaming APIs is strongly suggested since APIs that
require an in memory representation of XML documents do not handle
large traces well.
Section 4.1 of this document defines a RELAX NG schema [OASISRNG] for
representing SNMP traffic traces in XML. The schema captures all
relevant details of an SNMP message in the XML format. Note that the
XML format retains some information about the original ASN.1/BER
encoding to support message size analysis.
A lightweight alternative to the full blown XML representation based
on comma separated values (CSV) is defined in Section 4.2. The CSV
format only captures selected parts of SNMP messages and is thus more
compact and faster to process.
As explained in the previous sections, analysis programs which
process raw pcap files should have an option to ignore incorrect
checksums caused by TCP/IP offloading. In addition, analysis
programs which process raw pcap files should be able to perform IP
reassembly for SNMP messages that got fragmented at the IP layer.
The snmpdump [6] package has been developed to convert raw pcap files
into XML and CSV format. The snmpdump program reads pcap, XML, or
CSV files as input and produces XML files or CSV files as output.
Specific elements can be filtered as required to protect sensitive
data.
2.3. Filtering Traffic Traces
Filtering sensitive data (e.g., access control lists or community
strings) can be achieved by manipulating the XML representation of an
SNMP trace. Standard XSLT processors (e.g., xsltproc [7]) can be
used for this purpose. People familiar with the scripting language
Perl might be interested in choosing a suitable Perl module to
manipulate XML documents [8].
The snmpdump program, for example, can filter out sensitive
information, e.g., by deleting or clearing all XML elements whose
name matches a regular expression. In addition, data type specific
Schoenwaelder Expires August 15, 2008 [Page 8]
Internet-Draft SNMP Traffic Measurements February 2008
anonymization transformations that maintain lexicographic ordering
for values that appear in instance identifiers [HS06] can be applied.
Note that anonymization transformations are often bound to an
initialization key and depend on the data being anonymized in an
anonymization run. As a consequence, users must be careful when they
merge data from independently anonymized traces. More information
about network traffic trace anonymization techniques can be found in
[XFA02], [FXAM04], [PAPL06], and [RW07].
2.4. Storing Traffic Traces
The raw pcap traces as well as the XML / CSV formatted traces should
be stored in a stable archive or repository. Such an archive or
repository might either be maintained by research groups (e.g., the
NMRG) or by network operators or both. It is of key importance that
captured traces are not lost or modified as they may form the basis
of future research projects and may also be needed to verify
published research results. Access to the archive might be
restricted to those who have signed some sort of a non-disclosure
agreement.
While this document recommends that raw traces should be kept, it
must be noted that there are situations where this may not be
feasible. The recommendation to keep raw traces may be ignored for
example to comply with data protection laws or to protect a network
operator from being forced to provide the data to other
organizations.
Lossless compression algorithms embodied in programs such as gzip or
bzip2 can be used to compress even large trace files down to a size
where they can be burned on DVDs for cheap long-term storage.
It must be stressed again that it is important to keep the original
pcap traces in addition to the XML / CSV formatted traces since the
pcap traces are the most authentic source of information.
Improvements in the tool chain may require to go back to the original
pcap traces and to rebuild all intermediate formats from them.
2.5. Analyzing Traffic Traces
Scripts that analyze traffic traces must be verified for correctness.
Ideally, all scripts used to analyze traffic traces would be
publically accessible so that third parties can verify them.
Furthermore, sharing scripts will enable other parties to repeat an
analysis on other traffic traces and to extend such analysis scripts.
A common versioned repository for analysis scripts might be useful to
establish.
Schoenwaelder Expires August 15, 2008 [Page 9]
Internet-Draft SNMP Traffic Measurements February 2008
Due to the availability of XML parsers and the simplicity of the CSV
format, trace files can be processed with tools written in almost any
programming language. However, in order to facilitate a common
vocabulary and to allow operators to easily read scripts they execute
on trace files, it is suggested that analysis scripts are written in
scripting languages such as Perl using suitable Perl modules to
manipulate XML documents [8]. Using a scripting language such as
Perl instead of system programming languages such as C or C++ has the
advantage to reduce development time and to make scripts more
accessible to operators who may want to verify scripts before running
them on trace files which potentially contain sensitive data.
It should be noted here that the snmpdump tool provides an API to
process SNMP messages in C/C++. While the coding of trace analysis
programs in C/C++ should in general be avoided for code readability,
verifiability and portability reasons, using C/C++ might be the only
option to deal with very large traces efficiently.
It should be noted that any results produced by analyzing a trace
must be interpreted in the context of the trace. The nature of the
network, the attachment point used to collect the trace, the nature
of the applications generating SNMP traffic, or the events that
happened while the trace was collected clearly influence the result.
It is therefore important to be careful when drawing general
conclusions based on a potentially (too) limited data set.
Schoenwaelder Expires August 15, 2008 [Page 10]
Internet-Draft SNMP Traffic Measurements February 2008
3. Analysis of Traffic Traces
This section discusses several questions that can be answered by
analyzing SNMP traffic traces. The questions raised in the following
subsections are meant to be illustrative and no attempt has been made
to provide a complete list.
3.1. Basic Statistics
Basic statistics cover things such as
o protocol version used,
o protocol operations used,
o message size distribution,
o error message type frequency, or
o usage of authentication and encryption mechanisms.
In addition, the OID names of the objects manipulated can be
categorized into OID subtrees, for example to identify
'standardized', 'proprietary', and 'experimental' objects.
3.2. Periodic vs. Aperiodic Traffic
SNMP is used to periodically poll devices as well as to retrieve
information on request of an operator or application. The periodic
polling leads to periodic traffic patterns while the on demand
information retrieval causes more aperiodic traffic patterns. It is
worthwhile to understand what the relationship is between the amount
of periodic and aperiodic traffic. In addition, it will be
interesting to research whether there are multiple levels of
periodicity at different time scales.
The periodic polling behavior may dependent on the application and
the polling engine it uses. For example, some management platforms
allow applications to specify how long polled values may be kept in a
cache before it is polled again. Such optimizations needs to be
considered when analyzing traces for periodic and aperiodic traffic.
3.3. Message Size and Latency Distributions
SNMP messages are size constrained by the transport mappings used and
the buffers provided by the SNMP engines. For the further evolution
of the SNMP framework, it would be useful to know what the actual
message size distributions are. In addition, it would be useful to
Schoenwaelder Expires August 15, 2008 [Page 11]
Internet-Draft SNMP Traffic Measurements February 2008
understand the latency distributions, especially the distribution of
the processing times by SNMP command responders. Some SNMP
implementations approximate networking delays by measuring request-
response times and it would be useful to understand to what extent
this is a viable approach.
Some SNMP implementations update their counters from the underlying
instrumentation following adaptive algorthms, not necessarily
periodically, and not necessarily on-demand. The granularity of
internal counter updates may impact latency measurements and should
be taken into account.
3.4. Concurrency Levels
SNMP allows management stations to retrieve information from multiple
agents concurrently. It will be interesting to identify what the
typical concurrency level is that can be observed on production
networks or whether management applications prefer more sequential
ways of retrieving data.
In addition, it will be interesting to analyze how many redundant
requests coming from applications are processed almost simultaneously
by a device.
3.5. Table Retrieval Approaches
Tables can be read in several different ways. The simplest and most
inefficient approach is to retrieve tables object-by-object in
column-by-column order. More advanced approaches try to read tables
row-by-row or even multiple-rows-by-multiple-rows. In addition, the
retrieval of index elements can be suppressed in most cases or only a
subset of columns of a table are retrieved. It will be useful to
know which of these approaches are actually used on production
networks.
3.6. Trap-Directed Polling - Myths or Reality?
SNMP is built around a concept called trap-directed polling.
Management applications are responsible to periodically poll SNMP
agents to determine their status. SNMP agents can in addition send
traps to notify SNMP managers about events so that SNMP managers can
adopt their polling strategy and basically react faster than normal
polling would allow to do.
Analysis of SNMP traffic traces can identify whether trap-directed
polling is actually deployed. In particular, the question that
should be addressed is whether SNMP notifications lead to changes in
the short-term polling behavior of management stations. In
Schoenwaelder Expires August 15, 2008 [Page 12]
Internet-Draft SNMP Traffic Measurements February 2008
particular, it should be investigated to what extent SNMP managers
use automated procedures to track down the meaning of the event
conveyed by an SNMP notification.
3.7. Popular MIB Definitions
An analysis of object identifier prefixes can identify the most
popular MIB modules and the most important object types or
notification types defined by these modules. Such information would
be very valuable for the further maintenance and development of these
and related MIB modules.
3.8. Usage of Obsolete Objects
Several objects from the early days have been obsoleted because they
cannot properly represent today's networks. A typical example is the
ipRouteTable which was obsoleted because it was not able to represent
classless routing, introduced and deployed on the Internet in 1993.
Some of these obsolete objects are still mentioned in popular
publications as well as research papers. It will be interesting to
find out whether they are also still used by management applications
or whether management applications have been updated to use the
replacement objects.
Depending on the data recorded in a trace, it might be possible to
determine the age of devices by looking at values of objects such as
sysObjectID and sysDecr [RFC3418]. The age of a device can then be
taken into consideration when analyzing the use of obsolete and
deprecated objects.
3.9. Encoding Length Distributions
It will be useful to understand the encoding length distributions for
various data types. Assumptions about encoding length distributions
are sometimes used to estimate SNMP message sizes in order to meet
transport and buffer size constraints.
3.10. Counters and Discontinuities
Counters can experience discontinuities [RFC2578]. A widely used
discontinuity indicator is the sysUpTime scalar of the SNMPv2-MIB
[RFC3418], which can be reset through a warm start to indicate
counter discontinuities. Some MIB modules introduce more specific
discontinuity indicators, e.g., the ifCounterDiscontinuityTime of the
IF-MIB [RFC2863]. It will be interesting to study to what extent
these objects are actually used by management applications to handle
discontinuity events.
Schoenwaelder Expires August 15, 2008 [Page 13]
Internet-Draft SNMP Traffic Measurements February 2008
3.11. Spin Locks
Cooperating command generators can use advisory locks to coordinate
their usage of SNMP write operations. The snmpSetSerialNo scalar of
the SNMPv2-MIB [RFC3418] is the default course-grain coordination
object. It will be interesting to find out whether there are command
generators which coordinate themselves using these spin locks.
3.12. Row Creation
Row creation is an operation not natively supported by the protocol
operations. Instead, conceptual tables supporting row creation
typically provide a control column which uses the RowStatus textual
convention defined in the SNMPv2-TC [RFC2579] module. The RowStatus
itself supports different row creation modes, namely createAndWait
(dribble-mode) and createAndGo (one-shot mode). In addition,
different approaches can be used to derive the instance identifier if
it does not have special semantics associated. It will be useful to
study which of the various row creation approaches are actually used
by management applications on production networks.
Schoenwaelder Expires August 15, 2008 [Page 14]
Internet-Draft SNMP Traffic Measurements February 2008
4. Trace Exchange Formats
4.1. XML Representation
The XML format has been designed to keep all information associated
with SNMP messages. The format is specified in RELAX NG compact
notation [OASISRNC]. Freely available tools such as trang [9] can be
used to convert RELAX NG compact syntax to other XML schema
notations.
# Relax NG grammar for the XML SNMP trace format.
#
# Published as part of RFC XXXX.
#
# $Id: measure.xml 2660 2008-01-08 11:55:28Z schoenw $
default namespace = "urn:ietf:params:xml:ns:snmp-trace-1.0"
start =
element snmptrace {
packet.elem*
}
packet.elem =
element packet {
element time-sec { xsd:unsignedInt },
element time-usec { xsd:unsignedInt },
element src-ip { ipaddress.type },
element src-port { xsd:unsignedInt },
element dst-ip { ipaddress.type },
element dst-port { xsd:unsignedInt },
snmp.elem
}
snmp.elem =
element snmp {
length.attrs?,
message.elem
}
message.elem =
element version { length.attrs, xsd:int },
element community { length.attrs, xsd:hexBinary },
pdu.elem
message.elem |=
element version { length.attrs, xsd:int },
element message {
Schoenwaelder Expires August 15, 2008 [Page 15]
Internet-Draft SNMP Traffic Measurements February 2008
length.attrs,
element msg-id { length.attrs, xsd:unsignedInt },
element max-size { length.attrs, xsd:unsignedInt },
element flags { length.attrs, xsd:hexBinary },
element security-model { length.attrs, xsd:unsignedInt }
},
usm.elem?,
element scoped-pdu {
length.attrs,
element context-engine-id { length.attrs, xsd:hexBinary },
element context-name { length.attrs, xsd:string },
pdu.elem
}
usm.elem =
element usm {
length.attrs,
element auth-engine-id { length.attrs, xsd:hexBinary },
element auth-engine-boots { length.attrs, xsd:unsignedInt },
element auth-engine-time { length.attrs, xsd:unsignedInt },
element user { length.attrs, xsd:hexBinary },
element auth-params { length.attrs, xsd:hexBinary },
element priv-params { length.attrs, xsd:hexBinary }
}
pdu.elem =
element trap {
length.attrs,
element enterprise { length.attrs, oid.type },
element agent-addr { length.attrs, ipv4address.type },
element generic-trap { length.attrs, xsd:int },
element specific-trap { length.attrs, xsd:int },
element time-stamp { length.attrs, xsd:int },
element variable-bindings { length.attrs, varbind.elem* }
}
pdu.elem |=
element (get-request | get-next-request | get-bulk-request |
set-request | inform-request | snmpV2-trap |
response | report) {
length.attrs,
element request-id { length.attrs, xsd:int },
element error-status { length.attrs, xsd:int },
element error-index { length.attrs, xsd:int },
element variable-bindings { length.attrs, varbind.elem* }
}
varbind.elem =
Schoenwaelder Expires August 15, 2008 [Page 16]
Internet-Draft SNMP Traffic Measurements February 2008
element varbind { length.attrs, name.elem, value.elem }
name.elem =
element name { length.attrs, oid.type }
value.elem =
element null { length.attrs, empty } |
element integer32 { length.attrs, xsd:int } |
element unsigned32 { length.attrs, xsd:unsignedInt } |
element counter32 { length.attrs, xsd:unsignedInt } |
element counter64 { length.attrs, xsd:unsignedLong } |
element timeticks { length.attrs, xsd:unsignedInt } |
element ipaddress { length.attrs, ipv4address.type } |
element octet-string { length.attrs, xsd:hexBinary } |
element object-identifier { length.attrs, oid.type } |
element opaque { length.attrs, xsd:hexBinary } |
element no-such-object { length.attrs, empty } |
element no-such-instance { length.attrs, empty } |
element end-of-mib-view { length.attrs, empty }
# The blen attribute indicates the number of octets used by the BER
# encoded tag / length / value triple. The vlen attribute indicates
# the number of octets used by the BER encoded value alone.
length.attrs =
( attribute blen { xsd:unsignedShort },
attribute vlen { xsd:unsignedShort } )?
oid.type =
xsd:string {
pattern =
"""[0-2](\.[0-9]+)+"""
}
# The types below are for IP addresses. Note that SNMP's buildin
# IpAddress type only supports IPv4 addresses; IPv6 addresses are only
# introduced to cover SNMP over IPv6 endpoints.
ipv4address.type =
xsd:string {
pattern =
"""[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*"""
}
ipv6address.type =
xsd:string {
pattern =
"""(([0-9a-fA-F]+:){7}[0-9a-fA-F]+)|
Schoenwaelder Expires August 15, 2008 [Page 17]
Internet-Draft SNMP Traffic Measurements February 2008
(([0-9a-fA-F]+:)*[0-9a-fA-F]+)?::(([0-9a-fA-F]+:)*[0-9a-fA-F]+)?"""
}
ipaddress.type = ipv4address.type | ipv6address.type
The following example shows an SNMP trace file in XML format
containing an SNMPv1 get-next-request message for the OID
1.3.6.1.2.1.1.3 (sysUpTime) and the response message returned by the
agent.
Schoenwaelder Expires August 15, 2008 [Page 18]
Internet-Draft SNMP Traffic Measurements February 2008
1147212206
739609
192.0.2.1
60371
192.0.2.2
12345
1
7075626c6963
1804289383
0
0
1.3.6.1.2.1.1.3
1147212206
762891
192.0.2.2
12345
192.0.2.1
60371
1
7075626c6963
1804289383
0
0
1.3.6.1.2.1.1.3.0
26842224
Schoenwaelder Expires August 15, 2008 [Page 19]
Internet-Draft SNMP Traffic Measurements February 2008
4.2. CSV Representation
The comma separated values (CSV) format has been designed to capture
only the most relevant information about an SNMP message. In
situation where all information about an SNMP message must be
captured, the XML format defined above must be used. The CSV format
uses the following fields:
1. Time-stamp in the format seconds.microseconds since 1970, for
example "1137764769.425484".
2. Source IP address in dotted quad notation (IPv4), for example
"192.0.2.1", or compact hexadecimal notation (IPv6), for example
"2001:DB8::1".
3. Source port number represented as a decimal number, for example
"4242".
4. Destination IP address in dotted quad notation (IPv4), for
example "192.0.2.1", or compact hexadecimal notation (IPv6), for
example "2001:DB8::1".
5. Destination port number represented as a decimal number, for
example "161".
6. Size of the SNMP message (a decimal number) counted in octets,
for example "123". The size excludes all transport, network,
and link-layer headers.
7. SNMP message version represented as a decimal number. The
version 0 stands for SNMPv1, 1 for SNMPv2c, and 3 for SNMPv3,
for example "3".
8. SNMP protocol operation indicated by one of the keywords get-
request, get-next-request, get-bulk-request, set-request, trap,
snmpV2-trap, inform-request, response, report.
9. SNMP request-id in decimal notation, for example "1511411010".
10. SNMP error-status in decimal notation, for example "0".
11. SNMP error-index in decimal notation, for example "0".
12. Number of variable-bindings contained in the varbind-list in
decimal notation, for example "5".
13. For each varbind in the varbind list, three output elements are
generated
Schoenwaelder Expires August 15, 2008 [Page 20]
Internet-Draft SNMP Traffic Measurements February 2008
1. Object name given as object identifier in dotted decimal
notation, for example "1.3.6.1.2.1.1.3.0". Object names are
separated by commas.
2. Object base type name or exception name, that is one of the
following: null, integer32, unsigned32, counter32,
counter64, timeticks, ipaddress, octet-string, object-
identifier, opaque, no-such-object, no-such-instance, and
end-of-mib-view.
3. Object value is printed as a number if the underlying base
type is numeric. An IPv4 addresses is rendered in the
dotted quad notation and an IPv6 address is rendered in the
usual hexadecimal notation. An octet string value is
printed in hexadecimal format while an object identifier
value is printed in dotted decimal notation. In case of an
exception, the object value is an empty string.
Note that the format does not preserve the information needed to
understand SNMPv1 traps. It is therefore recommended that
implementations are able to convert the SNMPv1 trap format into the
trap format used by SNMPv2c and SNMPv3, according to the rules
defined in [RFC3584]. The activation of trap format conversion
should be the user's choice.
The following example shows an SNMP trace file in CSV format
containing an SNMPv1 get-next-request message for the OID
1.3.6.1.2.1.1.3 (sysUpTime) and the response message returned by the
agent. (Note that the example uses backslash line continuation marks
in order to fit the example into the RFC format. Backslash line
continuations are not part of the CSV format.)
1147212206.739609,192.0.2.1,60371,192.0.2.2,12345,42,1,\
get-next-request,1804289383,0,0,1,1.3.6.1.2.1.1.3,null,
1147212206.762891,192.0.2.2,12345,192.0.2.1,60371,47,1,\
response,1804289383,0,0,1,1.3.6.1.2.1.1.3.0,timeticks,26842224
Schoenwaelder Expires August 15, 2008 [Page 21]
Internet-Draft SNMP Traffic Measurements February 2008
5. Security Considerations
SNMP traffic traces usually contain sensitive information. It is
therefore necessary to (a) remove unwanted information and (b) to
anonymize the remaining necessary information before traces are made
available for analysis. In addition, it is recommended to encrypt
traces when they are archived.
Implementations that generate XML traces from raw pcap files should
have an option to suppress or anonymize values. Note that instance
identifiers of tables also include values and it might therefore be
necessary to suppress or anonymize (parts of) the instance
identifiers. Similarly, the packet and message headers typically
contain sensitive information about the source and destination of
SNMP messages as well as authentication information (community
strings or user names).
Anonymization techniques can be applied to keep information in traces
which could otherwise reveal sensitive information. When using
anonymization, values should only be added when the underlying data
type is known and an appropriate anonymization transformation is
available (filter-in principle). For values appearing in instance
identifiers, it is usually desirable to maintain the lexicographic
order. Special anonymization transformations which preserve this
property have been developed, although their anonymization strength
is usually reduced compared to transformations that do not preserve
lexicographic ordering [HS06].
Schoenwaelder Expires August 15, 2008 [Page 22]
Internet-Draft SNMP Traffic Measurements February 2008
6. IANA Considerations
This document requests that IANA register a URI for the SNMP XML
trace format namespace in the IETF XML registry [RFC3688]. Following
the format in RFC 3688, the following registration is requested.
URI: Please assign the URI "urn:ietf:params:xml:ns:snmp-trace-1.0"
for use by the SNMP trace format.
Registrant Contact: The NMRG of the IRTF.
XML: N/A, the requested URI is an XML namespace.
Schoenwaelder Expires August 15, 2008 [Page 23]
Internet-Draft SNMP Traffic Measurements February 2008
7. Acknowledgements
This document was influenced by discussions within the Network
Management Research Group (NMRG). Special thanks to Remco van de
Meent for writing the initial Perl script that lead to the
development of the snmpdump software package and Matus Harvan for his
work on lexicographic order preserving anonymization transformations.
Aiko Pras contributed ideas to Section 3 while David Harrington
helped to improve the readability of this document.
Last call reviews have been received from Bert Wijnen, Aiko Pras,
Frank Strauss, Remco van de Meent, Giorgio Nunzi, Wes Hardacker, Liam
Fallon, Sharon Chisholm, David Perkins, Deep Medhi, Randy Bush, David
Harrington, Dan Romascanu, Luca Deri, and Marc Burgess.
Part of this work was funded by the European Commission under grant
FP6-2004-IST-4-EMANICS-026854-NOE.
Schoenwaelder Expires August 15, 2008 [Page 24]
Internet-Draft SNMP Traffic Measurements February 2008
8. References
8.1. Normative References
[RFC2578] McCloghrie, K., Perkins, D., and J. Schoenwaelder,
"Structure of Management Information Version 2 (SMIv2)",
STD 58, RFC 2578, April 1999.
[OASISRNG]
Clark, J. and M. Makoto, "RELAX NG Specification",
OASIS Committee Specification, December 2001.
[OASISRNC]
Clark, J., "RELAX NG Compact Syntax", OASIS Committee
Specification, November 2002.
[RFC3584] Frye, R., Levi, D., Routhier, S., and B. Wijnen,
"Coexistence between Version 1, Version 2, and Version 3
of the Internet-standard Network Management Framework",
RFC 3584, August 2003.
[RFC3688] Mealling, M., "The IETF XML Registry", RFC 3688,
January 2004.
8.2. Informative References
[RFC1052] Cerf, V., "IAB Recommendations for the Development of
Internet Network Management Standards", RFC 1052,
April 1998.
[RFC2579] McCloghrie, K., Perkins, D., and J. Schoenwaelder,
"Textual Conventions for SMIv2", STD 58, RFC 2579,
April 1999.
[RFC3418] Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S.
Waldbusser, "Management Information Base (MIB) for the
Simple Network Management Protocol (SNMP)", STD 62,
RFC 3418, December 2002.
[RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group
MIB", RFC 2863, June 2000.
[RFC3410] Case, J., Mundy, R., Partain, D., and B. Stewart,
"Introduction and Applicability Statements for Internet
Standard Management Framework", RFC 3410, December 2002.
[RFC4022] Raghunarayan, R., "Management Information Base for the
Transmission Control Protocol (TCP)", RFC 4022,
Schoenwaelder Expires August 15, 2008 [Page 25]
Internet-Draft SNMP Traffic Measurements February 2008
March 2005.
[PDMQ04] Pras, A., Drevers, T., van de Meent, R., and D. Quartel,
"Comparing the Performance of SNMP and Web Services based
Management", IEEE electronic Transactions on Network and
Service Management 1(2), November 2004.
[Pat01] Pattinson, C., "A Study of the Behaviour of the Simple
Network Management Protocol", Proc. 12th IFIP/IEEE
Workshop on Distributed Systems: Operations and
Management , October 2001.
[DSR01] Du, X., Shayman, M., and M. Rozenblit, "Implementation and
Performance Analysis of SNMP on a TLS/TCP Base", Proc. 7th
IFIP/IEEE International Symposium on Integrated Network
Management , May 2001.
[CT04] Corrente, A. and L. Tura, "Security Performance Analysis
of SNMPv3 with Respect to SNMPv2c", Proc. 2004 IEEE/IFIP
Network Operations and Management Symposium , April 2004.
[PFGL04] Pavlou, G., Flegkas, P., Gouveris, S., and A. Liotta, "On
Management Technologies and the Potential of Web
Services", IEEE Communications Magazine 42(7), July 2004.
[SM99] Sprenkels, R. and J. Martin-Flatin, "Bulk Transfers of MIB
Data", Simple Times 7(1), March 1999.
[Mal02] Malowidzki, M., "GetBulk Worth Fixing", Simple
Times 10(1), December 2002.
[HS06] Harvan, M. and J. Schoenwaelder, "Prefix- and
Lexicographical-order-preserving IP Address
Anonymization", IEEE/IFIP Network Operations and
Management Symposium NOMS 2006, April 2006.
[XFA02] Xu, J., Fan, J., and M. Ammar, "Prefix-Preserving IP
Address Anonymization: Measurement-based Security
Evaluation and a New Cryptography-based Scheme", 10th IEEE
International Conference on Network Protocols ICNP'02,
November 2002.
[FXAM04] Fan, J., Xu, J., Ammar, M., and S. Moon, "Prefix-
Preserving IP Address Anonymization", Computer
Networks 46(2), October 2004.
[PAPL06] Pang, R., Allman, M., Paxson, V., and J. Lee, "The Devil
and Packet Trace Anonymization", Computer Communication
Schoenwaelder Expires August 15, 2008 [Page 26]
Internet-Draft SNMP Traffic Measurements February 2008
Review 36(1), January 2006.
[RW07] Ramaswamy, R. and T. Wolf, "High-Speed Prefix-Preserving
{IP} Address Anonymization for Passive Measurement
Systems", IEEE Transactions on Networking 15(1),
February 2007.
Schoenwaelder Expires August 15, 2008 [Page 27]
Internet-Draft SNMP Traffic Measurements February 2008
URIs
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
Schoenwaelder Expires August 15, 2008 [Page 28]
Internet-Draft SNMP Traffic Measurements February 2008
Author's Address
Juergen Schoenwaelder
Jacobs University Bremen
Campus Ring 1
28725 Bremen
Germany
Phone: +49 421 200-3587
Email: j.schoenwaelder@jacobs-university.de
Schoenwaelder Expires August 15, 2008 [Page 29]
Internet-Draft SNMP Traffic Measurements February 2008
Full Copyright Statement
Copyright (C) The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
Schoenwaelder Expires August 15, 2008 [Page 30]