Guidelines for Performing Safe Measurement on the Internet
Tor Project
irl@torproject.org
safety
measurement
monitoring
research
internet
Researchers from industry and academia will often use Internet measurements
as a part of their work. While these measurements can give insight into the
functioning and usage of the Internet, they can come at the cost of user privacy.
This document describes guidelines for ensuring that such measurements can
be carried out safely.
Comments are solicited and should be addressed to the research group's mailing list at pearg@irtf.org and/or the author(s).
The sources for this draft are at:
https://github.com/irl/draft-safe-internet-measurement
Introduction
When performing research using the Internet, as opposed to an isolated testbed
or simulation platform, means that you research co-exists in a space with other
users. This document outlines guidelines for academic and industry researchers
that might use the Internet as part of scientific experiementation.
Following the guidelines contained within this document is not a substitute for
any institutional ethics review process you may have, although these guidelines
could help to inform that process. Similarly, these guidelines are not legal
advice and local laws should be considered before starting any experiment that
could have adverse impacts on user privacy.
Considerations are grouped into two categories: those that primarily apply to
active measurements and those that primarily apply to passive measurements.
Some of these considerations may be applicable to both depending on the
experiment design.
Active measurements
Active measurements generate traffic. Performance measurements such as TCP
throughput testing or functional measurements
such as the feature-dependent connectivity failure tests performed by
both fall into this category.
Use a testbed
Wherever possible, use a testbed. An isolated network means that there are no
other users sharing the infrastructure you are using for your experiments.
When measuring performance, competing traffic can have negative effects on
the performance of your test traffic and so the testbed approach can also
produce more accurate and repeatable results than experiments using the public
Internet.
WAN link conditions can be emulated through artificial delays and/or packet
loss using a tool like . Competing traffic can
also be emulated using traffic generators.
Only record your own traffic
When performing measurements be sure to only capture traffic that you have
generated. Traffic may be identified by IP ranges or by some token that is
unlikely to be used by other users.
Again, this can help to improve the accuracy and repeatability of your
experiment. , for performance benchmarking,
requires that any frames received that were not part of the test traffic are
discarded and not counted in the results.
Be respectful of other's infrastructure
If your experiment is designed to trigger a response from infrastructure that
is not your own, consider what the negative consequences of that may be. At the
very least your experiment will consume bandwidth that may have to be paid for.
In more extreme circumstances, you could cause traffic to be generated that
causes legal trouble for the owner of that infrastructure. The Internet is a
global network crossing many legal jurisdictions and so what may be legal for
you is not necessarily legal for everyone.
If you are sending a lot of traffic quickly, or otherwise generally deviate
from typical client behaviour, a network may identify this as an attack which
means that you will not be collecting results that are representative of what
a typical client would see.
Passive measurements
Performing passive measurements requires existing traffic. Passive
measurements can help to inform new developments in Internet protocols but can
also carry risk.
Consider the expectation of privacy
If you are in a position to perform passive measurements of live network
traffic, you are also in a position of responsibility. Users of a network will
have certain expectations of privacy and those expectations may not align with
the privacy guarantees offered by the technologies they are using. As a thought
experiment, consider how users might respond if you asked for their informed
consent for the measurements you'd like to perform.
Only collect data that is safe to make public
When deciding on the data to collect, assume that any data collected might
become public. There are many ways that this could happen, through operation
security mistakes or compulsion by a judicial system.
Minimization
For all data collected, consider whether or not it is really needed.
Aggregation
When collecting data, consider if the granularity can be limited by using
bins or adding noise. XXX: Differential privacy.
Source Aggregation
Do this at the source, definitely do it before you write to disk.
presents a case-study on the in-memory
statistics in the software used by the Tor network, as an example.
Risk Analysis
The benefits should outweigh the risks.
Consider auxiliary data (e.g. third-party data sets) when assessing the risks.
Security Considerations
Take reasonable security precautions, e.g. about who has access to your data sets or experimental systems.
IANA Considerations
This document has no actions for IANA.
Acknowledgements
Many of these considerations are based on those from the
adapted and generalised to be applied to
Internet research.
Informative References
Framework for TCP Throughput Testing
This framework describes a practical methodology for measuring end- to-end TCP Throughput in a managed IP network. The goal is to provide a better indication in regard to user experience. In this framework, TCP and IP parameters are specified to optimize TCP Throughput. This document is not an Internet Standards Track specification; it is published for informational purposes.
PATHspider: A tool for active measurement of path transparency
University of Aberdeen
ETH Zurich
ETH Zurich
University of Aberdeen
Network emulation with NetEm
Open Source Development Lab
Benchmarking Methodology for Network Interconnect Devices
This document is a republication of RFC 1944 correcting the values for the IP addresses which were assigned to be used as the default addresses for networking test equipment. This memo provides information for the Internet community.
Tor Research Safety Board
Tor Project
Privacy analysis of Tor's in-memory statistics
Tor Project