Internet-Draft ISTN Methodology Problems and Requests October 2021
Lai, et al. Expires 28 April 2022 [Page]
Benchmarking Methodology Working Group
Intended Status:
Z. Lai
Tsinghua University
H. Li
Tsinghua University
Y. Deng
Tsinghua University
Q. Wu
Tsinghua University
J. Liu
Tsinghua University

Problems and Requirements of Evaluation Methodology for Integrated Space and Terrestrial Networks


With the rapid evolution of the aerospace industry, many "NewSpace" upstarts are actively deploying their mega-constellations in low Earth orbits (LEO) and building integrated space and terrestrial networks (ISTN), promising to provide pervasive, low-latency, and high-throughput Internet service globally. Due to the high manufacturing, launching, and updating cost of LEO mega-constellations, it is expected that ISTNs can be well designed and evaluated before the launch of satellites. However, the progress of designing, assessing, and understanding new network functionalities and protocols for futuristic ISTNs faces a substantial obstacle: lack of standardized evaluation methodology with acceptable realism, flexibility, and cost that can involve the unique dynamic behaviors of ISTNs. This memo first reviews the unique characteristics of LEO mega-constellations. Further, it analyzes the limitation of existing network evaluation and analysis methodologies under ISTN environments. Finally, it outlines the key requirements of future evaluation methodology tailored for ISTNs.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 28 April 2022.

Table of Contents

1. Introduction

Integrated Space and Terrestrial Networks (ISTN), combining diverse spacecraft and ground infrastructures, are extending the frontier of today's terrestrial network, promising to provide low-latency, high-bandwidth Internet access with broader coverage globally.

Low Earth Orbit (LEO) satellites are the key building block for constructing ISTN. Recently, we have witnessed a renaissance in the space industry, stimulating an exponential increase in constructing mega-constellations. As compared to their predecessor, cutting-edge satellites can be equipped with high-resolution sensors, space-grade multi-core processors, high-data-rate communication links, and multifunctional space software.

While ISTNs hold great promise, to completely unleash the network potential of emerging ISTN, it still needs to address a series of new research issues. The unique characteristics of LEO satellites (e.g., high-dynamics), not only impose new challenges at various layers of the ISTN networking stack but also open the door to many new research problems. With many unexplored problems facing the "NewSpace" industry, it is thus foreseen that in the near future, there will be a surge of new research (e.g. topology, addressing, routing, transport, etc.) to rethink and reshape network functionalities and protocols in ISTNs. In addition, the cost/ timeline of manufacturing, launching, operating, and updating satellite constellations is typically much higher/longer than that in traditional terrestrial networks. Therefore, it is expected that new network functionalities and protocols can be well evaluated before they are launched and deployed in realistic satellite constellations.

However, the network community lacks the proper analysis tools and evaluation methodologies that can mimic the unique dynamic behavior to analyze many of the ISTN challenges that have been highlighted by prior studies. At high level, existing evaluation methodologies in the network community can typically be grouped into three major categories: live networks or platforms, simulation, and emulation. However, the feasibility and flexibility of live satellite networks are technically and economically limited. The abstraction level of network simulation might be too high to capture low-level system effects. Existing network emulators fail to characterize the high dynamicity of LEO satellites and thus cannot accomplish an environment with acceptable fidelity. The community hence needs a reasonable and standardized evaluation methodology to build proper experimental environments which can mimic the behavior of ISTNs, supporting the community to deeply understand the problems, and to evaluate new functionalities and protocols (e.g. for topology, addressing, routing, transport, etc.) for ISTNs, before the mega-constellation is completely deployed. In this memo, we first review the unique characteristics of emerging LEO mega-constellations and the key challenges of integrating satellites and terrestrial Internet. Further, we analyze the limitation of existing network analysis tools and evaluation methodologies in ISTNs. Finally, we outline key requirements of evaluation methodologies tailored for ISTNs.

2. Notation and Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

This document uses the following acronyms and terminologies:

Mega-constellation: A group of satellites working as a system.

LEO: Low Earth Orbit with an altitude no more than 2000 km.

MEO: Medium Earth Orbit with an altitude from 2000 km to 35786 km.

GEO: Geostationary Earth Orbit with an altitude of 35786 km.

NGSO: Non-Geostationary Orbit

LSN: LEO Satellite Networks

ISTN: Integrated Satellite and Terrestrial Network

ISL: Inter-satellite Links

EO: Earth Observation

GS: Ground Station

AS: Autonomous System

EOS: Earth Observation Satellite

BGP: Border Gateway Protocol [RFC4271]

OSPF: Open Shortest Path First [RFC2328]

VM: Virtual Machine

3. Quick Primer for Integrated Space and Terrestrial Networks

Emerging mega-constellations with inter-satellite links (ISLs) can build a satellite network in outer space, and further be integrated with terrestrial ground infrastructures to construct an integrated space and terrestrial network (ISTN).

3.1. Mega-constellation

A constellation is a group of satellites working as a system to give a coverage of the Earth surface, among which satellites are positioned in fixed orbital planes with regular trajectories. LEO and MEO satellites often belong to a constellation, because a single satellite only covers a small area with high angular velocity. Thus, continuous coverage over an area could be maintained by the relay within a constellation, as compared with GEO satellites that only provides a permanent coverage over a target area. Walker Delta constellation is the most common formation for constellations. It is defined as a bunch of circular orbits with a fixed inclination, satellite number, number of equally spaced planes and the relative spacing between satellites in adjacent planes. The famous Ballard rosette constellation is another name of Walker Delta constellation, where it uses a different notation. Near-polar Walker Star is one of this kind, initially used by Iridium [Iridium]. Constellations with a higher inclination give the polar regions more chances to get accessed. The well-known emerging commercial constellations are Starlink [Starlink-Fcc], Kuiper [Kuiper-Fcc] and Telesat [Telesat-Fcc], as shown in Table 1 below. And all of them contain more than one shell.

Table 1: Mega-constellation information.
Name and Shell Altitude (km) Inclination (degree) # of orbits # of satellites per orbit
Starlink S1 550 53 72 22
Starlink S2 1110 53.8 32 50
Starlink S3 1130 74 8 50
Starlink S4 1275 81 5 75
Starlink S5 1325 70 6 75
Kuiper K1 630 51.9 34 34
Kuiper K2 610 42 36 36
Kuiper K3 590 33 28 28
Telesat T1 1015 98.98 27 13
Telesat T2 1325 50.88 40 33

3.2. Topological Dynamics

Unlike geostationary satellite networks or terrestrial core infrastructure that keep a stable topology, LEO satellite networks suffer from high topological dynamics, since LEO satellites move fast, causing short-lived coverage for fixed terrestrial users. For example, considering the first shell of Starlink Phase-I, a fixed user sees each satellite for only up to 3 minutes in one pass, after which the satellite moves away from the user's perspective. Table 2 shows the medium space-ground link churn intervals [link-churn-interval] between existing GS and constellations. If each GS only uses one antenna to connect the satellite with the shortest distance, the medium interval is no more than one minute. This kind of high dynamic motion incurs frequent link changes between LEO satellites and GS or users, thus causing frequent topology changes. Moreover, inter-satellites visibility may also change if LEO satellites move in different directions or in different shells, resulting in connectivity change of ISLs.

Such high LEO dynamics can impose significant challenges in the networking stack of ISTNs. The high dynamics make the logical network and mega-constellations and physical ISTN inconsistent. One big challenge is how to overcome the routing oscillation properly in the high dynamic ISTN environment. Frequent satellite-GS link changes make the inter-connectivity of space and ground segments in ISTNs unstable. Thus, the routing have to be re-calculated every time the link changes. In addition, the topological dynamics also result in RTT fluctuations in end-to-end paths, involving new challenges for congestion control in ISTNs, as a RTT variation observed by end-host might not indicate congestions.

3.3. Long Manufacturing and Deployment Duration

Different from terrestrial network infrastructures, the timeline of manufacturing and deploying satellite networks could be much longer due to the high cost and complex process during the development and launch period. Satellites, as well as the orbit and spectrum they used, have to be regulated, and launches have to be carefully scheduled (e.g. to avoid the impact of poor weather conditions). In addition, the maintenance and update cost of a satellite network is also typically much higher than that of a terrestrial network.

For example, a review of 24 Air Force and Navy space vehicle (SV) development programs found that on average it took about 7.5 years from contract start to launch a government satellite. [Development-Timeline] Commercial satellite programs typically take 2 to 3 years from contract start to launch. [Production-Cycles] SpaceX's Starlink constellation plan to launch about 42,000 satellites to construct a mega-constellation in outer space. On 15 October 2019, the United States Federal Communications Commission (FCC) submitted filings to the International Telecommunication Union (ITU) on SpaceX's behalf to arrange spectrum for 30,000 additional Starlink satellites to supplement the 12,000 Starlink satellites already approved by the FCC. As of the date of September 2021, two years after the first launch in May 2019, SpaceX has launched about 1791 Starlink satellites, which is about 4% of the ultimate constellation plan consisting of 42,000 satellites. Foreseeably, it may take many years to complete the entire constellation deployment. Even the first phase of Starlink which consists of about 4400 satellites is not expected to be completed until 2024.

4. Problem Statement: We Need the Right Evaluation Methodology

The unique characteristics of LEO mega-constellations involve new challenges on various layers of the networking stack of ISTNs. On one hand, it is foreseen that in the near future, there will be a surge of new network functionalities and protocols designed or optimized for ISTNs. On the other hand, because the cost/timeline of manufacturing, launching, operating, and updating satellite constellations is typically much higher/longer than that in traditional terrestrial networks, it is expected that those new network functionalities and protocols tailored for ISTNs should be well evaluated before they are launched and deployed in realistic satellite constellations.

Existing methodologies for testing, assessing, and understanding a network functionality or protocol can typically be classified into three categories: (1) live networks; (2) network simulators; and (3) network emulators. The subsections discuss these three categories of network evaluation methodologies, along with their using deficiencies and possible remedies respectively.

4.1. Live networks and platforms

Representative platforms such as Emulab [Emulab] and Sparta [Sparta] are successful pioneers that build a large-scale experimental network environment. These test environments are originally designed to provide special and exclusive test services for affiliated universities, scientific research institutions or Internet business companies. And for the resource competition, each independent experiment needs to completely monopolize a part of the test bed, so the researcher cannot deploy the experiment until being allocated with enough nodes. PlanetLab [PlanetLab] is truly global ground testbed prototype. Started from 2003, it consists of 1353 nodes at 717 sites spanning 48 countries. Together the nodes form a global network system to support new design of network services.

The live platforms described above were initially proposed for terrestrial networks and they are developed and repaired at the same time. The key limitation of them in ISTN environment is that they are designed for terrestrial network experiments, and do not incorporate the realistic characteristic of LEO mega-constellations to support experiments and evaluations in ISTNs.

We may search for help from live satellites, but still there is only limited help. It seems that with the help of live ISTN, researchers are capable to assess, verify and evaluate their ideas and thoughts. Live ISTN can give a real constellation-consistency and stack-consistency testing environment. However, current satellites only provide users a bent-pipe service, which is purely relaying the transmission messages, such as the current deployment of Starlink [Starlink]. The construction is far from a comprehensive ISTN, so the research scope is limited. Even if there is a live ISTN, it lacks flexibility, owning to the inconvenient control over satellites. Besides, the access to satellites is also limited.

Therefore, live networks or platforms for terrestrial networks can give us a large-scale experimental environment but they lack the support for ISTN characteristics. On the other hand, live ISTN is able to guarantee a real space environment, but it is not that affordable and flexible.

4.2. Network Simulators

Simulators are testing methodology tools that enable researchers to reproduce their testing experiments by simulating a real-world process or system over time. Simulators work by using discrete event simulation to calculate the interactive states among all the network entities, ranging from switches, routers, nodes, access points, links and so on. While working fast and efficiently, the fidelity is only brought by the state variable changes at discrete points.

Such tools like Systems Tool Kit (STK) [Systems-Tool-Kit] and General Mission Analysis Tool (GMAT) [General-Mission-Analysis-Tool] are good for orbit analysis. STK is a powerful tool to help researchers to model the behavior of mission entities in aerospace, telecommunications and so forth. It also provides visualization and analysis functions. GMAT is a similar tool for space trajectory optimization and mission modeling. Nevertheless, both tools do not support networking simulations such as topology and protocol simulations. ns-3 [ns-3] goes a step further with support for Internet simulation, but on the contrary, it was not designed for ISTN and lacks the support for high-dynamics of ISTN. StarPerf [StarPerf] is a simulator that helps researchers to study network performance under a range of constellation conditions. But still, it lacks the ability to support interactive network traffic simulation and system codes in the systems.

Overall, while flexible and low-cost, the realism of simulators is not content enough, because they are too abstract to realize the low-level characteristics. In other words, simulators are being too object-oriented to involve additional overhead in the actual execution of programs. Besides, when accessing the network performance, a number of recent emerging algorithms for congestion control, reliable transmission or even protocols are not supported, for example ns-3 [ns-3] only supports basic congestion control like Reno [RFC6582] and so forth, so the need to work with some new algorithms cannot be satisfied and the research to discover new mechanisms, such as new routing algorithms and re-transmission schemes, is extensively prohibited. Another problem of simulators, such as ns-3 [ns-3], is that it difficult to trace or understand the previous codes, without appropriate documentations. Simulators usually face the additional compatibility problem, which means they is not portable with other systems, or they do not support kernel codes. Since there are multiple simulators developed by different group of users, sometimes users are required to be familiar with the writing language, scripting style and modelling technique. So, the Tool Command Language might be difficult to understand and write.

4.3. Network Emulators

Emulators are another kind of paradigm for testing methodology tool over a virtual network. The difference between a simulator and an emulator is that emulators leverage VM or containers to keep the realism which is close to actual performances. Therefore, in emulators, virtual nodes. virtual network links, virtual models of traffic, and protocols are all applied. Emulators are capable to run real kernel and application code. Thus, emulators not only support diverse topology design, but also protocol emulation in a synthetic network environment. They emulate the network behavior in a more real way. Mininet [Mininet] is commonly regarded as the most illustrious emulator for networking with its strong ability to support experiments with Software-Defined Networking (SDN) [Software-defined-networking] systems. EstiNet [EstiNet] is another emulator that supports evaluating and testing the performances of software-defined networks. Based on containers, they can emulate real TCP/IP protocol stack in the Linux kernel. However, existing emulation tools lack the ability to construct the dynamic links and orbits in ISTN like simulators. Thus, more problems could happen in higher-level protocols such as routing protocols (e.g. OSPF and BGP). Besides, since emulators run containers or virtual machines which occupy more software overhead, compared with simulators, it will be hard to emulate the large-scale mega-constellations. Existing work has shown the capability of 25 physical machines working together as a system to emulate 250 network nodes, but still, it is far less for ISTN scalability.

To conclude, emulators are relatively good methodologies for network experiments, but emulators still have limitations when using them for ISTN research. While keeping a moderate realism by using VM or containers for entity emulation and flexibility, emulators still lack the supports for ISTN characteristics, such as frequent link changes, satellite network topology uncertainty, and so on. More specifically, current emulators only support fixed network topology emulation. It is not flexible to emulate the time-varying link packet loss, bandwidth, and other traits. A possible way is to frequently replace the link with a new one from time to time sequentially for the entire ISTN. However, it is far from the real situation. Besides, VM or containers are able to deploy a range of network nodes in a physical server, but the actual CPU, memory and other resources should not be shared in reality for each satellite. In addition, it is still difficult to emulate thousands or ten thousand of satellites for ISTN even with VM or containers, subject to hardware limitations. For flexibility, some emulators do not support a good network animator tool. Especially in ISTN emulation, GUI is important for users to observe and analyze orbit trajectories and real time satellite positions.

4.4. Summary

In this section, we explain the necessity of an evaluation methodology specifically for ISTN. Then we demonstrate the problems with existing methodologies related to ISTN. The performance comparison result is shown in Table 3. Above all, ISTN should be designed first and then launched. Live satellites enable good realism but they lack flexibility and require very high cost as well as a very long deployment period. Other testing tools such as simulators and emulators are either functional for merely aerospace analysis or simply terrestrial networks. None of the existing methodologies guarantees a practical and user-friendly methodology while keeping the evaluation environment realism with low costs.

Table 3: Existing platforms/tools for network analysis and evaluation. (Y for yes/N for no)
Platform/Tool Realism Flexibility Cost Cross-domain Dataset Support
Live satellite network Y N High Y
ns-3 [ns-3] N Y Low N
Hypatia [Hypatia] N Y Low N
StarPerf [StarPerf] N Y Low N
Mininet [Mininet] N Y Low N

5. Requirements: New Evaluation Methodology Tailored for ISTNs

A proper evaluation methodology tailored for ISTNs is expected to help developers, researchers, engineers to explore various design-space of the networking stack of ISTNs in a technically and economically feasible manner. Based on the comparative analysis results in the prior section, we sum up the following requirements for the new evaluation methodology in ISTNs.

5.1. Realism

The first requirement is realism. Realism represents the testing authenticity and fidelity, compared with real ISTN. It could be further divided into constellation-consistency and networking stack realism. Constellation-consistency requires the testing to keep the actual characteristics of mega-constellations both spatially and temporally. In other words, the orbit-level information including the latitude, longitude, and height of each satellite in any given time and the same information for GS and elevation angles of antennas of each GS spatially. Note that the spatial information also determines the visibility, links and even topology of ISTN. Since the mega-constellations are unstable, how the temporal satellite locations, visibility, link propagation delays and so on should also be considered carefully. Similarly, the networking stack realism requires the network nodes to communicate and negotiate their messages following the actual protocol process. For example, when doing a test for OSPF in an ISTN, we would like the nodes to send Hello packets, Link-State-Request (LSR) packets, Link-State-Update (LSU) packets and so on. A real network stack is preferred to provide researchers an opportunity to see the performance of different protocols in ISTN.

5.2. Flexibility

Another requirement is flexibility and feasibility. The testing methodology should be technically easy to use and easy to learn. Without extra modifications or process, the methodology should help researchers learn and use it without much effort and can evaluate their ideas as they wish, which means it should support multiple environments for researchers.

5.3. Low-cost and Easy-to-use

Meanwhile, the evaluation methodology is expected to be low-cost. A well-acceptable methodology should be economically feasible for users to create an experimental network environment. Researchers do not want to conduct their tests all in live ISTN, which is over- cumbersome and unaffordable, let alone launching their own spacecraft. Even if there are a number of orbiting satellites, whether users can easily gain access to satellites is also a problem.

5.4. Cross-domain Dataset Support

The evaluation methodology is expected to be driven by realistic datasets from multi-dimensions to support its realism. Multi-dimension refers to multi-disciplinary research on ISTN. Since a standard ISTN evaluation methodology not only contains high-level benchmarks from topology, routing to transmission, but also considers the low-level traits such as wireless link conditions, weather conditions and Earth rotations. To be more concrete, the former one requires knowledge in networks while the latter one relies more on aerospace. Hence, to build a high-fidelity methodology, we need community efforts both from networks and aerospace. On the other hand, an authentic dataset is an indispensable element for data driven testing methodology. Actual data is the first step to obtain a realistic emulation. with characteristics of a real ISTN. Thus, the dataset is a collection of messages for testing, in which geographical mega-constellation information (orbit number, satellite number, height), orbital information (orbit inclination angle and link strategies), weather information as well as ground station information (positions, antenna angle and so forth) are involved.

6. Conclusion

To conclude, the emergence of mega-constellations brings us new opportunities for the development of ISTN that extends the Internet to the space era. Combined with terrestrial networks, ISTN is expected to supply pervasive, low-latency and high-speed services to users globally, which greatly enhances the current Internet. At the same time, the unique characteristics (e.g. high-dynamics) of ISTN impose challenges in topology, routing, transportation, application, and security. However, we simply believe addressing the challenges also gives us open opportunities for future research by our community-driven effort. To accelerate the research speed and to help make testing more feasible, new methodologies that satisfy user requirements should be proposed. To this extent, this draft reviews the limitation of existing network analysis tools in ISTNs, considering the unique characteristics of emerging LSNs and the key challenges. This draft further analyzes the limitation of existing evaluation methodologies in ISTN environments. Finally, this draft outlines key requirements of evaluation methodologies tailored for future ISTNs.

7. Acknowledgements

8. IANA Considerations

This memo includes no request to IANA.

9. Security Considerations

This entire draft discusses security considerations from different perspectives of ISTN in Section 3.

10. References

10.1. Normative References

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <>.
Moy, J., "OSPF Version 2", STD 54, RFC 2328, DOI 10.17487/RFC2328, , <>.
Rekhter, Y., Li, T., and S. Hares, "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, , <>.
Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 6582, DOI 10.17487/RFC6582, , <>.

10.2. Informative References

"Development-Timeline", <>.
"Emulab", <>.
"EstiNet", <>.
"General-Mission-Analysis-Tool", <>.
"Hypatia", <>.
"Iridium", <>.
"Kuiper-Fcc", <>.
"link-churn-interval", <>.
"Mininet", <>.
"ns-3", <>.
"PlanetLab", <>.
"Production-Cycles", < n_Cycles_0504.pdf>.
"Software-defined-networking", <>.
"Sparta", <>.
"Starlink", <>.
"Starlink-Fcc", <>.
"StarPerf", <>.
"Systems-Tool-Kit", <>.
"Telesat-Fcc", <>.

Authors' Addresses

Zeqi Lai
Tsinghua University
30 ShuangQing Ave
Hewu Li
Tsinghua University
30 ShuangQing Ave
Yangtao Deng
Tsinghua University
30 ShuangQing Ave
Qian Wu
Tsinghua University
30 ShuangQing Ave
Jun Liu
Tsinghua University
30 ShuangQing Ave