ALTO WG D. Lachos Internet-Draft C. Rothenberg Intended status: Informational Unicamp Expires: September 10, 2020 Q. Xiang Y. Yang Yale University B. Ohlman Ericsson Research S. Randriamasy Nokia Bell Labs F. Boten Sprint LM. Contreras Telefonica J. Zhang Tongji University K. Gao Sichuan University March 9, 2020 Supporting Multi-domain Use Cases with ALTO draft-lachos-multi-domain-use-cases-alto-00 Abstract The goal of this document is to summarize current standardization efforts in the IETF ALTO working group to support important multi- domain use cases and show how they can benefit from network information exposure using ALTO. Besides, key design requirements of network information exposure to support multi-domain use cases are also presented along with information about novel mechanisms and abstractions to improve the base ALTO framework in multi-domain scenarios. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any Lachos, et al. Expires September 10, 2020 [Page 1] Internet-Draft Multi-domain use cases with ALTO March 2020 time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on September 10, 2020. Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Network Information Exposure: Current ALTO Context . . . . . 4 3. Multi-domain Use Cases . . . . . . . . . . . . . . . . . . . 5 3.1. Multi-domain, collaborative data sciences . . . . . . . . 5 3.1.1. How can multi-domain resource orchestration benefit from ALTO? . . . . . . . . . . . . . . . . . . . . . 6 3.1.2. Example . . . . . . . . . . . . . . . . . . . . . . . 6 3.2. Multi-domain SFC . . . . . . . . . . . . . . . . . . . . 7 3.2.1. How can multi-domain SFC benefit from ALTO? . . . . . 8 3.2.2. Example . . . . . . . . . . . . . . . . . . . . . . . 8 3.3. Multi-domain SDN . . . . . . . . . . . . . . . . . . . . 10 3.3.1. How can flexible interdomain routing benefit from ALTO? . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3.2. Example . . . . . . . . . . . . . . . . . . . . . . . 11 4. Requirements on Multi-domain Network Information Exposure . . 11 4.1. Design Requirements . . . . . . . . . . . . . . . . . . . 11 4.2. Existing Efforts in the ALTO Working Group . . . . . . . 12 5. Novel Multi-domain Mechanisms and Abstractions . . . . . . . 13 5.1. Multi-domain Aggregation . . . . . . . . . . . . . . . . 13 5.1.1. Workflow . . . . . . . . . . . . . . . . . . . . . . 14 5.2. Multi-resource Abstraction . . . . . . . . . . . . . . . 15 5.2.1. Mathematical Programming Constraints: Basic Idea . . 15 5.2.2. Removing Redundant Linear Inequalities . . . . . . . 16 5.2.3. From Single Domain to Multiple Domains . . . . . . . 16 5.3. Multi-domain Programming Information Abstraction . . . . 19 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 Lachos, et al. Expires September 10, 2020 [Page 2] Internet-Draft Multi-domain use cases with ALTO March 2020 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.1. Normative References . . . . . . . . . . . . . . . . . . 20 8.2. Informative References . . . . . . . . . . . . . . . . . 20 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 1. Introduction Many multi-domain use cases are emerging with the development of new technologies, such as software-defined networking (SDN), network function virtualization (NFV), and 5G. Examples of such use cases include multi-domain, collaborative data sciences [CMS][LCLS][LHC][SKA], multi-domain service function chaining (SFC) [NGMN-5G][SFC-MD][MD-ORCH-NFV][ETSI-ZSM], and multi- domain SDN [SFP][SDX][RFC5575]. Such use cases can benefit substantially from the exposure of network information, with which users can perform application-layer resource optimization to improve the performance. The Application-Layer Optimization Protocol (ALTO) [RFC7285] already introduces basic mechanisms (e.g., modularity, dependency) and abstractions (e.g., map services) for applications to take optimized actions based on network information. However, exposing network information to support multi-domain use cases places additional requirements that existing solutions such as the current ALTO design do not satisfy. First, abstractions that aggregate multiple networks into a single, virtual network are required to simplify the application-layer optimization conducted by end-users. Second, such abstractions need to provide a unified representation of multiple resources (e.g., networking, computation, and storage) in multiple networks. This document reviews the current ALTO architecture to expose network information for applications (Section 2), and it then presents several important multi-domain use cases that can benefit substantially from network information exposure (Section 3). Next, it elaborates the key design requirements of network information exposure to support these use cases (Section 4), followed by proposed extensions in the ALTO working group to satisfy the design requirements [MD-E2E-NS][UNIF-REPR] [MD-USE-CASES][BROKER-MDO][MD-ANALYTICS][MD-SFC-ALTO]. Finally, it summarizes novel mechanisms and abstractions based on recent research to improve the ALTO framework in the multi-domain set- up [MERCATOR][BOXOPT][SDI] (Section 5). Lachos, et al. Expires September 10, 2020 [Page 3] Internet-Draft Multi-domain use cases with ALTO March 2020 2. Network Information Exposure: Current ALTO Context ALTO already provides a generic framework to expose network information for applications to improve their performance. Figure 1 presents a high-level overview of key ALTO mechanisms and abstractions. In particular, ALTO introduces generic mechanisms such as: o Modularity and flexibility through an explicit division of ALTO network information into (network) information resources. o An information resource directory (IRD) providing a list of available information resources in an ALTO server. o Information consistency (tag, dependency, multi-info resources [ALTO-MULTIPART]) to specify a dependency among different information resources. o A generic framework with Server-Sent Events (SSEs) [ALTO-SSE] to perform stream-control, push, incremental update of information resources. ALTO also introduces abstractions modules, such as: o Network and cost maps to provide network location groupings and costs between them. o A multi-cost map [RFC8189] to retrieve several cost metrics in a single query/response transaction. o The path vector abstraction [ALTO-PATH] to provide more detailed routing information using Abstract Network Elements (ANEs). o Capability maps (e.g., CDNI [ALTO-CDNI] and unified property Map [ALTO-PROP]). Another generic concept introduced is filter so that information resources can be filtered (e.g., Filtered network map, filtered cost map). Besides, each individual information resource is provided as a RESTful service with a very simple, but well-working grammar (essentially JSON grammar [RFC7159]). Server discovery [RFC7286] and cross-domain server discovery [ALTO-XDOM] are also introduced in order to identify a topologically nearby ALTO server or ALTO servers outside of a network domain, respectively. Lachos, et al. Expires September 10, 2020 [Page 4] Internet-Draft Multi-domain use cases with ALTO March 2020 ALTO SERVER +--------------------------------------------------------------------+ | | | ABSTRACTIONS | | +----------------------------------------------------------------+ | | | | | | |+-------------+ +-------------+ +-------------+ +-------------+ | | | || | | | | | | Capability | | | | || Network | | Cost Map | | Path | | Footprints/ | | | | || Map | | (Calendar, | | Vector | | Unified | | | | || | | ...) | | | | Properties | | | | |+-------------+ +-------------+ +-------------+ +-------------+ | | | +----------------------------------------------------------------+ | | | | MECHANISMS | | +----------------------------------------------------------------+ | | | | | | | +------------------+ +------------------+ +------------------+ | | | | | Information | | Information | | Information | | | | | | Resource | | Consistency | | Update Model | | | | | | Directory | | (Tag,Dependency) | |(SSE,Incremental) | | | | | +------------------+ +------------------+ +------------------+ | | | +----------------------------------------------------------------+ | +----------------------------------^---------------------------------+ | | | +------------------------------v-----------------------------+ | Infrastructure | +------------------------------------------------------------+ Figure 1: High Level ALTO Architecture. 3. Multi-domain Use Cases Multi-domain network information exposure can be beneficial in supporting multiple important use cases, including multi-domain, collaborative data sciences, multi-domain SFC, and multi-domain SDN. 3.1. Multi-domain, collaborative data sciences Many of today's premier science experiments, such as the Large Hadron Collider (LHC) [LHC] and the Square Kilometre Array (SKA) [SKA], rely on finely-tuned workflows that coordinate geographically distributed resources (e.g., instrument, compute, storage) to enable scientific discoveries. One example is the movement of LHC data from Tier 0 (i.e., the data center at the European Organization for Nuclear Lachos, et al. Expires September 10, 2020 [Page 5] Internet-Draft Multi-domain use cases with ALTO March 2020 Research, known as CERN) to Tier 1 (i.e., national laboratories) storage sites around the world. Another example is that the Fermilab is experimenting with moving the exascale LHC workflow to Amazon EC2 for more computation power [HEPCLOUD]. The key to supporting these distributed workflows is the ability to orchestrate multiple resources across multiple network domains to facilitate predictable workflow performance (e.g., available bandwidth, packet loss rate [ALTO-METRICS]). As such, multi-domain network information exposure is a cornerstone to enable this ability. 3.1.1. How can multi-domain resource orchestration benefit from ALTO? One key design challenge for multi-domain resource orchestration is its resource information model. Existing design options such as resource graph and ClassAds are inadequate because they cannot simultaneously (1) allow member networks to provide accurate information on different types of resource, (2) avoid the exposure of private information of member networks such as topology, and (3) allow data analytics jobs to describe their requirements of different types of resources accurately. In contrast, ALTO is well suited for providing a generic representation that (1) allows different types of data analytics jobs to accurately describe their resource requirements and (2) allows member networks to provide accurate information on different types of resources they own and at the same time maintain their privacies. 3.1.2. Example Consider an example of three member networks in Figure 2, where s1 and s2 are storage endpoints, and d1 and d2 are computation endpoints. Assume a data analytics job is composed of two parallel tasks T1 and T2. T1 needs dataset X as input, and T2 needs dataset Y as input. Lachos, et al. Expires September 10, 2020 [Page 6] Internet-Draft Multi-domain use cases with ALTO March 2020 .------------. | Network B | .-------------. ingB| | | Network A o--------|o-----d1 | | /| '------------' | s1\ / | | o--o | .------------. | s2/ \ | | Network C | | \| ingC| | | o--------|o-----d2 | '-------------' '------------' Figure 2: Multi-domain resource orchestration. Using the ALTO endpoint property service, an ALTO client in the resource orchestrator can discover that d1 satisfies the computing requirements of T1, and d2 satisfies the computing requirements of T2. Hence there are only two candidate endpoint pairs: (s1, d1) and (s2, d2). Afterward, using the ALTO path vector extension, the ALTO client can retrieve the bandwidth sharing information of task T1 and T2, denoted as x1 and x2, respectively, as follows: A: x1 + x2 <= 10Mbps B: x1 <= 3Mbps C: x2 <= 3Mbps With such information, the resource orchestrator can make the optimal resource orchestration decision to reserve 3 Mbps bandwidth for task T1, and 3 Mbps bandwidth for task T2. 3.2. Multi-domain SFC This use case refers to building end-to-end services by composing multiple service functions (SFs) in an abstract sequence across multiple network domains [SFC-MD]. It is identified as an important value-added service in 5G [MD-ORCH-NFV][ETSI-ZSM]. Exposing multi- domain network and resource information (e.g., link bandwidth, CPU utilization) can substantially improve the efficiency of constructing and managing such SFCs. Lachos, et al. Expires September 10, 2020 [Page 7] Internet-Draft Multi-domain use cases with ALTO March 2020 3.2.1. How can multi-domain SFC benefit from ALTO? A "dialogue" between potential domains that provide multi-domain SFC could be beneficial for more efficient use of resources and increasing the SFC performance. However, constrained knowledge of the network services and underlying network topology based only on localized views from the point of view of a single domain limits the potential and scope for multi-domain SFC. ALTO (and customized ALTO extensions) can be used to offer aggregated/abstracted views on various types of information, including domain-level topology, storage resources, computation resources, networking resources, and SF capabilities. This generic representation contributes to a more simple and scalable solution for resource and service discovery in multi-domain, multi-technology environments. 3.2.2. Example Figure 3 shows a SFC eXchange Platform (SXP), connecting three different domains (AS1, AS2, AS3). A SXP is a logical entity to make possible the negotiation between different domains, and it could be deployed, for example, in future Software-defined IXP (as a trusted third-party platform) [HH-MDSFC]. In this scenario, each domain provides different SFs: AS1 = {SF1}; AS2 = {SF2, SF3}; and AS3 = {SF3}. The SXP also includes an ALTO server component to provide abstract topology, resource, and service information for the high-level control plane in each domain [MD-SFC-ALTO]. Lachos, et al. Expires September 10, 2020 [Page 8] Internet-Draft Multi-domain use cases with ALTO March 2020 SFC eXchange Platform +--------------------+ | +--------+ | | | ALTO | | | | Server | | | +--------+ | +----> <-----+ | +----------^---------+ | | | | | | | | | | +----v---+ +-----v--+ +------v-+ |Control | |Control | |Control | |Plane | |Plane | |Plane | +--------+ +--------+ +--------+ | | | | | | | | | +--------+ +--------+ +--------+ |Data | |Data | |Data | |Plane -------Plane -------Plane | +--------+ +--------+ +--------+ SF1 SF2 SF3 SF3 [----AS1----][-----AS2-----][-----AS3-----] Figure 3: Multi-domain SFC Architecture with ALTO. The ALTO Property Map Service [DRAFT-ALTO-PM] can provide a clear global view of the resource information offered by other domains. This information allows discovering which candidate domains may be contacted to deliver the remaining requirements of a requested end- to-end service deployment. In our example, the Property Map (see Table 1) includes a property value grouped by AS. This value contains the supported SFs. Additional properties could be considered, such as resource availability (e.g., CPUs, Memory, and Storage), orchestrator entry points, etc. Lachos, et al. Expires September 10, 2020 [Page 9] Internet-Draft Multi-domain use cases with ALTO March 2020 +-----+--------------+-------------+-----+-----+---------+-----+ | | Capabilities | Entry Point | CPU | MEM | Storage | ... | +-----+--------------+-------------+-----+-----+---------+-----+ | AS1 | {SF1} | http://... | ... | ... | ... | ... | | AS2 | {SF2, SF3} | http://... | ... | ... | ... | ... | | AS3 | {SF3} | http://... | ... | ... | ... | ... | +-----+--------------+-------------+-----+-----+---------+-----+ Table 1: ALTO Property Map Once the candidate domains are discovered, it is necessary to compute multi-domain SF paths to select the SF location from those different candidate domains. The connectivity information among discovered domains can be retrieved by an ALTO Cost Map service. In our example, the Cost Map defines a path vector as an array of ASes, representing the AS-level topological distance for a given SFC request. Table 2 below shows a brief example of a service request and its multi-domain SF path response containing a list of potential domains to be traversed to deliver such service. +---------------+---------------------------------------+ | SFC Request | Multi-domain Service Function Path(s) | +---------------+---------------------------------------+ | SF1->SF2->SF3 | 1:{AS1:SF1->AS2:SF2->AS2:SF3} | | | 2:{AS1:SF1->AS2:SF2->AS3:SF3} | +---------------+---------------------------------------+ Table 2: ALTO Cost Map 3.3. Multi-domain SDN Network providers are expanding the fine-grained capability of SDN from intradomain set-up to multi-domain settings to provide flexible interdomain routing as a valuable service [SFP][SDX][RFC5575]. Users of this service can specify routing actions at the provider network based on flexible matching conditions of flow parameters such as TCP/ IP 5-tuple. This service requires provider networks to expose their available routing information to users. However, handling routing information of each network individually is too complex for users. As such, a multi-domain network exposure solution that aggregates information of multiple networks into a single abstraction can simplify the use of this service. 3.3.1. How can flexible interdomain routing benefit from ALTO? ALTO provides provider ASes a standardized approach to expose its routing capability to client ASes. Traditional interdomain routing protocols such as BGP are not good options because they only expose Lachos, et al. Expires September 10, 2020 [Page 10] Internet-Draft Multi-domain use cases with ALTO March 2020 the currently used routes, limiting client ASes' choices to specify flexible routes. In contrast, ALTO and its extensions provide interfaces for provider ASes to expose not only currently used routes, but also available yet unused routes, to client ASes so that they can have the flexibility to specify different routes for different data traffic. 3.3.2. Example Consider the example in Figure 4. AS A is compromised and being used to send DDoS traffic to AS E. Without flexible interdomain routing, AS E can setup a firewall locally, but normal traffic from B to E will still be congested at C-D-E due to the existence of malicious traffic from A to E. If AS C provides flexible interdomain routing service, AS E can specify such a firewall at AS C to block DDoS traffic from A, and at the same time avoid the congestion of normal traffic from B to E. +-----+ DDoS traffic | AS A|_ | | \ +--+---+ +---+--+ +------+ +-----+ \ | | | | | | \_| AS C +---------+ AS D |--------| AS E | +-----+ / | | | | | | | AS B|__/ +------+ +------+ +------+ | | +-----+ Normal traffic Figure 4: Flexible interdomain routing for DDoS mitigation. 4. Requirements on Multi-domain Network Information Exposure Supporting previous use cases with multi-domain network information exposure places new, key requirements, which are not fully satisfied by existing exposure solutions. This section discusses these requirements and briefly review existing efforts in the ALTO working group aiming to satisfy them. 4.1. Design Requirements o Unified Resource Capability Representation. Modern use cases require information on properties and capabilities of diverse in- network resources, including transport resources (e.g., available bandwidth), processing resources (e.g., SFs), and storage resources. These use cases may then conduct orchestration of multiple resources in multiple networks (e.g., RAN, transport, Lachos, et al. Expires September 10, 2020 [Page 11] Internet-Draft Multi-domain use cases with ALTO March 2020 core in 5G). As such, a unified representation of capabilities of multiple resources is key requirement for multi-domain network information exposure to support multi-domain use cases. o Multi-domain, easy-to-compose, end-to-end representation. Existing representations (e.g., ALTO network/cost maps, generic YANG models) tend to focus on a single domain. In multi-domain use cases, related information can be retrieved from multiple networks to compute end-to-end information. As such, abstractions that support aggregation of multiple networks into a single, virtual network ("one-big-network") are a key requirement. o Providing interfaces for more flexible queries. With the emerging of new networking architecture (e.g., SDN and NFV) and the fine- grained resource requirement of applications (e.g., link-disjoint paths and endpoint precedence), applications need a more flexible interface to specify queries of resource information. 4.2. Existing Efforts in the ALTO Working Group Several documents have been submitted to the ALTO working group, with the aim to satisfy one or more of the design requirements discussed above. o [DRAFT-RSA][DRAFT-UNICORN-INFO] documents propose and apply the ALTO path vector extension to provide accurate networking resource information to support multi-domain resource orchestration. o [BROKER-MDO] proposes to use ALTO to support resource orchestration for multi-domain SFC, and proposes a new ALTO extension to retrieve AS path of network functions across different networks. o [DRAFT-CONTEXT] proposes to extend cost information specified in [RFC7285] by providing several possible cost values for the same cost metric where each value depends on qualitative criteria as opposed to quantitative criteria such as time. o [UNIF-REPR] makes a proposal to use mathematical programming constraint as a generic representation of multiple resources. o [DRAFT-FCS] proposes a flexible flow query extension service to allow applications to specify query entities based on flexible matching conditions (e.g., TCP/IP 5-tuple) instead of IP addresses only. o ... Lachos, et al. Expires September 10, 2020 [Page 12] Internet-Draft Multi-domain use cases with ALTO March 2020 5. Novel Multi-domain Mechanisms and Abstractions This section presents novel mechanisms and abstractions based on recent research to allow the ALTO framework supporting the important multi-domain settings. 5.1. Multi-domain Aggregation Figure 5 shows a generic aggregation framework of using ALTO in multi-domain use cases [BROKER-MDO][MERCATOR][BOXOPT]. It includes a multi-domain aggregation mechanism on top of the existing single domain architecture. This new mechanism aggregates network information from ALTO servers in multiple networks to provide a single, consistent, updated, "virtual" domain abstraction. Network maps, cost maps, unified entity properties, network capabilities, and routing path abstractions (path vectors) of individual networks are consistently integrated to provide the abstraction of a single, coherent network to the users, satisfying the multi-domain aggregation requirement. Lachos, et al. Expires September 10, 2020 [Page 13] Internet-Draft Multi-domain use cases with ALTO March 2020 +-------------+ | Application | +------|------+ |(1) | +---------------+ (2) +-------v-------+ (3)| <-----+ Multi-domain | xxxxx> AGGREGATOR | | Orchestrator | x | +-----> | x +-----------^---^ (5) +-------|-------+ x x x -/ |(6) \-- x x x -/ | \-- x x x -/ | \- x x xxxxxxxxxxxxxxxxxxxxxxxxxx \-- x x -/ | x \-- x(4) x -/ | x \-- +----v---+ +------+ x/ +--------+ +---v--+ +v-------+ +-\>---+ | ALTO | | Dom Server | | Orch | | Server | | Orch | +--------+ +------+ +--------+ +------+ +--------+ +------+ +-----------------+ +-----------------+ +-----------------+ | Infrastructure | | Infrastructure | | Infrastructure | +-----------------+ +-----------------+ +-----------------+ Network 1 Network 2 Network 3 Figure 5: Generic framework of using ALTO in multi-domain use cases. 5.1.1. Workflow The basic workflow of this framework is as follows: o A user (e.g., an application) submits a network service request to the Multi-domain Orchestrator (MdO). o The MdO receives the network service and queries the aggregator to discover multi-resource network information (e.g., bandwidth, delay, and loss rate). o The aggregator determines the member networks that the network service will traverse, and queries the ALTO server in each of these member networks to discover their resource information. o Upon receiving the query from the aggregator, each ALTO server computes the resource abstraction of the corresponding member network, and send the network resource information to the aggregator. Lachos, et al. Expires September 10, 2020 [Page 14] Internet-Draft Multi-domain use cases with ALTO March 2020 o The aggregator collects the resource abstractions from the relevant member networks, and derives this aggregated, unified resource information to the MdO. o Based on the received information, the MdO determines the resource allocation for the network service, and sends a reservation request to the Domain Orchestrators (DOs). 5.2. Multi-resource Abstraction Although the existing abstractions (network/cost map, unified property, and path vector) are already powerful, they cannot handle the multi-resource information requirement. To this end, this document presents a recent, unified resource abstraction, based on mathematical programming constraints as a generic representation of the feasible resource capability of networks [MERCATOR], which users can consume via different resource management systems. 5.2.1. Mathematical Programming Constraints: Basic Idea Consider a network as shown in Figure 6. Assume that the bandwidth of every link is 100 Gbps. An application (e.g., a large data analytics system) wants to reserve two circuits from S1 to D1 and S2 to D2, respectively. Assume the application is only interested in the bandwidth of both circuits. The route of f1 (S1 -> D1) is S1 -> sw1 -> sw5 -> sw6 -> sw8 -> sw3 -> D1, and the route of f2 (S2 -> D2) is S2 -> sw2 -> sw5 -> sw6 -> sw8 -> sw4 -> D2. The routes for the two circuits share common links l3 and l4, making it infeasible for both circuits to each reserve a 100 Gbps bandwidth. +-----+ l2 l3 +-----+ l4 l5 +-----+ +--+ l1 | -----+ +----- |-----+ +----- | l6 +--+ |S1-------- SW1 | | | | SW6 | | | | SW3 -------|D1| +--+ +-----+ +-|--|+ +-----+ +-|--|+ +-----+ +--+ | | | | | SW5 | | SW8 | +-----+ +-|--|+ +-----+ +-|--|+ +-----+ +--+ l7 | | | | | | | | | | l12 +--+ |S2-------- SW2 -----+ +----- SW7 |-----+ +----- SW4 -------|D2| +--+ +-----+ l8 l9 +-----+ l10 l11 +-----+ +--+ Figure 6: Network Topology Example. The use of mathematical programming constraints is to generate a set of linear inequalities to provide a compact representation for different important properties of network resources (e.g., bandwidth, Lachos, et al. Expires September 10, 2020 [Page 15] Internet-Draft Multi-domain use cases with ALTO March 2020 delay, loss rate). In the previous example, the following set of linear inequalities are generated: f1 <= 100, for {l1, l2, l5, l6} ..... (le1) f2 <= 100, for {l7, l8, l11, l12} ... (le2) f1 + f2 <= 100, for {l3, l4} ............. (le3) which accurately captures the bandwidth sharing among two circuits' routes. 5.2.2. Removing Redundant Linear Inequalities Taking a deeper look at the set of previous linear inequalities, one can conclude that inequalities of le1 and le2 can be implicitly derived from that of le3. Thus, these inequalities are considered redundant. The problem of finding redundant linear constraints has been widely studied [PAULRAJ10COMP]. Specifically, redundant linear inequalities are removed via a polynomial-time, optimal algorithm [KARMARKAR84]. In our example, the compressed set will only contain one inequality: f1 + f2 <= 100, for {l3, l4} (le3). 5.2.3. From Single Domain to Multiple Domains To illustrate the basic aggregation abstraction from a single network to multiple networks, consider a collaboration network composed of three member networks, as shown in Figure 7. A user wants to reserve bandwidth for three circuits, from source host S to destination hosts D1 , D2, and D3. Lachos, et al. Expires September 10, 2020 [Page 16] Internet-Draft Multi-domain use cases with ALTO March 2020 Member Member Member Network 1 Network 2 Network 3 +xxxxxxxxxxxxxx+ +xxxxxxxxxxxxxxxxxxxxxxxxxxxx+ +xxxxxxxxxxxxxxxxx+ x x x x x x x +---+ x x +---+ +---+ x x +---+ x x | |100Gbps x x | | 40Gbps | | x x | | x x | --------------| ----------- --------------| ---+ x x | | x x | | | |100Gbps x x | | | x x +|--+ x x +|--+ +---+ x x +---+ |10Gbps x x | x x | x x | x x |40Gbps x x | x x +-|-+ x x | x x | +---+ x x | | x x +|--+ x x | 10Gbps | | x x +---- -----+ x x | | x x +-------------- | x x | | | | x x | | x x | | x x | +---+ | x x +|--+ x x +|--+ x x | 10Gbps| x x | x x | x x | | x x |100Gbps x x 10Gbps| x x |10Gbps | x x | x x | x x | | x +xx|xxxxxxxxxxx+ +xxxxxxxxxxxxxxxxx|xxxxxxxxxx+ +xx|xxxxxxxxxxxx|x+ | | | | +|-+ +-|+ +--+ +--+ |S | |D1| |D2| |D3| +--+ +--+ +--+ +--+ Figure 7: A collaboration network composed of three member networks. The resource abstraction captures the constraints from all networks using the set of linear inequalities, as depicted in Figure 8. Specifically, the variables f1, f2, f3 represent the available bandwidth that can be reserved for (S, D1 ), (S, D2), and (S, D3), respectively. Lachos, et al. Expires September 10, 2020 [Page 17] Internet-Draft Multi-domain use cases with ALTO March 2020 +-----------+------------------------------------+ | Network | Linear Inequalities | +-----------+------------------------------------+ | | f1 + f2 + f3 <= 100 ....... (le11) | | Member | f1 + f2 + f3 <= 40 ........ (le12) | | Network 1 | f1 + f2 + f3 <= 100 ....... (le13) | +-----------+------------------------------------+ | Member | f2 + f3 <= 40, f1 <= 10 ... (le21) | | Network 2 | f2 + f3 <= 100, f1 <= 10 .. (le21) | +-----------+------------------------------------+ | | f2 + f3 <= 10 ............. (le31) | | Member | f2 <= 10 ............. (le32) | | Network 3 | f3 <= 100 ............ (le33) | +-----------+------------------------------------+ Figure 8: Resource abstraction for the reservation request from Figure 4. Each linear inequality represents a constraint on the reservable bandwidths over different shared resources by the three circuits. For example, the inequality le11 indicates that all three circuits share a common resource and that the sum of their bandwidths can not exceed 100 Gbps. After removing the redundant inequalities of each member network, the resource abstraction of each member network is: +-----------+------------------------------------+ | Network | Linear Inequalities | +-----------+------------------------------------+ | Network | f1 + f2 + f3 <= 40 ........ (le12) | | Member 1 | | +-----------+------------------------------------+ | Network | f2 + f3 <= 40, f1 <= 10 ... (le21) | | Member 2 | | +-----------+------------------------------------+ | Network | f2 + f3 <= 10 ............. (le31) | | Member 3 | | +-----------+------------------------------------+ Figure 9: Resource abstraction of each member network after removing the redundant inequalities. Although each domain may already conduct redundancy optimization, there can be cross-domain redundancy. For example, the constraint Lachos, et al. Expires September 10, 2020 [Page 18] Internet-Draft Multi-domain use cases with ALTO March 2020 le31 at member network 3 (f2 + f3 <= 10) can eliminate those at member network 1 (f1 + f2 + f3 <= 40) and member network 2 (f2 + f3 <= 40). Using a classic compression algorithm [TELGEN83], we can remove this cross domain redundancy. Therefore, the compressed multi-domain set of linear inequalities will contain: f1 <= 10, f2 + f3 <= 10 5.3. Multi-domain Programming Information Abstraction Multi-domain SDN requires multi-domain SDN resource and programming abstraction. A novel multi-domain network programming framework is recently proposed to achieve programmable, end-to-end interdomain route control. Specifically, in this framework, each autonomus system (AS) deploys an ALTO server to provide a programming abstraction. Collectively, these ALTO servers provide the client a single, abstract, programmable network spanning multiple individual networks. Unlike existing SDN abstractions (e.g., OpenFlow and P4), it includes a built-in layer to extract and learn the interactions of interdomain policies of individual networks (e.g., route selection preference), providing a unified abstraction. Specifically, given a destination IP prefix p specified by a client, each autonomous system (AS) exposes its routing information base (RIB), i.e., all available routes it has to reach p, and the corresponding price for the client to use each route, by deploying an ALTO server. A client queries the available routes and the corresponding prices at different ASes. With the retrieved information, it then itereatively selects the best route based on its internal objective function and budget, and interacts with different ASes to check if the selected route is policy compliant, i.e., whether all ASes along this route will export the preceding segment to its upstream neighbor. After finding the best policy compliant route, the client then interacts with ASes of the route in a backward order along the route to setup the AS path. A blackbox based optimization algorithm has been developed to allow the client to swiftly find the best policy compliant route, and detailsed can be find in [SDI]. 6. IANA Considerations This document includes no request to IANA. Lachos, et al. Expires September 10, 2020 [Page 19] Internet-Draft Multi-domain use cases with ALTO March 2020 7. Security Considerations TBD. 8. References 8.1. Normative References [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., and D. McPherson, "Dissemination of Flow Specification Rules", RFC 5575, DOI 10.17487/RFC5575, August 2009, . [RFC7159] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2014, . [RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S., Previdi, S., Roome, W., Shalunov, S., and R. Woundy, "Application-Layer Traffic Optimization (ALTO) Protocol", RFC 7285, DOI 10.17487/RFC7285, September 2014, . [RFC7286] Kiesel, S., Stiemerling, M., Schwan, N., Scharf, M., and H. Song, "Application-Layer Traffic Optimization (ALTO) Server Discovery", RFC 7286, DOI 10.17487/RFC7286, November 2014, . [RFC8189] Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost Application-Layer Traffic Optimization (ALTO)", RFC 8189, DOI 10.17487/RFC8189, October 2017, . 8.2. Informative References [ALTO-CDNI] Seedorf, J., Yang, Y., Ma, K., Peterson, J., Lin, X., and J. Zhang, "Content Delivery Network Interconnection (CDNI) Request Routing: CDNI Footprint and Capabilities Advertisement using ALTO", draft-ietf-alto-cdni-request- routing-alto-08 (work in progress), November 2019. [ALTO-METRICS] WU, Q., Yang, Y., Lee, Y., Dhody, D., and S. Randriamasy, "ALTO Performance Cost Metrics", draft-ietf-alto- performance-metrics-08 (work in progress), November 2019. Lachos, et al. Expires September 10, 2020 [Page 20] Internet-Draft Multi-domain use cases with ALTO March 2020 [ALTO-MULTIPART] Zhang, J., "Multiple ALTO Resources Query Using Multipart Message", draft-zhang-alto-multipart-02 (work in progress), March 2019. [ALTO-PATH] Gao, K., Lee, Y., Randriamasy, S., Yang, Y., and J. Zhang, "ALTO Extension: Path Vector", draft-ietf-alto-path- vector-09 (work in progress), November 2019. [ALTO-PROP] Roome, W., Randriamasy, S., Yang, Y., Zhang, J., and K. Gao, "Unified Properties for the ALTO Protocol", draft- ietf-alto-unified-props-new-10 (work in progress), November 2019. [ALTO-SSE] Roome, W. and Y. Yang, "ALTO Incremental Updates Using Server-Sent Events (SSE)", draft-ietf-alto-incr-update- sse-17 (work in progress), July 2019. [ALTO-XDOM] Kiesel, S. and M. Stiemerling, "Application Layer Traffic Optimization (ALTO) Cross-Domain Server Discovery", draft- ietf-alto-xdom-disc-06 (work in progress), August 2019. [BOXOPT] Xiang, Q., Zhang, J., Wang, T., Liu, J., Guok, C., Le, F., MacAuley, J., Newman, H., and R. Yang, "Optimizing in the Dark: Learning an Optimal Solution Through a Simple Interface", Conference AAAI 2019, 2019. [BROKER-MDO] Perez, D. and C. Rothenberg, "ALTO-based Broker-assisted Multi-domain Orchestration", draft-lachosrothenberg-alto- brokermdo-02 (work in progress), March 2019. [CMS] The CMS Collaboration, "The CMS experiment at the CERN LHC", 2008, . [DRAFT-CONTEXT] Randriamasy, S., "ALTO Contextual Cost Values", draft- randriamasy-alto-cost-context-02 (work in progress), July 2017. Lachos, et al. Expires September 10, 2020 [Page 21] Internet-Draft Multi-domain use cases with ALTO March 2020 [DRAFT-FCS] Zhang, J., Gao, K., Wang, J., Xiang, Q., and Y. Yang, "ALTO Extension: Flow-based Cost Query", draft-gao-alto- fcs-06 (work in progress), July 2018. [DRAFT-RSA] Gao, K., xinwang2014@hotmail.com, x., Xiang, Q., Gu, C., Yang, Y., and G. Chen, "Compressing ALTO Path Vectors", draft-gao-alto-routing-state-abstraction-08 (work in progress), March 2018. [DRAFT-UNICORN-INFO] Xiang, Q., Newman, H., Bernstein, G., duhaizhou@gmail.com, d., Gao, K., Mughal, A., Balcas, J., Zhang, J., and Y. Yang, "Resource Orchestration for Multi-Domain Data Analytics", draft-xiang-alto-exascale-network- optimization-03 (work in progress), July 2017. [ETSI-ZSM] ETSI, "Zero Touch Network and Service Management", 2020, . [HEPCLOUD] Holzman, B., Bauerdick, L., Bockelman, B., Dykstra, D., Fisk, I., Fuess, S., Garzoglio, G., Girone, M., Gutsche, O., and D. Hufnagel, "Hepcloud, a new paradigm for hep facilities: Cms amazon web services investigation", Journal Computing and Software for Big Science, Volume 1, Number 1, Pages 1, Publisher Springer, 2017. [HH-MDSFC] Li, G., Li, G., Xu, Q., Zhou, H., Feng, B., Feng, X., and Z. Yu, "Hybrid Hierarchical Multi-Domain Service Function chaining", draft-li-sfc-hhsfc-07 (work in progress), September 2019. [KARMARKAR84] Karmarkar, N., "A new polynomial-time algorithm for linear programming", Book Title Proceedings of the sixteenth annual ACM symposium on Theory of computing, Pages 302-- 311, 1984. [LCLS] SLAC National Accelerator Laboratory, "The Linac Coherent Light Source", 2020, . Lachos, et al. Expires September 10, 2020 [Page 22] Internet-Draft Multi-domain use cases with ALTO March 2020 [LHC] CERN: European Council for Nuclear Research, "The Large Hadron Collider (LHC) Experiment", 2020, . [MD-ANALYTICS] Xiang, Q., Le, F., Yang, Y., Newman, H., and H. Du, "Unicorn: Resource Orchestration for Multi-Domain, Geo- Distributed Data Analytics", draft-xiang-alto-multidomain- analytics-02 (work in progress), July 2018. [MD-E2E-NS] Perez, D. and C. Rothenberg, "Multi-domain E2E Network Services", draft-lachosrothenberg-alto-md-e2e-ns-00 (work in progress), March 2019. [MD-ORCH-NFV] Katsalis, K., Nikaein, N., and A. Edmonds, "Multi-domain orchestration for NFV: Challenges and research directions", focus 189--195, 2016. [MD-SFC-ALTO] Perez, D., Xiang, Q., Rothenberg, C., and Y. Yang, "Multi- domain Service Function Chaining with ALTO", draft-lachos- multi-domain-sfc-alto-00 (work in progress), July 2018. [MD-USE-CASES] Xiang, Q., Le, F., and Y. Yang, "ALTO for Multi-Domain Applications: A Review of Use Cases and Design Requirements", draft-xiang-alto-multidomain-usecases-00 (work in progress), March 2019. [MERCATOR] Xiang, Q., Zhang, J., Wang, T., Liu, J., Guok, C., Le, F., MacAuley, J., Newman, H., and R. Yang, "Fine-grained, multi-domain network resource abstraction as a fundamental primitive to enable high-performance, collaborative data sciences", focus 58--70, 2018. [NGMN-5G] Alliance, NGMN, "5G White Paper", 2015, . [PAULRAJ10COMP] Paulraj, S. and P. Sumathi, "A comparative study of redundant constraints identification methods in linear programming problems", Journal Mathematical Problems in Engineering, Volume 2010, Publisher Hindawi Limited, 2010. Lachos, et al. Expires September 10, 2020 [Page 23] Internet-Draft Multi-domain use cases with ALTO March 2020 [SDI] Xiang, Q., Zhang, J., Gao, K., Lim, Y., Le, F., Li, G., and R. Yang, "Toward Optimal Software-Defined Interdomain Routing", Conference 39th Annual IEEE International Conference on Computer Communications (INFOCOM'20), 2020. [SDX] Gupta, A., Vanbever, L., Shahbaz, M., Donovan, S., Schlinker, B., Feamster, N., Rexford, J., Shenker, S., Clark, R., and E. Katz-Bassett, "Sdx: A software defined internet exchange", focus 551--562, 2015. [SFC-MD] Sun, G., Li, Y., Liao, D., and V. Chang, "Service function chain orchestration across multiple domains: A full mesh aggregation approach", Journal IEEE Transactions on Network and Service Management, Volumen 15, Number 3, Pages 1175--1191, Publisher IEEE, 2018. [SFP] Xiang, Q., Guok, C., Le, F., MacAuley, J., Newman, H., and R. Yang, "SFP: Toward Interdomain Routing for SDN Networks", focus 87--89, 2018. [SKA] SKA Organisation, "The Square Kilometre Array", 2020, . [TELGEN83] Telgen, J., "Identifying redundant constraints and implicit equalities in systems of linear constraints", Journal Management Science, Volume 29, Number 10, Pages 1209--1222, Publisher INFORMS, 1983. [UNIF-REPR] Xiang, Q., Le, F., and Y. Yang, "ALTO Extension: Unified Resource Representation", draft-xiang-alto-unified- representation-01 (work in progress), March 2019. Authors' Addresses Danny Alex Lachos Perez University of Campinas Av. Albert Einstein 400 Campinas, Sao Paulo 13083-970 Brazil Email: dlachosp@dca.fee.unicamp.br URI: https://intrig.dca.fee.unicamp.br/danny-lachos/ Lachos, et al. Expires September 10, 2020 [Page 24] Internet-Draft Multi-domain use cases with ALTO March 2020 Christian Esteve Rothenberg University of Campinas Av. Albert Einstein 400 Campinas, Sao Paulo 13083-970 Brazil Email: chesteve@dca.fee.unicamp.br URI: https://intrig.dca.fee.unicamp.br/christian/ Qiao Xiang Yale University 51 Prospect Street New Haven, CT USA Email: qiao.xiang@cs.yale.edu Y. Richard Yang Yale University 51 Prospect St New Haven, CT USA Email: yang.r.yang@gmail.com Borje Ohlman Ericsson Research S-16480 Stockholm Sweden Email: Borje.Ohlman@ericsson.com Sabine Randriamasy Nokia Bell Labs Route de Villejust NOZAY 91460 FRANCE Email: Sabine.Randriamasy@nokia-bell-labs.com Lachos, et al. Expires September 10, 2020 [Page 25] Internet-Draft Multi-domain use cases with ALTO March 2020 Farni Boten Sprint USA Email: farni.weaver@sprint.com Luis M. Contreras Telefonica Ronda de la Comunicacion, s/n Madrid 28050 Spain Email: luismiguel.contrerasmurillo@telefonica.com URI: http://lmcontreras.com/ Jingxuan Jensen Zhang Tongji University 4800 Caoan Road Shanghai 201804 China Email: jingxuan.n.zhang@gmail.com Kai Gao Sichuan University Chengdu 610000 China Email: kaigao@scu.edu.cn Lachos, et al. Expires September 10, 2020 [Page 26]