T2TRG Hong, Choong Seon Internet-Draft Kyung Hee University Intended status: Standards Track Pandey, Shashi Raj Expires: August 09, 2021 Kyung Hee University Suhail, Sabah Kyung Hee University Tun, Yan Kyaw Kyung Hee University Kim, Kitae Kyung Hee University October 13, 2020 Resource Allocation Strategy for Latency Sensitive IoT Traffic draft-hongcs-t2trg-ras-00 Abstract An efficient resource allocation scheme directly affects the overall system's network utilization, and notably, the wireless resource such as bandwidth is itself an expensive commodity. In this regards, to address the requirement of massively increased IoT traffic at the edge, as a solution approach, a number of small cell base stations (SBSs) have been deployed with certain computational capabilities. However, it is still limited, and any inappropriate resource allocation scheme for the associated nodes with traffic characterized by ultra-reliability and low-latency (URLLC) requirements can impact the system's resource utilization and performance. Considering a reserve resource for such kind of traffic, the resource allocation problem in each time slot behaves as a node selection problem with contention amongst active nodes for the resource. In this paper, we have formulated this as an index problem, and with simulation results have shown that the cumulative reward in terms of network utility is maximized following this approach. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Hong, et al. Expires August 09, 2021 [Page 1] Internet-Draft Resource Allocation for IoT Traffic October, 2020 This Internet-Draft will expire on August 09, 2021. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . .. . . . . . 2 1.1. Terminology and Requirements Language. . . . . . . . . 2 2. System Model . . . . . . . . . . . . . . . . . . . . . . . . .2-3 3. Problem Formulation . . . . . . . . . . . . . . . . . . . . . .3-5 4. Results . . . . . . . . . . . . .. . . . . . . . . . . . . . . 6 5. IANA Considerations . . . . . . .. . . . . . . . . . . . . . . 6 6. Security Considerations . . . . . . . . . . . . . . . . . . . 6 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 7.1. Normative References . . . . . . . . . . . . . . . . . . . . . 8 7.2. Informative References . . . . . .. . . . . . . . . . . . . 8-9 Authors' Addresses . . . . . . . . . . . . . . . . . . . . .. . . . 10 1. Introduction Radio resources such as bandwidth is considered scare, and is therefore an expensive utility whose management has been of a great challenge for the evolution of mobile networks. On the top of that, traffic with strict latency and reliability requirements in 5G networks requires an efficient resource allocation scheme to make it a success [1]. On the other hand, the massive growth of IoT networks has brought up numerous challenges while handling such traffic in an effective way [2]. Wireless Network virtualization has emerged as an promising alternative to manage the wireless resources amongst number of participating users. It invoked the concept of network slicing, and flexible allocation of the slices of network resources, such as bandwidth [3], [4], [5] to the users. In [6], authors have introduced an online network slice broker to facilitate the network resources for improving the overall network utilization ratio. The problem was formulated as a bandit problem (MAB) corresponding to the mobile traffic forecasting scenario [7]. As with the MAB problem, there exist the dilemma of exploration and exploitation to attain the cumulative reward, while minimizing the total regret in the system. The solution for this can be found using Bellman's equation [10], however, the solution cost is expensive for the larger systems. Hong, et al. Expires August 09, 2021 [Page 2] Internet-Draft Resource Allocation for IoT Traffic October, 2020 In case of IoT systems, with abrupt traffic response and latency sensitive requirements, such approach may appear to be impractical. Thus, there should be a mechanism to efficiently allocate the network resources for addressing these stringent demands of IoT traffic while improving the overall network utility. We formulate this scenario by firstly employing the technique of reserve resources for latency sensitive traffic of IoT nodes in terms of mini-slots, as mentioned in 3GPP standard for 5G networks. We define the network scenario as a family of bandit process, and implement the index theory approach [8] for a family of semi-decision Markov Process to prioritize the nodes that could maximize the overall network utilization upon reserve resource allocation. 1.1. Terminology and Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. System Model We consider n IoT nodes associated with a single small cell base station (SBS) (see Figure 1). Each node can collect sensory data, perform certain amount of computation upon it and forward it to the SBS for diverse service oriented applications. We further categorize the traffic generated by the IoT nodes in terms of low-latency and ultra-reliability requirements. Here, for each node i, we will define its state at time t, x_i(t) with the fraction of reserve resource, bandwidth B demanded from the SBS, by quantizing the reserved resource into N levels denoted by a set N = {1, 2, . . . , N}, where x_i(t) belongs to set N. If node i in state x_i(t) is chosen for the reserved resource in the time slot t, the reward value obtained by the SBS in terms of resource utilization is defined as r_i(x_i(t)).This way a sequential node selection scenario exist for the SBS to allocate reserve resource to one of the requesting node. Because the reward distribution is unknown, the SBS can allocate the reserve resource following the solution approach for the multiarmed bandit problem, to maximize its cumulative reward over the time. Alternatively, if we can prioritize a node i, given its state x_i(t) and corresponding reward r_i(x_i(t)), the SBS can sequentially resolve the node selection problem in an efficient way. Hong, et al. Expires August 09, 2021 [Page 3] Internet-Draft Resource Allocation for IoT Traffic October, 2020 +---------------+ +------------+ | SBS | | IoT Nodes | | | | | +---------------+ +------------+ | | | | | +------------------+ | | | Communication | | | | link | | | | | | | +------------------+ | | | | <------------------------> | Figure 1: System model 3. Problem Formulation The defined problem can be represented as a n-arm bandit and a single player (SBS) scenario, where at each time t, the player (SBS) chooses one arm (IoT node) to play (allocate its reserve resource as illustrated in Figure 2). The process can be extended in reference with the sequence of time t_i and states of the nodes, x_i(t), for all i, and be consider a family of bandit process F = {F_1, F_2, . . . , F_n} as in [8]. Here, the bandit process F_i, for all i is defined with the state x_i(t), and reward at the state r_i(x_i(t)) > 0, and is considered to be an exponentially discounted semi-Markov decision process with a constant duration between the decision times. For convenience, we keep it as 1. Hong, et al. Expires August 09, 2021 [Page 4] Internet-Draft Resource Allocation for IoT Traffic October, 2020 +-------------------+-------------------------------+ | Reserve Resource | | B | | | | | | +-------------------+-------------------------------+ \ Mini slots / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / +------------+-----------+-----------+-----------+ | | | | | +------------+-----------+-----------+-----------+ Time slots (T) ----------------> Figure 2: Resource reservation We adapt the control set u = {0, 1}, where the control 0 freezes the process. That means, there is no change in state and no reward is obtained from the process. Similarly, control 1 is defined as the continuation control that returns an immediate reward a_t r_i(x_i(t)) = exponential(-gamma t) r_i(x_i(t)). Here, the parameters a(0 < a < 1) and gamma (gamma > 0) are defined to be the discount factor and the discount parameter respectively for obtaining a bounded reward Hong, et al. Expires August 09, 2020 [Page 5] Internet-Draft Resource Allocation for IoT Traffic October, 2020 value. This way, the presented problem resembles with a discounted- reward Markov decision process which solution can be found using dynamic programing equations [10]. However, the solution becomes difficult to solve for a n-bandit process where the problem grows exponentially. Therefore, using these definitions, we refer [9] which states that for a discrete time Markov decision process, there exists an optimal policy defined as index policy, which is characterized by a real-valued index, v(F_i, x_i(t)), and it is to continue the bandit process having greatest index. +---------------------------+ | Input Bandit processes, | | discount factor, | | system parameters | | | +---------------------------+ | | v +-----------------------------------+ |Calculate the index value v(F_i); | | Removed the checked node i; | <-------------+ | | | +-----------------------------------+ | | | v | / \ | / \ | / \ | / Are \ | / all \ No | / nodes i \ ----------------------------+ \ checked ?/ \ / \ / \ / \ / \ / | |Yes v +--------------------------------+ | Return maximum index value, | | and its corresponding | | index. | +--------------------------------+ | v ( End ) Figure 3: Algorithm 1: Index based node association Hong, et al. Expires August 09, 2021 [Page 6] Internet-Draft Resource Allocation for IoT Traffic October, 2020 This means, the return of reward with the choice of control u applied for the bandit process will be improved by selecting continuation control on the bandit with the greatest index. Therefore, for the node selection problem at time t, we can evaluate the index values at the bandit processes given state x_i(t), for all i. Then after, we can choose to apply the continuation control to the bandit with highest index value which guarantees for better discounted cumulative reward. The detail implementation for this scenario is presented in Algorithm 1 (see Figure 3). 4. Results An index base resource allocation scheme considers latency sensitive IoT traffic and improves the overall network utilization. The network utilization can be defined in terms of cumulative reward while effectively allocating reserve resource to the family of bandit processes. We observe the improvement in cumulative reward while implementing index based node selection approach for allocating the reserved resource. 5. IANA Considerations There are no IANA considerations related to this document. 6. Security Considerations There are no security considerations related to this document. the reserved resource. Hong, et al. Expires August 09, 2021 [Page 7] Internet-Draft Resource Allocation for IoT Traffic October, 2020 7. References 7.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [1] Bennis, Mehdi and Debbah, Merouane and Poor, H Vincent, "Ultra- Reliable and Low-Latency Wireless Communication: Tail, Risk and Scale.", arXiv:1801.01270 , 2018. [2] Chen, Shanzhi and Zhao, Jian, "The requirements, challenges, and technologies for 5G of terrestrial mobile telecommunication.", IEEE Communications Magazine, vol. 52, no. 5, pp. 36-43, 2014. [3] Liang, Chengchao and Yu, F Richard, "Wireless network virtualization: A survey, some research issues and challenges.", IEEE Communications Surveys & Tutorials, vol. 17, no. 1, pp. 358-380, 2015. [4] Nakao, Akihiro and Du, Ping and Kiriha, Yoshiaki and Granelli, Fabrizio and Gebremariam, Anteneh Atumo and Taleb, Tarik and Bagaa, Miloud,"End-to-end network slicing for 5g mobile networks.", Journal of Information Processing, vol. 25, pp. 153-163, 2017. [5] Zhang, Haijun and Liu, Na and Chu, Xiaoli and Long, Keping and Aghvami, Abdol-Hamid and Leung, Victor CM, "Network slicing based 5G and future mobile networks: mobility, resource management, and challenges.", IEEE Communications Magazine, vol. 55, no. 8, 138-145, 2017. [6] Sciancalepore, Vincenzo and Zanzi, Lanfranco and Costa-Perez, Xavier and Capone, Antonio, "ONETS: Online Network Slice Broker From Theory to Practice", arXiv:1801.03484, 2018. [7] Sciancalepore, Vincenzo and Samdanis, Konstantinos and Costa-Perez ,Xavier and Bega, Dario and Gramaglia, Marco and Banchs, Albert, "Mobile traffic forecasting for maximizing 5G network slicing resource utilization", In INFOCOM 2017-IEEE Conference on Computer Communications, IEEE, pp. 1-9. IEEE, 2017 Hong, et al. Expires August 09, 2021 [Page 8] Internet-Draft Resource Allocation for IoT Traffic October, 2020 [8] Gittins, John and Glazebrook, Kevin and Weber, Richard., 2011 Multiarmed bandit allocation indices. [9] Weber, Richard,1992 "On the Gittins index for multiarmed bandits" The Annals of Applied Probability,1024-1033. [10] Bellman, Richard, 2013 Dynamic programming. 7.2. Informative References Hong, et al. Expires August 09, 2021 [Page 9] Internet-Draft Resource Allocation for IoT Traffic October, 2020 Authors' Addresses Choong Seon Hong Computer Science and Engineering Department, Kyung Hee University Yongin, South Korea Phone: +82 (0)31 201 2532 Email: cshong@khu.ac.kr Shashi Raj Pandey Computer Science and Engineering Department, Kyung Hee University Yongin, South Korea Phone: +82 (0)10 3855 8816 Email: shashiraj@khu.ac.kr Sabah Suhail Computer Science and Engineering Department, Kyung Hee University Yongin, South Korea Phone: Email: sabah@khu.ac.kr Yan Kyaw Tun Computer Science and Engineering Department, Kyung Hee University Yongin, South Korea Phone: Email: ykyawtun7@khu.ac.kr Kitae Kim Computer Science and Engineering Department, Kyung Hee University Yongin, South Korea Phone: Email: glideslope@khu.ac.kr Hong, et al. Expires August 09, 2021 [Page 10]