Resource Scheduling Algorithm Based on DQN in Satellite CDN

Jiaran ZHANG; Yating YANG; Tian SONG

doi:10.11959/j.issn.2096-8930.2022042

您当前的位置：

首页 >

文章列表页 >

Resource Scheduling Algorithm Based on DQN in Satellite CDN

Special Issue: Satellite Internet User Access Control | 更新时间：2024-06-03

- Resource Scheduling Algorithm Based on DQN in Satellite CDN
- Space-Integrated-Ground Information Networks Vol. 3, Issue 4, Pages: 45-54(2022)
- 作者机构：
  
  1. 北京理工大学计算机学院，北京 100081
  2. 北京理工大学网络空间安全学院，北京 100081
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-8930.2022042
  CLC： TN393
- Published Online：2022-12，
  
  Published：20 December 2022
- 稿件说明：
移动端阅览
Jiaran ZHANG, Yating YANG, Tian SONG. Resource Scheduling Algorithm Based on DQN in Satellite CDN[J]. Space-integrated-ground information networks, 2022, 3(4): 45-54.
DOI：

Jiaran ZHANG, Yating YANG, Tian SONG. Resource Scheduling Algorithm Based on DQN in Satellite CDN[J]. Space-integrated-ground information networks, 2022, 3(4): 45-54. DOI： 10.11959/j.issn.2096-8930.2022042.

摘要

随着空间与信息技术迅速发展，热点内容分发密集型场景将成为卫星网络应用的重点方向，而卫星内容分发网络（CDN）是提高空天内容分发效率的重要手段。针对卫星CDN体系架构中存在的业务需求时空不均、卫星资源稀缺、现有编排算法适应性不足等问题，提出一种基于深度Q学习（Deep Q-Learning

DQN）的资源编排算法。该方法首先对用户请求进行分类，依据卫星时变运行轨迹和星地资源情况，计算出卫星可通信的最短路径集合；之后通过马尔可夫模型建模量化卫星和用户的相关信息，利用DQN算法计算出最优的卫星CDN存储节点，达到降低用户请求时延，降低星地资源占用率，提高缓存命中率的效果。

Abstract

With the rapid development of space and information fi eld

hot content distribution intensive scenes will become one of the key directions of satellite network application

and satellite content delivery network (CDN) network is an important means to improve the effi ciency of air and space content distribution.In the architecture of satellite CDN network

due to the uneven time and space of business requirements

the scarcity of satellite resources and the insuffi cient adaptability of existing scheduling algorithms

scheduling algorithms for satellite resources are faced with problems such as high resource dimension

many computing states and large amount of computation

which will reduce the accuracy

response speed and computing performance of scheduling decisions.To solve this problem

a resource scheduling algorithm based on Deep Q-Learning (DQN) algorithm was proposed to improved the effi ciency and accuracy of satellite resource scheduling

and intelligently and quickly perceived the resource situation to make scheduling decisions.Firstly

the user requests were classifi ed

and the shortest path set that the satellite could communicated with was calculated according to the time-varying trajectory of the satellite and the resources of the satellite and the ground.After that

the related information of satellites and users was quantifi ed by Markov model modeling

and the optimal CDN storage node of satellites was calculated by DQN algorithm

which achieved the eff ects of reduced user request delay

reduced satellite-ground resource occupancy rate and improved cache hit rate.

关键词

Keywords

references

WAN P , YAN J , CAO Z G , et al . Dynamic Source Routing algorithm in low-earth orbit Satellite Constellation [C ] // Proceedings of 2006 International Conference on Communication Technology . Piscataway:IEEE Press , 2006 : 1 - 4 .

陈乃辉 . 卫星网络环境下数据一致性研究 [D ] . 武汉:华中科技大学 , 2018 .

CHEN N H . Research on data consistency in satellite network environment [D ] . Wuhan:Huazhong University of Science and Technology , 2018 .

肖华 . CDN质量发展情况与优化方向探讨 [J ] . 广播电视网络 , 2022 , 29 ( 2 ): 87 - 89 .

XIAO H . Discussion on CDN quality development and optimization direction [J ] . Radio ＆ Television Network , 2022 , 29 ( 2 ): 87 - 89 .

张美蓉 , 镐梦婷 , 王闯 , 等 . 高低轨卫星异构网络资源管控策略与技术研究 [J ] . 天地一体化信息网络 , 2021 , 2 ( 4 ): 67 - 74 .

ZHANG M R , GAO M T , WANG C , et al . Research on strategies and technologies for resource management and control of heterogeneous network of high and low orbit satellites [J ] . Space-Integrated-Ground Information Networks , 2021 , 2 ( 4 ): 67 - 74 .

HAN C , LIU A J , HUO L Y , et al . A prediction-based resource matching scheme for rentable LEO satellite communication network [J ] . IEEE Communications Letters , 2020 , 24 ( 2 ): 414 - 417 .

唐宏 , 陈戈 , 陈步华 , 等 . 内容分发网络原理与实践 [J ] . 电信科学 , 2018 , 34 ( 10 ): 181 .

TANG H , CHEN G , CHEN B H , et al . Principle and practice of CDN [J ] . Telecommunications Science , 2018 , 34 ( 10 ): 181 .

张焘 . 时变网络流分析与调度方法研究 [D ] . 西安:西安电子科技大学 , 2020 .

ZHANG T . Network flow analysis and scheduling in time-varying netwoks [D ] . Xi'an:Xidian University , 2020 .

曹洋 , 朱玉峰 , 吕俊锋 , 等 . 面向天地一体化内容分发业务的网内缓存机制研究 [J ] . 天地一体化信息网络 , 2020 , 1 ( 2 ): 48 - 56 .

CAO Y , ZHU Y F , LYU J F , et al . Research on in-network caching mechanisms for space-integrated-ground content delivery service [J ] . Space-Integrated-Ground Information Networks , 2020 , 1 ( 2 ): 48 - 56 .

LI Y , WANG Y , ZHANG Q Y , et al . TCDS:a time-relevant graph based topology control in triple-layer satellite networks [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 3 ): 424 - 428 .

吕俊锋 . 面向天地一体化网络内容分发的网内缓存机制研究 [D ] . 武汉:华中科技大学 , 2020 .

LYU J F . In-network caching mechanisms for content delivery in space-ground integration network [D ] . Wuhan:Huazhong University of Science and Technology , 2020 .

WANG P , ZHANG X S , ZHANG S , et al . Time-expanded graphbased resource allocation over the satellite networks [J ] . IEEE Wireless Communications Letters , 2019 , 8 ( 2 ): 360 - 363 .

徐媚琳 , 贾敏 , 郭庆 . 基于SDN/NFV的卫星互联网服务功能资源分配研究 [J ] . 天地一体化信息网络 , 2022 , 3 ( 1 ): 44 - 49 .

XU M L , JIA M , GUO Q . Research on resource allocation of satellite Internet service function based on SDN/NFV [J ] . Space-IntegratedGround Information Networks , 2022 , 3 ( 1 ): 44 - 49 .

李红艳 , 张焘 , 张靖乾 , 等 . 基于时变图的天地一体化网络时间确定性路由算法与协议 [J ] . 通信学报 , 2020 , 41 ( 10 ): 116 - 129 .

LI H Y , ZHANG T , ZHANG J Q , et al . Time deterministic routing algorithm and protocol based on time-varying graph over the spaceground integrated network [J ] . Journal on Communications , 2020 , 41 ( 10 ): 116 - 129 .

ZHOU D , SHENG M , WANG Y X , et al . Machine learning-based resource allocation in satellite networks supporting Internet of remote things [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 10 ): 6606 - 6621 .

DUN C Y , TAN S H , JIN F L , et al . Traffic allocation strategy based on reinforcement learning in two-layer satellite network [C ] // Proceedings of 2021 International Conference on Internet,Education and Information Technology (IEIT) . Piscataway:IEEE Press , 2021 : 159 - 163 .

CEVALLOS M J F , SATTLER R , CAULIER CISTERNA R P , et al . Online service function chain deployment for live-streaming in virtualized content delivery networks:a deep reinforcement learning approach [J ] . Future Internet , 2021 , 13 ( 11 ): 278 .

CHEN Z Y , HU J , MIN G Y , et al . Adaptive and efficient resource allocation in cloud datacenters using actor-critic deep reinforcement learning [J ] . IEEE Transactions on Parallel and Distributed Systems , 2022 , 33 ( 8 ): 1911 - 1923 .

LIU Y J , LU D J , ZHANG G J , et al . Q-Learning based content placement method for dynamic cloud content delivery networks [J ] . IEEE Access , 7 : 66384 - 66394 .

HUANG W Y , SONG T , AN J P . QA2:QoS-guaranteed access assistance for space–air–ground Internet of vehicle networks [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 8 ): 5684 - 5695 .

薛乃阳 , 丁丹 , 贾玉童 , 等 . 基于DQN的异构测控资源联合调度方法 [J ] . 系统仿真学报 , 2022 : 1 - 13 .

XUE N Y , DING D , JIA Y T , et al . Joint scheduling method of heterogeneous measurement and control resources based on DQN [J ] . Journal of System Simulation , 2022 : 1 - 13 .

李贺武 , 吴茜 , 徐恪 , 等 . 天地一体化网络研究进展与趋势 [J ] . 科技导报 , 2016 , 34 ( 14 ): 95 - 106 .

LI H W , WU Q , XU K , et al . Progress and tendency of space and earth integrated network [J ] . Science ＆ Technology Review , 2016 , 34 ( 14 ): 95 - 106 .

王凯东 , 易克初 , 田斌 . 低轨道(LEO)宽带卫星网络最短路由优化算法 [J ] . 中国科学E辑:信息科学 , 2005 , 35 ( 8 ): 850 - 874 .

WANG K D , YI K C , TIAN B . Shortest route optimization algorithm for low orbit (LEO) broadband satellite network [J ] . Science in China,SerE , 2005 , 35 ( 8 ): 850 - 874 .

倪爽 . 星地一体化网络接入与存储资源协同管控技术研究 [D ] . 西安:西安电子科技大学 , 2021 .

NI S . Coordinated access and cache resource management technology in terrestrial-satellite integrated network [D ] . Xi'an:Xidian University , 2021 .

徐建中 . 云计算环境下资源分配技术的研究 [D ] . 南京:南京理工大学 , 2018 .

XU J Z . Research on resource allocation technology in cloud computing environment [D ] . Nanjing:Nanjing University of Science and Technology , 2018 .

曾首元 . 基于蚁群优化的低轨卫星网络负载均衡路由算法研究 [D ] . 广州:广州大学 , 2022 .

ZENG S Y . Research on load balancing routing algorithm of LEO satellite network based on ant colony optimization [D ] . Guangzhou:Guangzhou University , 2022 .

杜琴 . 基于卫星切片网络的计算资源和无线资源联合分配机制的研究 [D ] . 北京:北京邮电大学 , 2020 .

DU Q . Research on joint allocation mechanism of computing resource and wireless resource based on satellite slicing network [D ] . Beijing:Beijing University of Posts and Telecommunications , 2020 .

Views

417

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰