面向巨型星座的智能负载均衡算法

罗树欣; 张超; 肖勇; 刘建平

doi:10.11959/j.issn.2096-8930.2023042

您当前的位置：

首页 >

文章列表页 >

面向巨型星座的智能负载均衡算法

研究 | 更新时间：2024-06-03

- 面向巨型星座的智能负载均衡算法
- Intelligent Load Balancing Algorithm of Mega Constellation
- 天地一体化信息网络 2023年4卷第4期页码：49-60
- 作者机构：
  
  1. 西安交通大学电子与信息学部信息与通信工程学院，陕西西安710049
  2. 西安卫星测控中心宇航动力学国家重点实验室，陕西西安710043
- 作者简介：
  
  [ "罗树欣（1998- ），男，西安交通大学硕士生，主要研究方向为巨星座路由及仿真开发" ]
  [ "张超（1982- ），男，博士，西安交通大学教授，主要研究方向为卫星互联网、6G技术等" ]
  [ "肖勇（1984- ），男，博士，西安卫星测控中心高级工程师，主要研究方向为航天器飞行任务规划与调度" ]
  [ "刘建平（1975- ），男，博士，西安卫星测控中心教授级高级工程师，主要研究方向为航天器飞行任务规划与调度" ]
- 基金信息：
  
  国家重点研发计划资助项目;National Key Research and Development Program of China(2020YFB1806102);陕西省重点研发计划资助项目;Key Research and Development Program of Shaanxi(2023-YBGY-251)
- DOI：10.11959/j.issn.2096-8930.2023042
  中图分类号： TP393
- 网络出版日期：2023-12，
  
  纸质出版日期：2023-12-20
- 稿件说明：
移动端阅览
罗树欣, 张超, 肖勇, 等. 面向巨型星座的智能负载均衡算法[J]. 天地一体化信息网络, 2023,4(4):49-60.

Shuxin LUO, Chao ZHANG, Yong XIAO, et al. Intelligent Load Balancing Algorithm of Mega Constellation[J]. Space-integrated-ground information networks, 2023, 4(4): 49-60.
罗树欣, 张超, 肖勇, 等. 面向巨型星座的智能负载均衡算法[J]. 天地一体化信息网络, 2023,4(4):49-60. DOI： 10.11959/j.issn.2096-8930.2023042.

Shuxin LUO, Chao ZHANG, Yong XIAO, et al. Intelligent Load Balancing Algorithm of Mega Constellation[J]. Space-integrated-ground information networks, 2023, 4(4): 49-60. DOI： 10.11959/j.issn.2096-8930.2023042.

摘要

针对巨型星座中卫星数量众多容易引发局部拥塞的问题，提出基于协作多智能体深度强化学习的巨型星座负载均衡算法。首先对巨型星座中的卫星进行分簇设计，实现巨型星座的分布式管理，降低网络管理开销。然后，利用Q-混合多智能体神经网络深度强化学习设计各卫星自主决策的路由规划方案，实现多传输任务的簇内协同。此外，提出基于自动编码器的簇状态压缩机制，提高多智能体深度强化学习的效率。仿真结果表明，所提算法相比于传统的单任务路由算法，传输成功率可提升40%以上，证明所提算法能够避免局部拥塞的发生，提高巨型星座的传输效率。

Abstract

To overcome the local traffic congestion caused by the huge number of satellites in mega-constellation

a load balancing algorithm based on multi-agent deep reinforcement learning was proposed.Firstly

the satellites in the mega constellation were divided into clusters to perform the distributed management of the mega constellation

which could reduce the overhead of whole network.Then

based on the coordinated multi-agent deep reinforcement learning model

routing planning

which could be individually operated by satellites in the mega constellation

was designed to achieve the intra-cluster coordination.Additionally

a cluster state compression mechanism with autoencoder was proposed to compress the state space and improve the efficiency of multi-agent deep reinforcement learning.Finally

simulation results showed that compared with the traditional single-task routing algorithm

the proposed algorithm could increase the transmission success rate by more than 40% and the proposed algorithm could efficiently avoid local traffic congestion.

关键词

Keywords

references

SHENG M , ZHOU D , BAI W G , et al . 6G service coverage with mega satellite constellations [J ] . China Communications , 2022 , 19 ( 1 ): 64 - 76 .

阮永井 , 胡敏 , 云朝明 . 低轨巨型星座构型设计与控制研究进展与展望 [J ] . 中国空间科学技术 , 2022 , 42 ( 1 ): 1 - 15 .

RUAN Y J , HU M , YUN C M . Advances and prospects of the configuration design and control research of the LEO mega- constellations [J ] . Chinese Space Science and Technology , 2022 , 42 ( 1 ): 1 - 15 .

CHAUDHRY A U , YANIKOMEROGLU H . Laser intersatellite links in a starlink constellation:a classification and analysis [J ] . IEEE Vehicular Technology Magazine , 2021 , 16 ( 2 ): 48 - 56 .

RADHAKRISHNAN R , EDMONSON W W , AFGHAH F , et al . Survey of inter-satellite communication for small satellite systems:physical layer to network layer view [J ] . IEEE Communications Surveys ＆ Tutorials , 2016 , 18 ( 4 ): 2442 - 2473 .

YANG X . Low earth orbit (LEO) mega constellations:satellite and terrestrial integrated communication networks [D ] . Guildford:University of Surrey , 2019 .

陈全 , 杨磊 , 郭剑鸣 , 等 . 低轨巨型星座网络:组网技术与研究现状 [J ] . 通信学报 , 2022 , 43 ( 5 ): 177 - 189 .

CHEN Q , YANG L , GUO J M , et al . LEO mega-constellation network:networking technologies and state of the art [J ] . Journal on Communications , 2022 , 43 ( 5 ): 177 - 189 .

周笛 , 盛敏 , 郝琪 , 等 . 巨型星座系统的网络运维与资源管控技术 [J ] . 天地一体化信息网络 , 2020 , 1 ( 1 ): 26 - 35 .

ZHOU D , SHENG M , HAO Q , et al . Network operation,maintenance and resource management in mega constellation system [J ] . Space-Integrated-Ground Information Networks , 2020 , 1 ( 1 ): 26 - 35 .

CHEN Q , GIAMBENE G , YANG L , et al . Analysis of inter-satellite link paths for LEO mega-constellation networks [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 3 ): 2743 - 2755 .

QU Z C , ZHANG G X , CAO H T , et al . LEO satellite constellation for Internet of Things [J ] . IEEE Access , 2017 , 5 : 18391 - 18401 .

LEE Y , CHOI J P . Connectivity analysis of mega-constellation satellite networks with optical intersatellite links [J ] . IEEE Transactions on Aerospace and Electronic Systems , 2021 , 57 ( 6 ): 4213 - 4226 .

XIE H R , ZHAN Y F , ZENG G M , et al . LEO mega-constellations for 6G global coverage:challenges and opportunities [J ] . IEEE Access , 2021 , 9 : 164223 - 164244 .

TANG F L , ZHANG H T , YANG L T . Multipath cooperative routing with efficient acknowledgement for LEO satellite networks [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 1 ): 179 - 192 .

巢孟愿 . 卫星网络多路径路由算法与切换策略研究 [D ] . 杭州:浙江大学 , 2014 .

CHAO M Y . Research on multipath routing algorithm and handover strategy in satellite networks [D ] . Hangzhou:Zhejiang University , 2014 .

ZENG G M , ZHAN Y F , PAN X H . Failure-tolerant and low-latency telecomm and in mega-constellations:the redundant multi-path routing [J ] . IEEE Access , 2021 , 9 : 34975 - 34985 .

LIU Y L , ZHU L D . A suboptimal routing algorithm for massive LEO satellite networks [C ] // Proceedings of 2018 International Symposium on Networks,Computers and Communications (ISNCC) . Piscataway:IEEE Press , 2018 : 1 - 5 .

LI J , LU H C , XUE K P , et al . Temporal netgrid model-based dynamic routing in large-scale small satellite networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 6 ): 6009 - 6021 .

QI X X , ZHANG B , QIU Z L . A distributed survivable routing algorithm for mega-constellations with inclined orbits [J ] . IEEE Access , 2020 , 8 : 219199 - 219213 .

倪少杰 , 岳洋 , 左勇 , 等 . 卫星网络路由技术现状及展望 [J ] . 电子与信息学报 , 2023 , 45 ( 2 ): 383 - 395 .

NI S J , YUE Y , ZUO Y , et al . The status quo and prospect of satellite network routing technology [J ] . Journal of Electronics ＆ Information Technology , 2023 , 45 ( 2 ): 383 - 395 .

郑爽 , 张兴 , 王文博 . 低轨卫星通信网络路由技术综述 [J ] . 天地一体化信息网络 , 2022 , 3 ( 3 ): 97 - 105 .

ZHENG S , ZHANG X , WANG W B . Survey of low earth orbit satellite communication network routing technology [J ] . Space- Integrated- Ground Information Networks , 2022 , 3 ( 3 ): 97 - 105 .

石晓东 , 李勇军 , 赵尚弘 , 等 . 基于SDN的卫星网络多QoS目标优化路由算法 [J ] . 系统工程与电子技术 , 2020 , 42 ( 6 ): 1395 - 1401 .

SHI X D , LI Y J , ZHAO S H , et al . Multi-QoS objective optimization routing algorithm of satellite network based on SDN [J ] . Systems Engineering and Electronics , 2020 , 42 ( 6 ): 1395 - 1401 .

LIU J , ZHANG X Y , ZHANG R , et al . Reliable and low-overhead clustering in LEO small satellite networks [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 16 ): 14844 - 14856 .

TANG F X , HOFNER H , KATO N , et al . A deep reinforcement learning-based dynamic traffic offloading in space-air-ground integrated networks (SAGIN) [J ] . IEEE Journal on Selected Areas in Communications , 2022 , 40 ( 1 ): 276 - 289 .

QI X G , MA J L , WU D , et al . A survey of routing techniques for satellite networks [J ] . Journal of Communications and Information Networks , 2016 , 1 ( 4 ): 66 - 85 .

沙鹏翔 . 基于分簇的巨型低轨星座高效管理方法研究 [D ] . 南京:南京大学 , 2021 .

SHA P X . Research on efficient management method of giant LEO constellation based on clustering [D ] . Nanjing:Nanjing University , 2021 .

刘全 , 翟建伟 , 章宗长 , 等 . 深度强化学习综述 [J ] . 计算机学报 , 2018 , 41 ( 1 ): 1 - 27 .

LIU Q , ZHAI J W , ZHANG Z C , et al . A survey on deep reinforcement learning [J ] . Chinese Journal of Computers , 2018 , 41 ( 1 ): 1 - 27 .

裴培 , 何绍溟 , 王江 , 等 . 一种深度强化学习制导控制一体化算法 [J ] . 宇航学报 , 2021 , 42 ( 10 ): 1293 - 1304 .

PEI P , HE S M , WANG J , et al . Integrated guidance and control for missile using deep reinforcement learning [J ] . Journal of Astronautics , 2021 , 42 ( 10 ): 1293 - 1304 .

NAZARI M , OROOJLOOY A , TAKÁČ M , et al . Reinforcement learning for solving the vehicle routing problem [C ] // Proceedings of the 32nd International Conference on Neural Information Processing Systems . New York:ACM , 2018 : 9861 - 9871 .

孙彧 , 曹雷 , 陈希亮 , 等 . 多智能体深度强化学习研究综述 [J ] . 计算机工程与应用 , 2020 , 56 ( 5 ): 13 - 24 .

SUN Y , CAO L , CHEN X L , et al . Overview of multi-agent deep reinforcement learning [J ] . Computer Engineering and Applications , 2020 , 56 ( 5 ): 13 - 24 .

ZHANG K , YANG Z , BAŞAR T . Multi-agent reinforcement learning:A selective overview of theories and algorithms [J ] . Handbook of reinforcement learning and control , 2021 : 321 - 384 .

RASHID T , SAMVELYAN M , DE WITT C S , et al . QMIX:monotonic value function factorisation for deep multi-agent reinforcement learning [EB ] . 2018 .

陈亮 , 梁宸 , 张景异 , 等 . Actor-Critic 框架下一种基于改进DDPG 的多智能体强化学习算法 [J ] . 控制与决策 , 2021 , 36 ( 1 ): 75 - 82 .

CHEN L , LIANG C , ZHANG J Y , et al . A multi-agent reinforcement learning algorithm based on improved DDPG in Actor-Critic framework [J ] . Control and Decision , 2021 , 36 ( 1 ): 75 - 82 .

刘建伟 , 高峰 , 罗雄麟 . 基于值函数和策略梯度的深度强化学习综述 [J ] . 计算机学报 , 2019 , 42 ( 6 ): 1406 - 1438 .

LIU J W , GAO F , LUO X L . Survey of deep reinforcement learning based on value function and policy gradient [J ] . Chinese Journal of Computers , 2019 , 42 ( 6 ): 1406 - 1438 .

韩冲 , 王俊丽 , 吴雨茜 , 等 . 基于神经进化的深度学习模型研究综述 [J ] . 电子学报 , 2021 , 49 ( 2 ): 372 - 379 .

HAN C , WANG J L , WU Y X , et al . A review of deep learning models based on neuroevolution [J ] . Acta Electronica Sinica , 2021 , 49 ( 2 ): 372 - 379 .

LIU J H , ZHAO B K , XIN Q , et al . DRL-ER:an intelligent energy-aware routing protocol with guaranteed delay bounds in satellite mega-constellations [J ] . IEEE Transactions on Network Science and Engineering , 2021 , 8 ( 4 ): 2872 - 2884 .

浏览量

591

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

巨型星座路由技术综述

面向巨型星座系统的多地面站协同测控技术

基于多智能体深度强化学习的测运控一体化资源调度方法

软件定义卫星网络辅助的空基网络多域路由策略

基于负载均衡的大规模低轨卫星互联网路由算法