基于深度强化学习的卫星互联网路由优化研究

魏琳慧; 刘国文; 刘雨; 望育梅

doi:10.11959/j.issn.2096-8930.2022033

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的卫星互联网路由优化研究

专题：天地融合软件定义网络 | 更新时间：2024-06-03

- 基于深度强化学习的卫星互联网路由优化研究
- Research on Routing Optimization in Satellite Internet Based on Deep Reinforcement Learning
- 天地一体化信息网络 2022年3卷第3期页码：65-71
- 作者机构：
  
  1. 北京邮电大学人工智能学院，北京 100876
  2. 鹏城实验室，广东深圳 518000
- 作者简介：
  
  [ "魏琳慧（1997-），女，北京邮电大学人工智能学院博士生，主要研究方向为卫星互联网、软件定义网络、多媒体传输技术等" ]
  [ "刘国文（1998-），男，北京邮电大学人工智能学院硕士生，主要研究方向为卫星互联网、机器学习在低轨卫星网络中的应用等" ]
  [ "刘雨（1978-），女，北京邮电大学人工智能学院副教授，博士生导师，鹏城实验室网络与通信研究中心副教授，主要研究方向为卫星互联网、图像处理、分布式源编码等" ]
  [ "望育梅（1974-），女，北京邮电大学人工智能学院副教授，硕士生导师，主要研究方向为卫星互联网、多媒体信号处理、无线多媒体传输和分布式视频编码等" ]
- 基金信息：
  
  国家重点研发计划;The National Key Research and Development Program of China(2019YFB1803103);北京邮电大学博士生创新基金项目;BUPT Excellent Ph.D.Students Foundation(CX2021113)
- DOI：10.11959/j.issn.2096-8930.2022033
  中图分类号： TN927.2
- 网络出版日期：2022-09，
  
  纸质出版日期：2022-09-20
- 稿件说明：
移动端阅览
魏琳慧, 刘国文, 刘雨, 等. 基于深度强化学习的卫星互联网路由优化研究[J]. 天地一体化信息网络, 2022,3(3):65-71.

Linhui WEI, Guowen LIU, Yu LIU, et al. Research on Routing Optimization in Satellite Internet Based on Deep Reinforcement Learning[J]. Space-integrated-ground information networks, 2022, 3(3): 65-71.
魏琳慧, 刘国文, 刘雨, 等. 基于深度强化学习的卫星互联网路由优化研究[J]. 天地一体化信息网络, 2022,3(3):65-71. DOI： 10.11959/j.issn.2096-8930.2022033.

Linhui WEI, Guowen LIU, Yu LIU, et al. Research on Routing Optimization in Satellite Internet Based on Deep Reinforcement Learning[J]. Space-integrated-ground information networks, 2022, 3(3): 65-71. DOI： 10.11959/j.issn.2096-8930.2022033.

摘要

随着卫星通信技术的飞速发展，卫星互联网成为6G网络实现全球覆盖、全时接入、全场景服务的核心关键技术。卫星网络的高动态性及有限的卫星容量，导致面临以异构网络管理、动态资源分配为代表的一系列管控挑战。由于机器学习技术在网络设计等方面具有显著优势，因此提出软件定义的卫星互联网智能化架构。针对卫星互联网的智能路由问题，利用基于双延迟深度确定性策略梯度的深度强化学习算法，解决网络的实时路由优化问题。实验结果表明，TD3算法相较于DDPG算法，平均网络时延降低了19.19%。

Abstract

With the rapid development of satellite communication

the satellite internet is one of the core technologies of 6G network to realize global coverage

full-time access and full scene service.The high dynamics and limited capacity of satellite network lead to a series of management and control challenges such as heterogeneous network management

dynamic resource allocation and so on.Since the machine learning-based technologies have strength in network design

the intelligent architecture of software-defi ned satellite internet was put forward.In view of the intelligent routing in satellite internet

and leverages the deep reinforcement algorithm based on double delayed deep deterministic policy gradient (TD3) to solve the network routing optimization problem.The experimental results showed that compared with DDPG algorithm

the TD3 algorithm reduced the delay by 19.19%.

关键词

Keywords

references

吴巍 . 卫星互联网发展综述 [J ] . 天地一体化信息网络 , 2020 , 1 ( 1 ): 1 - 16 .

WU W . Survey on the development of space-integrated-ground information network [J ] . Space-Integrated-Ground Information Networks , 2020 , 1 ( 1 ): 1 - 16 .

KREUTZ D , RAMOS F M V , VERÍSSIMO P E , , et al . Softwaredefined networking,a comprehensive survey [J ] . Proceedings of the IEEE , 2015 , 103 ( 1 ): 14 - 76 .

TANG Z , ZHAO B K , YU W R , et al . Software defined satellite networks,benefits and challenges [C ] // Proceedings of 2014 IEEE Computers,Communications and IT Applications Conference . Piscataway,IEEE Press , 2014 : 127 - 132 .

FORTZ B , THORUP M . Internet traffic engineering by optimizing OSPF weights [C ] // Proceedings of IEEE INFOCOM 2000.Conference on Computer Communications.Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat.No.00CH37064) . Piscataway,IEEE Press , 2000 : 519 - 528 .

XIE J F , YU F R , HUANG T , et al . A survey of machine learning techniques applied to software defined networking (SDN),research issues and challenges [J ] . IEEE Communications Surveys ＆ Tutorials , 2019 , 21 ( 1 ): 393 - 430 .

WANG M W , CUI Y , WANG X , et al . Machine learning for networking,workflow,advances and opportunities [J ] . IEEE Network , 2018 , 32 ( 2 ): 92 - 99 .

ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning,a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .

YAO H P , WANG L Y , WANG X D , et al . The space-terrestrial integrated network,an overview [J ] . IEEE Communications Magazine , 2018 , 56 ( 9 ): 178 - 185 .

安建平 , 李建国 , 于季弘 , 等 . 空天通信网络关键技术综述 [J ] . 电子学报 , 2022 , 50 ( 2 ): 470 - 479 .

AN J P , LI J G , YU J H , et al . Key technologies of space-air-ground communication networks,a survey [J ] . Acta Electronica Sinica , 2022 , 50 ( 2 ): 470 - 479 .

SHI Y P , LIU J J , FADLULLAH Z M , et al . Cross-layer data delivery in satellite-aerial-terrestrial communication [J ] . IEEE Wireless Communications , 2018 , 25 ( 3 ): 138 - 143 .

YAO S , GUAN J F , YAN Z W , et al . SI-STIN,a smart identifier framework for space and terrestrial integrated network [J ] . IEEE Network , 2019 , 33 ( 1 ): 8 - 14 .

徐晖 , 孙韶辉 . 面向6G的天地一体化信息网络架构研究 [J ] . 天地一体化信息网络 , 2021 , 2 ( 4 ): 2 - 9 .

XU H , SUN S H . Research on network architecture for the spaceintegrated-ground information network in 6G [J ] . Space-IntegratedGround Information Networks , 2021 , 2 ( 4 ): 2 - 9 .

BI Y G , HAN G J , XU S , et al . Software defined space-terrestrial integrated networks,architecture,challenges,and solutions [J ] . IEEE Network , 2019 , 33 ( 1 ): 22 - 28 .

ZHANG N , ZHANG S , YANG P , et al . Software defined space-airground integrated vehicular networks,challenges and solutions [J ] . IEEE Communications Magazine , 2017 , 55 ( 7 ): 101 - 109 .

杨丹 , 刘江 , 张然 , 等 . 基于SDN的卫星通信网络,现状、机遇与挑战 [J ] . 天地一体化信息网络 , 2020 ( 2 ): 34 - 41 .

YANG D , LIU J , ZHANG R , et al . SDN-based satellite networks:progress,opportunities and challenges [J ] . Space-Integrated-Ground Information Networks , 2020 ( 2 ): 34 - 41 .

MESTRES A , RODRIGUEZ-NATAL A ,, CARNER J , et al . Knowledge-defined networking [J ] . ACM SIGCOMM Computer Communication Review , 2017 , 47 ( 3 ): 2 - 10 .

STAMPA G , ARIAS M , SANCHEZ-CHARLES D , et al . A deepreinforcement learning approach for software-defined networking routing optimization [EB ] . 2017 .

CHEN J , XIAO Z W , XING H L , et al . STDPG,a spatiotemporal deterministic policy gradient agent for dynamic routing in SDN [C ] // Proceedings of ICC 2020-2020 IEEE International Conference on Communications . Piscataway,IEEE Press , 2020 : 1 - 6 .

HUANG X H , YUAN T T , QIAO G H , et al . Deep reinforcement learning for multimedia traffic control in software defined networking [J ] . IEEE Network , 2018 , 32 ( 6 ): 35 - 41 .

TU Z , ZHOU H C , LI K , et al . A routing optimization method for software-defined SGIN based on deep reinforcement learning [C ] // Proceedings of 2019 IEEE Globecom Workshops . Piscataway,IEEE Press , 2019 : 1 - 6 .

SHI X J , REN P Y , DU Q H . Reinforcement learning routing in space-air-ground integrated networks [C ] // Proceedings of 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway,IEEE Press , 2021 : 1 - 6 .

ZUO P L , WANG C , YAO Z , et al . An intelligent routing algorithm for LEO satellites based on deep reinforcement learning [C ] // Proceedings of 2021 IEEE 94th Vehicular Technology Conference . Piscataway:IEEE Press , 2021 : 1 - 5 .

李新桐 , 张亚生 . 一种适用于低轨卫星的SDN网络人工智能路由方法 [J ] . 电子测量技术 , 2020 , 43 ( 22 ): 109 - 114 .

LI X T , ZHANG Y S . Artificial intelligence routing method for SDN network suitable for LEO satellites [J ] . Electronic Measurement Technology , 2020 , 43 ( 22 ): 109 - 114 .

浏览量

1232

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于移动目标防御的卫星互联网防御系统设计

面向高通量卫星网络的多基带系统载波统一调配技术

卫星互联网应用与挑战

基于低轨卫星互联网的双模通信终端技术

卫星互联网安全风险及应对措施分析