基于DQN的低轨卫星网络多目标智能路由算法

罗宗屹; 金世超; 董涛; 殷杰

doi:10.11959/j.issn.2096-8930.2025003

您当前的位置：

首页 >

文章列表页 >

基于DQN的低轨卫星网络多目标智能路由算法

研究 | 更新时间：2025-04-10

- 基于DQN的低轨卫星网络多目标智能路由算法
- Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN
- 天地一体化信息网络 2025年6卷第1期页码：16-23
- 作者机构：
  
  1.航天恒星科技有限公司天地一体化信息技术国家重点实验室，北京 100095
  2.北京卫星信息工程研究所，北京 100095
- 作者简介：
  
  [ "罗宗屹（1998- ），男，硕士生，主要研究方向为卫星网络。" ]
  [ "金世超（1980- ），男，博士，航天恒星科技有限公司业务副总经理，研究员，主要研究方向为卫星通信系统与装备。" ]
  [ "董涛（1975- ），男，博士，航天恒星科技有限公司研究员，主要研究方向为卫星通信和网络。" ]
  [ "殷杰（1983- ），男，博士，航天恒星科技有限公司研究员，主要研究方向为卫星通信网络。" ]
- 基金信息：
  
  国家重点研发计划资助项目(2022YFB2902504)
- DOI：10.11959/j.issn.2096-8930.2025003
  中图分类号： TP393
- 收稿：2024-11-11，
  
  修回：2025-02-10，
  
  纸质出版：2025-03-20
- 稿件说明：
移动端阅览
罗宗屹,金世超,董涛等.基于DQN的低轨卫星网络多目标智能路由算法[J].天地一体化信息网络,2025,06(01):16-23.

LUO Zongyi,JIN Shichao,DONG Tao,et al.Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN[J].Space-Integrated-Ground Information Networks,2025,06(01):16-23.
罗宗屹,金世超,董涛等.基于DQN的低轨卫星网络多目标智能路由算法[J].天地一体化信息网络,2025,06(01):16-23. DOI： 10.11959/j.issn.2096-8930.2025003.

LUO Zongyi,JIN Shichao,DONG Tao,et al.Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN[J].Space-Integrated-Ground Information Networks,2025,06(01):16-23. DOI： 10.11959/j.issn.2096-8930.2025003.

摘要

近年来，低轨卫星网络发展迅速，被广泛应用于全球通信、互联网接入等领域。首先，对当前低轨卫星网络发展现状与主流的低轨网络路由算法进行总结。然后，针对星上路由算法设计面临的优化目标单一、收敛速度慢以及时延和丢包率等指标优化难度大等挑战，提出基于深度Q网络（DQN）的多目标智能路由算法，通过设计多目标奖励函数，实现了时延、丢包率、负载均衡多目标的性能优化。在时延方面，本算法优于最短通路优先算法；在流量分布指数和丢包率方面，本算法相较于最短通路优先算法与Dijkstra算法分别提高14 %和10 %以上。最后，阐述所提智能路由算法在多种应用场景中的使用方法。

Abstract

In recent years

LEO satellite networks have developed rapidly and are widely used in global communication and internet access. This paper first summarized the current development status of LEO networks and the main LEO network routing algorithms. At the same time

aimed at the challenges of single optimization objectives

slow convergence speed

and difficulty in optimizing indicators such as delay and packet loss

this paper proposed a multi-objective intelligent routing algorithm based on deep Q-network (DQN). With the designed DQN multi-objective reward function

it realized the optimization of delay

packet loss

load balancing. Regarding delay

the proposed algorithm is superior to the Shortest Path First (SPF) algorithm. In terms of traffic distribution index and packet loss rate

the performances of the proposed algorithm are improved by more than 14% and 10% compared with the SPF and Dijkstra algorithms respectively. In addition

this paper illustrated the application methods of the proposed intelligent routing algorithm in various scenarios.

关键词

Keywords

references

CHEN S Z , SUN S H , KANG S L . System integration of terrestrial mobile communication and satellite communication: the trends, challenges and key technologies in B5G and 6G [J ] . China Communications , 2020 , 17 ( 12 ): 156 - 171 .

刘帅军 , 徐帆江 , 刘立祥 , 等 . Starlink星座卫星入轨预测与仿真分析 [EB ] . 2023 .

LIU S J , XU F J , LIU L X , et al . Orbit prediction and simulation analysis of Starlink constellation satellites [EB ] . 2023 .

刘帅军 , 徐帆江 , 刘立祥 , 等 . Starlink二代GEN2系统混合组网与分阶段部署探讨 [EB ] . 2023 .

LIU S J , XU F J , LIU L X , et al . Discussion on hybrid networking and phased deployment of Starlink second generation GEN2 system [EB ] . 2023 .

王学宇 , 武坦然 . OneWeb低轨道卫星系统及其军事应用分析 [J ] . 航天电子对抗 , 2022 , 38 ( 4 ): 59 - 64 .

WANG X Y , WU T R . Analysis on OneWeb LEO satellite system and its military application [J ] . Aerospace Electronic Warfare , 2022 , 38 ( 4 ): 59 - 64 .

QI X G , MA J L , WU D , et al . A survey of routing techniques for satellite networks [J ] . Journal of Communications and Information Networks , 2016 , 1 ( 4 ): 66 - 85 .

WERNER M , DELUCCHI C , VOGEL H J , et al . ATM-based routing in LEO/MEO satellite networks with intersatellite links [J ] . IEEE Journal on Selected Areas in Communications , 1997 , 15 ( 1 ): 69 - 82 .

CHANG H S , KIM B W , LEE C G , et al . FSA-based link assignment and routing in low-earth orbit satellite networks [J ] . IEEE Transactions on Vehicular Technology , 1998 , 47 ( 3 ): 1037 - 1048 .

TAN H C , ZHU L D . A novel routing algorithm based on virtual topology snapshot in LEO satellite networks [C ] // Proceedings of the 2014 IEEE 17th International Conference on Computational Science and Engineering . Piscataway : IEEE Press , 2014 : 357 - 361 .

EKICI E , AKYILDIZ I F , BENDER M D . A distributed routing algorithm for datagram traffic in LEO satellite networks [J ] . IEEE/ACM Transactions on Networking , 2002 , 9 ( 2 ): 137 - 147 .

LIU X M , JIANG Z Q , LIU C H , et al . A low-complexity probabilistic routing algorithm for polar orbits satellite constellation networks [C ] // Proceedings of the 2015 IEEE/CIC International Conference on Communications in China (ICCC) . Piscataway : IEEE Press , 2015 : 1 - 5 .

NA Z Y , PAN Z , LIU X , et al . Distributed routing strategy based on machine learning for LEO satellite network [J ] . Wireless Communications and Mobile Computing , 2018 ( 1 ): 3026405 .

WANG X T , DAI Z Q , XU Z . LEO satellite network routing algorithm based on reinforcement learning [C ] // Proceedings of the 2021 IEEE 4th International Conference on Electronics Technology (ICET) . Piscataway : IEEE Press , 2021 : 1105 - 1109 .

MNIH V . Playing atari with deep reinforcement learning [J ] . arxiv preprint arxiv , 2013 .

YOU X Y , LI X J , XU Y D , et al . Toward packet routing with fully distributed multiagent deep reinforcement learning [J ] . IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2020 , 52 ( 2 ): 855 - 868 .

BHAVANASI S S , PAPPONE L , ESPOSITO F . Routing with graph convolutional networks and multi-agent deep reinforcement learning [C ] // Proceedings of the 2022 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN) . Piscataway : IEEE Press , 2022 : 72 - 77 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

SHORTLE J F , THOMPSON J M , GROSS D , et al . Fundamentals of queueing theory [M ] . Hoboken : John Wiley＆Sons , 2018 .

TALEB T , MASHIMO D , JAMALIPOUR A , et al . Explicit load balancing technique for NGEO satellite IP networks with on-board processing capabilities [J ] . IEEE/ACM Transactions on Networking , 2009 , 17 ( 1 ): 281 - 293 .

浏览量

206

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

混合激光/微波链路的导航星座拓扑管理与路由研究

基于分簇的甚低轨卫星通信系统馈电链路路由算法

基于多普勒频移的低轨卫星终端定位技术

低轨巨型星座组网：挑战与关键技术

基于遗传优化和人工蜂群的低轨卫星网络负载均衡路由算法