Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN

LUO Zongyi; JIN Shichao; DONG Tao; YIN Jie

doi:10.11959/j.issn.2096-8930.2025003

您当前的位置：

首页 >

文章列表页 >

Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN

Studies | 更新时间：2025-04-10

- Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN
- Space-Integrated-Ground Information Networks Vol. 6, Issue 1, Pages: 16-23(2025)
- 作者机构：
  
  1.航天恒星科技有限公司天地一体化信息技术国家重点实验室，北京 100095
  2.北京卫星信息工程研究所，北京 100095
- 作者简介：
- 基金信息：
  
  National Key Research and Development Program of China(2022YFB2902504)
- DOI：10.11959/j.issn.2096-8930.2025003
  CLC： TP393
- Received：11 November 2024，
  
  Revised：2025-02-10，
  
  Published：20 March 2025
- 稿件说明：
移动端阅览
罗宗屹,金世超,董涛等.基于DQN的低轨卫星网络多目标智能路由算法[J].天地一体化信息网络,2025,06(01):16-23.

LUO Zongyi,JIN Shichao,DONG Tao,et al.Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN[J].Space-Integrated-Ground Information Networks,2025,06(01):16-23.
罗宗屹,金世超,董涛等.基于DQN的低轨卫星网络多目标智能路由算法[J].天地一体化信息网络,2025,06(01):16-23. DOI： 10.11959/j.issn.2096-8930.2025003.

LUO Zongyi,JIN Shichao,DONG Tao,et al.Multi-Objective Intelligent Routing Algorithm for LEO Satellite Networks Based on DQN[J].Space-Integrated-Ground Information Networks,2025,06(01):16-23. DOI： 10.11959/j.issn.2096-8930.2025003.

摘要

近年来，低轨卫星网络发展迅速，被广泛应用于全球通信、互联网接入等领域。首先，对当前低轨卫星网络发展现状与主流的低轨网络路由算法进行总结。然后，针对星上路由算法设计面临的优化目标单一、收敛速度慢以及时延和丢包率等指标优化难度大等挑战，提出基于深度Q网络（DQN）的多目标智能路由算法，通过设计多目标奖励函数，实现了时延、丢包率、负载均衡多目标的性能优化。在时延方面，本算法优于最短通路优先算法；在流量分布指数和丢包率方面，本算法相较于最短通路优先算法与Dijkstra算法分别提高14 %和10 %以上。最后，阐述所提智能路由算法在多种应用场景中的使用方法。

Abstract

In recent years

LEO satellite networks have developed rapidly and are widely used in global communication and internet access. This paper first summarized the current development status of LEO networks and the main LEO network routing algorithms. At the same time

aimed at the challenges of single optimization objectives

slow convergence speed

and difficulty in optimizing indicators such as delay and packet loss

this paper proposed a multi-objective intelligent routing algorithm based on deep Q-network (DQN). With the designed DQN multi-objective reward function

it realized the optimization of delay

packet loss

load balancing. Regarding delay

the proposed algorithm is superior to the Shortest Path First (SPF) algorithm. In terms of traffic distribution index and packet loss rate

the performances of the proposed algorithm are improved by more than 14% and 10% compared with the SPF and Dijkstra algorithms respectively. In addition

this paper illustrated the application methods of the proposed intelligent routing algorithm in various scenarios.

关键词

Keywords

references

CHEN S Z , SUN S H , KANG S L . System integration of terrestrial mobile communication and satellite communication: the trends, challenges and key technologies in B5G and 6G [J ] . China Communications , 2020 , 17 ( 12 ): 156 - 171 .

刘帅军 , 徐帆江 , 刘立祥 , 等 . Starlink星座卫星入轨预测与仿真分析 [EB ] . 2023 .

LIU S J , XU F J , LIU L X , et al . Orbit prediction and simulation analysis of Starlink constellation satellites [EB ] . 2023 .

刘帅军 , 徐帆江 , 刘立祥 , 等 . Starlink二代GEN2系统混合组网与分阶段部署探讨 [EB ] . 2023 .

LIU S J , XU F J , LIU L X , et al . Discussion on hybrid networking and phased deployment of Starlink second generation GEN2 system [EB ] . 2023 .

王学宇 , 武坦然 . OneWeb低轨道卫星系统及其军事应用分析 [J ] . 航天电子对抗 , 2022 , 38 ( 4 ): 59 - 64 .

WANG X Y , WU T R . Analysis on OneWeb LEO satellite system and its military application [J ] . Aerospace Electronic Warfare , 2022 , 38 ( 4 ): 59 - 64 .

QI X G , MA J L , WU D , et al . A survey of routing techniques for satellite networks [J ] . Journal of Communications and Information Networks , 2016 , 1 ( 4 ): 66 - 85 .

WERNER M , DELUCCHI C , VOGEL H J , et al . ATM-based routing in LEO/MEO satellite networks with intersatellite links [J ] . IEEE Journal on Selected Areas in Communications , 1997 , 15 ( 1 ): 69 - 82 .

CHANG H S , KIM B W , LEE C G , et al . FSA-based link assignment and routing in low-earth orbit satellite networks [J ] . IEEE Transactions on Vehicular Technology , 1998 , 47 ( 3 ): 1037 - 1048 .

TAN H C , ZHU L D . A novel routing algorithm based on virtual topology snapshot in LEO satellite networks [C ] // Proceedings of the 2014 IEEE 17th International Conference on Computational Science and Engineering . Piscataway : IEEE Press , 2014 : 357 - 361 .

EKICI E , AKYILDIZ I F , BENDER M D . A distributed routing algorithm for datagram traffic in LEO satellite networks [J ] . IEEE/ACM Transactions on Networking , 2002 , 9 ( 2 ): 137 - 147 .

LIU X M , JIANG Z Q , LIU C H , et al . A low-complexity probabilistic routing algorithm for polar orbits satellite constellation networks [C ] // Proceedings of the 2015 IEEE/CIC International Conference on Communications in China (ICCC) . Piscataway : IEEE Press , 2015 : 1 - 5 .

NA Z Y , PAN Z , LIU X , et al . Distributed routing strategy based on machine learning for LEO satellite network [J ] . Wireless Communications and Mobile Computing , 2018 ( 1 ): 3026405 .

WANG X T , DAI Z Q , XU Z . LEO satellite network routing algorithm based on reinforcement learning [C ] // Proceedings of the 2021 IEEE 4th International Conference on Electronics Technology (ICET) . Piscataway : IEEE Press , 2021 : 1105 - 1109 .

MNIH V . Playing atari with deep reinforcement learning [J ] . arxiv preprint arxiv , 2013 .

YOU X Y , LI X J , XU Y D , et al . Toward packet routing with fully distributed multiagent deep reinforcement learning [J ] . IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2020 , 52 ( 2 ): 855 - 868 .

BHAVANASI S S , PAPPONE L , ESPOSITO F . Routing with graph convolutional networks and multi-agent deep reinforcement learning [C ] // Proceedings of the 2022 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN) . Piscataway : IEEE Press , 2022 : 72 - 77 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

SHORTLE J F , THOMPSON J M , GROSS D , et al . Fundamentals of queueing theory [M ] . Hoboken : John Wiley＆Sons , 2018 .

TALEB T , MASHIMO D , JAMALIPOUR A , et al . Explicit load balancing technique for NGEO satellite IP networks with on-board processing capabilities [J ] . IEEE/ACM Transactions on Networking , 2009 , 17 ( 1 ): 281 - 293 .

Views

206

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on Topology Management and Routing for Navigation Constellations with Hybrid Laser/Microwave Links

Feed Link Routing Algorithm for Cluster-Based Very Low Orbit Satellite Communication Systems

Doppler Shift Based Low Earth Orbit Satellite Terminal Positioning Technology

Mega LEO Satellite Constellations Networking: Challenges and Key Technologies

Load Balancing Routing Algorithm Based on Genetic Optimization and Artificial Bee Colony in LEO Satellite Network

Related Author

Song Kun

Li Li

Liu Yuqi

Chen Lin

Li Xintong

Zhu Lidong

Zhang Xinyi

LI Xin

Related Institution

National Key Laboratory of Wrieless Communications, University of Electronic Science and Technology of China

Southwest China Research Institute of Electronic Equipment

Chang Guang Satellite Technology Co., Ltd.

Shanghai Satellite Network Research Institute Co.,Ltd.

Shanghai Key Laboratory of Satellite Network

⁰