Two-stage deep reinforcement learning method for agile optical satellite scheduling problem
Abstract This paper investigates the agile optical satellite scheduling problem, which aims to arrange an observation sequence and observation actions for observation tasks. Existing research mainly aims to maximize the number of completed tasks or the total priorities of the completed tasks but ign...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01667-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571206196264960 |
|---|---|
| author | Zheng Liu Wei Xiong Zhuoya Jia Chi Han |
| author_facet | Zheng Liu Wei Xiong Zhuoya Jia Chi Han |
| author_sort | Zheng Liu |
| collection | DOAJ |
| description | Abstract This paper investigates the agile optical satellite scheduling problem, which aims to arrange an observation sequence and observation actions for observation tasks. Existing research mainly aims to maximize the number of completed tasks or the total priorities of the completed tasks but ignores the influence of the observation actions on the imaging quality. Besides, the conventional exact methods and heuristic methods can hardly obtain a high-quality solution in a short time due to the complicated constraints and considerable solution space of this problem. Thus, this paper proposes a two-stage scheduling framework with two-stage deep reinforcement learning to address this problem. First, the scheduling process is decomposed into a task sequencing stage and an observation scheduling stage, and a mathematical model with complex constraints and two-stage optimization objectives is established to describe the problem. Then, a pointer network with a local selection mechanism and a rough pruning mechanism is constructed as the sequencing network to generate an executable task sequence in the task sequencing stage. Next, a decomposition strategy decomposes the executable task sequence into multiple sub-sequences in the observation scheduling stage, and the observation scheduling process of these sub-sequences is modeled as a concatenated Markov decision process. A neural network is designed as the observation scheduling network to determine observation actions for the sequenced tasks, which is well trained by the soft actor-critic algorithm. Finally, extensive experiments show that the proposed method, along with the designed mechanisms and strategy, is superior to comparison algorithms in terms of solution quality, generalization performance, and computation efficiency. |
| format | Article |
| id | doaj-art-edbd02452dc44fbb9cb4c59c83a904de |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-edbd02452dc44fbb9cb4c59c83a904de2025-02-02T12:49:50ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111112210.1007/s40747-024-01667-xTwo-stage deep reinforcement learning method for agile optical satellite scheduling problemZheng Liu0Wei Xiong1Zhuoya Jia2Chi Han3National Key Laboratory of Space Target Awareness, Space Engineering UniversityNational Key Laboratory of Space Target Awareness, Space Engineering UniversityNational Key Laboratory of Space Target Awareness, Space Engineering UniversityNational Key Laboratory of Space Target Awareness, Space Engineering UniversityAbstract This paper investigates the agile optical satellite scheduling problem, which aims to arrange an observation sequence and observation actions for observation tasks. Existing research mainly aims to maximize the number of completed tasks or the total priorities of the completed tasks but ignores the influence of the observation actions on the imaging quality. Besides, the conventional exact methods and heuristic methods can hardly obtain a high-quality solution in a short time due to the complicated constraints and considerable solution space of this problem. Thus, this paper proposes a two-stage scheduling framework with two-stage deep reinforcement learning to address this problem. First, the scheduling process is decomposed into a task sequencing stage and an observation scheduling stage, and a mathematical model with complex constraints and two-stage optimization objectives is established to describe the problem. Then, a pointer network with a local selection mechanism and a rough pruning mechanism is constructed as the sequencing network to generate an executable task sequence in the task sequencing stage. Next, a decomposition strategy decomposes the executable task sequence into multiple sub-sequences in the observation scheduling stage, and the observation scheduling process of these sub-sequences is modeled as a concatenated Markov decision process. A neural network is designed as the observation scheduling network to determine observation actions for the sequenced tasks, which is well trained by the soft actor-critic algorithm. Finally, extensive experiments show that the proposed method, along with the designed mechanisms and strategy, is superior to comparison algorithms in terms of solution quality, generalization performance, and computation efficiency.https://doi.org/10.1007/s40747-024-01667-xAgile optical satellite schedulingTwo-stage schedulingDeep reinforcement learningTask sequencingObservation scheduling |
| spellingShingle | Zheng Liu Wei Xiong Zhuoya Jia Chi Han Two-stage deep reinforcement learning method for agile optical satellite scheduling problem Complex & Intelligent Systems Agile optical satellite scheduling Two-stage scheduling Deep reinforcement learning Task sequencing Observation scheduling |
| title | Two-stage deep reinforcement learning method for agile optical satellite scheduling problem |
| title_full | Two-stage deep reinforcement learning method for agile optical satellite scheduling problem |
| title_fullStr | Two-stage deep reinforcement learning method for agile optical satellite scheduling problem |
| title_full_unstemmed | Two-stage deep reinforcement learning method for agile optical satellite scheduling problem |
| title_short | Two-stage deep reinforcement learning method for agile optical satellite scheduling problem |
| title_sort | two stage deep reinforcement learning method for agile optical satellite scheduling problem |
| topic | Agile optical satellite scheduling Two-stage scheduling Deep reinforcement learning Task sequencing Observation scheduling |
| url | https://doi.org/10.1007/s40747-024-01667-x |
| work_keys_str_mv | AT zhengliu twostagedeepreinforcementlearningmethodforagileopticalsatelliteschedulingproblem AT weixiong twostagedeepreinforcementlearningmethodforagileopticalsatelliteschedulingproblem AT zhuoyajia twostagedeepreinforcementlearningmethodforagileopticalsatelliteschedulingproblem AT chihan twostagedeepreinforcementlearningmethodforagileopticalsatelliteschedulingproblem |