Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning
In contemporary society, the widespread application of robotics across various domains emphasizes the critical role of robotic systems in performing tasks that are too dangerous or complex for humans. However, individual robots often struggle with these intricate tasks, necessitating the collaborati...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10835107/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832586881835991040 |
|---|---|
| author | Tianyi Gao Zhanlan Li Zhixin Xiong Ling Wen Kai Tian Kewei Cai |
| author_facet | Tianyi Gao Zhanlan Li Zhixin Xiong Ling Wen Kai Tian Kewei Cai |
| author_sort | Tianyi Gao |
| collection | DOAJ |
| description | In contemporary society, the widespread application of robotics across various domains emphasizes the critical role of robotic systems in performing tasks that are too dangerous or complex for humans. However, individual robots often struggle with these intricate tasks, necessitating the collaboration of multi-robot systems. This study proposes a novel deep reinforcement learning (DRL)-based method for achieving queue formation, obstacle avoidance, and navigation in multi-robot systems. The approach includes a specialized state representation and reward mechanism tailored to queue formation, where the state design incorporates the relative positions and actions of robots in the queue to ensure they maintain formation and avoid obstacles. The reward mechanism encourages robots to adjust their speed and position, preserving the queue structure even in complex environments. To address the challenges associated with initial DRL training, we integrate curriculum learning, which facilitates smoother and faster convergence by gradually increasing task difficulty. This approach enables the system to efficiently complete the training process and enhance performance. Upon successful formation training, a speed control mechanism is introduced to improve the stability of the queue during navigation, ensuring robots maintain uniform speed and formation while avoiding obstacles. Simulation results demonstrate the effectiveness of the proposed method in managing queue formations across various robot numbers and environmental complexities. The results highlight robust performance with up to 10 robots, maintaining effectiveness in both static and dynamic environments. However, performance limitations are observed with larger fleets, indicating the need for further advancements in collision avoidance and speed control. |
| format | Article |
| id | doaj-art-b03ecb3ab2494695aa3f88e1f07bda05 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-b03ecb3ab2494695aa3f88e1f07bda052025-01-25T00:02:22ZengIEEEIEEE Access2169-35362025-01-0113140831410010.1109/ACCESS.2025.352764010835107Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement LearningTianyi Gao0https://orcid.org/0009-0003-7599-3326Zhanlan Li1https://orcid.org/0009-0007-7101-4118Zhixin Xiong2https://orcid.org/0009-0007-7123-3361Ling Wen3https://orcid.org/0009-0001-5993-134XKai Tian4https://orcid.org/0009-0001-1365-4498Kewei Cai5https://orcid.org/0000-0003-4144-3898School of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian, ChinaSchool of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian, ChinaSchool of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian, ChinaSchool of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian, ChinaSchool of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian, ChinaSchool of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian, ChinaIn contemporary society, the widespread application of robotics across various domains emphasizes the critical role of robotic systems in performing tasks that are too dangerous or complex for humans. However, individual robots often struggle with these intricate tasks, necessitating the collaboration of multi-robot systems. This study proposes a novel deep reinforcement learning (DRL)-based method for achieving queue formation, obstacle avoidance, and navigation in multi-robot systems. The approach includes a specialized state representation and reward mechanism tailored to queue formation, where the state design incorporates the relative positions and actions of robots in the queue to ensure they maintain formation and avoid obstacles. The reward mechanism encourages robots to adjust their speed and position, preserving the queue structure even in complex environments. To address the challenges associated with initial DRL training, we integrate curriculum learning, which facilitates smoother and faster convergence by gradually increasing task difficulty. This approach enables the system to efficiently complete the training process and enhance performance. Upon successful formation training, a speed control mechanism is introduced to improve the stability of the queue during navigation, ensuring robots maintain uniform speed and formation while avoiding obstacles. Simulation results demonstrate the effectiveness of the proposed method in managing queue formations across various robot numbers and environmental complexities. The results highlight robust performance with up to 10 robots, maintaining effectiveness in both static and dynamic environments. However, performance limitations are observed with larger fleets, indicating the need for further advancements in collision avoidance and speed control.https://ieeexplore.ieee.org/document/10835107/Deep reinforcement learningmulti-robotobstacle avoidance |
| spellingShingle | Tianyi Gao Zhanlan Li Zhixin Xiong Ling Wen Kai Tian Kewei Cai Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning IEEE Access Deep reinforcement learning multi-robot obstacle avoidance |
| title | Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning |
| title_full | Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning |
| title_fullStr | Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning |
| title_full_unstemmed | Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning |
| title_short | Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning |
| title_sort | queue formation and obstacle avoidance navigation strategy for multi robot systems based on deep reinforcement learning |
| topic | Deep reinforcement learning multi-robot obstacle avoidance |
| url | https://ieeexplore.ieee.org/document/10835107/ |
| work_keys_str_mv | AT tianyigao queueformationandobstacleavoidancenavigationstrategyformultirobotsystemsbasedondeepreinforcementlearning AT zhanlanli queueformationandobstacleavoidancenavigationstrategyformultirobotsystemsbasedondeepreinforcementlearning AT zhixinxiong queueformationandobstacleavoidancenavigationstrategyformultirobotsystemsbasedondeepreinforcementlearning AT lingwen queueformationandobstacleavoidancenavigationstrategyformultirobotsystemsbasedondeepreinforcementlearning AT kaitian queueformationandobstacleavoidancenavigationstrategyformultirobotsystemsbasedondeepreinforcementlearning AT keweicai queueformationandobstacleavoidancenavigationstrategyformultirobotsystemsbasedondeepreinforcementlearning |