Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance
In this research, we focus on developing an autonomous system for multiship collision avoidance. The proposed approach combines global path planning based on deep reinforcement learning (DRL) and local motion control to improve computational efficiency and alleviate the sensitivity to heading angle...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Open Journal of Intelligent Transportation Systems |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10587203/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590335403884544 |
|---|---|
| author | Luman Zhao Guoyuan Li Houxiang Zhang |
| author_facet | Luman Zhao Guoyuan Li Houxiang Zhang |
| author_sort | Luman Zhao |
| collection | DOAJ |
| description | In this research, we focus on developing an autonomous system for multiship collision avoidance. The proposed approach combines global path planning based on deep reinforcement learning (DRL) and local motion control to improve computational efficiency and alleviate the sensitivity to heading angle changes. To achieve this, firstly, DRL is used to learn a policy that maps observable states of target ships to a sequence of predicted waypoints. This learning task aims to generate a specific trajectory while avoiding collision with target ships complying with the international regulations for preventing collisions at sea (COLREGs). The learned policy is used as a global path planner during navigation. Secondly, the line-of-sight (LOS) guidance system is applied to calculate the desired course command based on the collision-free trajectory generated according to the policy. Lastly, a model-based control strategy is implemented to control the ship to the specific goal in collision-free space while satisfying the desired commands. We demonstrate the performance of the approach using an example of an autonomous surface vehicle. In comparison to other methods, our proposed control can provide a more stable and smoother maneuvering effect. |
| format | Article |
| id | doaj-art-738f0f9baccc4d098738efe34eccf784 |
| institution | Kabale University |
| issn | 2687-7813 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Intelligent Transportation Systems |
| spelling | doaj-art-738f0f9baccc4d098738efe34eccf7842025-01-24T00:02:41ZengIEEEIEEE Open Journal of Intelligent Transportation Systems2687-78132024-01-01542243210.1109/OJITS.2024.342458710587203Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision AvoidanceLuman Zhao0https://orcid.org/0000-0003-1555-5183Guoyuan Li1https://orcid.org/0000-0001-7553-0899Houxiang Zhang2https://orcid.org/0000-0003-0122-0964Department of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology, Ålesund, NorwayDepartment of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology, Ålesund, NorwayDepartment of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology, Ålesund, NorwayIn this research, we focus on developing an autonomous system for multiship collision avoidance. The proposed approach combines global path planning based on deep reinforcement learning (DRL) and local motion control to improve computational efficiency and alleviate the sensitivity to heading angle changes. To achieve this, firstly, DRL is used to learn a policy that maps observable states of target ships to a sequence of predicted waypoints. This learning task aims to generate a specific trajectory while avoiding collision with target ships complying with the international regulations for preventing collisions at sea (COLREGs). The learned policy is used as a global path planner during navigation. Secondly, the line-of-sight (LOS) guidance system is applied to calculate the desired course command based on the collision-free trajectory generated according to the policy. Lastly, a model-based control strategy is implemented to control the ship to the specific goal in collision-free space while satisfying the desired commands. We demonstrate the performance of the approach using an example of an autonomous surface vehicle. In comparison to other methods, our proposed control can provide a more stable and smoother maneuvering effect.https://ieeexplore.ieee.org/document/10587203/Control application of autonomous systemsdeep reinforcement learningmulti-ship collision avoidanceonline path followingthe international regulations for preventing collisions at sea (COLREGs) |
| spellingShingle | Luman Zhao Guoyuan Li Houxiang Zhang Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance IEEE Open Journal of Intelligent Transportation Systems Control application of autonomous systems deep reinforcement learning multi-ship collision avoidance online path following the international regulations for preventing collisions at sea (COLREGs) |
| title | Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance |
| title_full | Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance |
| title_fullStr | Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance |
| title_full_unstemmed | Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance |
| title_short | Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance |
| title_sort | global and local awareness combine reinforcement learning and model based control for collision avoidance |
| topic | Control application of autonomous systems deep reinforcement learning multi-ship collision avoidance online path following the international regulations for preventing collisions at sea (COLREGs) |
| url | https://ieeexplore.ieee.org/document/10587203/ |
| work_keys_str_mv | AT lumanzhao globalandlocalawarenesscombinereinforcementlearningandmodelbasedcontrolforcollisionavoidance AT guoyuanli globalandlocalawarenesscombinereinforcementlearningandmodelbasedcontrolforcollisionavoidance AT houxiangzhang globalandlocalawarenesscombinereinforcementlearningandmodelbasedcontrolforcollisionavoidance |