Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning
The merging behavior of vehicles at entry ramps and the speed differences between ramps and mainline traffic cause merging traffic bottlenecks. Current research, primarily focusing on single traffic control strategies, fails to achieve the desired outcomes. To address this issue, this paper explores...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/2/836 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589204311244800 |
|---|---|
| author | Juan Du Anshuang Yu Hao Zhou Qianli Jiang Xueying Bai |
| author_facet | Juan Du Anshuang Yu Hao Zhou Qianli Jiang Xueying Bai |
| author_sort | Juan Du |
| collection | DOAJ |
| description | The merging behavior of vehicles at entry ramps and the speed differences between ramps and mainline traffic cause merging traffic bottlenecks. Current research, primarily focusing on single traffic control strategies, fails to achieve the desired outcomes. To address this issue, this paper explores an integrated control strategy combining Variable Speed Limits (VSL) and Lane Change Control (LCC) to optimize traffic efficiency in ramp merging areas. For scenarios involving multiple ramp merges, a multi-agent reinforcement learning approach is introduced to optimize control strategies in these areas. An integrated control system based on the Factored Multi-Agent Centralized Policy Gradients (FACMAC) algorithm is developed. By transforming the control framework into a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), state and action spaces for heterogeneous agents are designed. These agents dynamically adjust control strategies and control area lengths based on real-time traffic conditions, adapting to the changing traffic environment. The proposed Factored Multi-Agent Centralized Policy Gradients for Integrated Traffic Control in Dynamic Areas (FM-ITC-Darea) control strategy is simulated and tested on a multi-ramp scenario built on a multi-lane Cell Transmission Model (CTM) simulation platform. Comparisons are made with no control and Factored Multi-Agent Centralized Policy Gradients for Integrated Traffic Control (FM-ITC) strategies, demonstrating the effectiveness of the proposed integrated control strategy in alleviating highway ramp merging bottlenecks. |
| format | Article |
| id | doaj-art-9fbab4e357674632a9d05dbcf7884e05 |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-9fbab4e357674632a9d05dbcf7884e052025-01-24T13:21:00ZengMDPI AGApplied Sciences2076-34172025-01-0115283610.3390/app15020836Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement LearningJuan Du0Anshuang Yu1Hao Zhou2Qianli Jiang3Xueying Bai4SILC Business School, Shanghai University, Shanghai 201800, ChinaSILC Business School, Shanghai University, Shanghai 201800, ChinaSILC Business School, Shanghai University, Shanghai 201800, ChinaSILC Business School, Shanghai University, Shanghai 201800, ChinaSILC Business School, Shanghai University, Shanghai 201800, ChinaThe merging behavior of vehicles at entry ramps and the speed differences between ramps and mainline traffic cause merging traffic bottlenecks. Current research, primarily focusing on single traffic control strategies, fails to achieve the desired outcomes. To address this issue, this paper explores an integrated control strategy combining Variable Speed Limits (VSL) and Lane Change Control (LCC) to optimize traffic efficiency in ramp merging areas. For scenarios involving multiple ramp merges, a multi-agent reinforcement learning approach is introduced to optimize control strategies in these areas. An integrated control system based on the Factored Multi-Agent Centralized Policy Gradients (FACMAC) algorithm is developed. By transforming the control framework into a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), state and action spaces for heterogeneous agents are designed. These agents dynamically adjust control strategies and control area lengths based on real-time traffic conditions, adapting to the changing traffic environment. The proposed Factored Multi-Agent Centralized Policy Gradients for Integrated Traffic Control in Dynamic Areas (FM-ITC-Darea) control strategy is simulated and tested on a multi-ramp scenario built on a multi-lane Cell Transmission Model (CTM) simulation platform. Comparisons are made with no control and Factored Multi-Agent Centralized Policy Gradients for Integrated Traffic Control (FM-ITC) strategies, demonstrating the effectiveness of the proposed integrated control strategy in alleviating highway ramp merging bottlenecks.https://www.mdpi.com/2076-3417/15/2/836variable speed limitslane change controlmulti-agent reinforcement learninghighway merging bottlenecksintegrated control strategy |
| spellingShingle | Juan Du Anshuang Yu Hao Zhou Qianli Jiang Xueying Bai Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning Applied Sciences variable speed limits lane change control multi-agent reinforcement learning highway merging bottlenecks integrated control strategy |
| title | Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning |
| title_full | Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning |
| title_fullStr | Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning |
| title_full_unstemmed | Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning |
| title_short | Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning |
| title_sort | research on integrated control strategy for highway merging bottlenecks based on collaborative multi agent reinforcement learning |
| topic | variable speed limits lane change control multi-agent reinforcement learning highway merging bottlenecks integrated control strategy |
| url | https://www.mdpi.com/2076-3417/15/2/836 |
| work_keys_str_mv | AT juandu researchonintegratedcontrolstrategyforhighwaymergingbottlenecksbasedoncollaborativemultiagentreinforcementlearning AT anshuangyu researchonintegratedcontrolstrategyforhighwaymergingbottlenecksbasedoncollaborativemultiagentreinforcementlearning AT haozhou researchonintegratedcontrolstrategyforhighwaymergingbottlenecksbasedoncollaborativemultiagentreinforcementlearning AT qianlijiang researchonintegratedcontrolstrategyforhighwaymergingbottlenecksbasedoncollaborativemultiagentreinforcementlearning AT xueyingbai researchonintegratedcontrolstrategyforhighwaymergingbottlenecksbasedoncollaborativemultiagentreinforcementlearning |