Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent
Abstract Adaptive traffic signal control is the core of the intelligent transportation system (ITS), which can effectively reduce the pressure on traffic congestion and improve travel efficiency. Methods based on deep Q-leaning network (DQN) have become the mainstream to solve single-intersection tr...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01651-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571206856867840 |
|---|---|
| author | Yanliu Zheng Juan Luo Han Gao Yi Zhou Keqin Li |
| author_facet | Yanliu Zheng Juan Luo Han Gao Yi Zhou Keqin Li |
| author_sort | Yanliu Zheng |
| collection | DOAJ |
| description | Abstract Adaptive traffic signal control is the core of the intelligent transportation system (ITS), which can effectively reduce the pressure on traffic congestion and improve travel efficiency. Methods based on deep Q-leaning network (DQN) have become the mainstream to solve single-intersection traffic signal control. However, most of them neglect the important difference of samples and the dependence of traffic states, and cannot quickly respond to randomly changing traffic flows. In this paper, we propose a new single-intersection traffic signal control method (Pri-DDQN) based on reinforcement learning and model the traffic environment as a reinforcement learning environment, and the agent chooses the best action to schedule the traffic flow at the intersection based on the real-time traffic states. With the goal of minimizing the waiting time and queue length at intersections, we use double DQN to train the agent, incorporate traffic state and reward into the loss function, and update the target network parameters asynchronously, to improve the agent’s learning ability. We try to use the power function to dynamically change the exploration rate to accelerate convergence. In addition, we introduce a priority-based dynamic experience replay mechanism to increase the sampling rate of important samples. The results show that Pri-DDQN achieves better performance, compared to the best baseline, it reduces the average queue length is reduced by 13.41%, and the average waiting time by 32.33% at the intersection. |
| format | Article |
| id | doaj-art-ef9144d119a748d7ab8bffae18561139 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-ef9144d119a748d7ab8bffae185611392025-02-02T12:50:16ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111410.1007/s40747-024-01651-5Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agentYanliu Zheng0Juan Luo1Han Gao2Yi Zhou3Keqin Li4College of Computer Science and Electronic Engineering, Hunan UniversityCollege of Computer Science and Electronic Engineering, Hunan UniversityCollege of Computer Science and Electronic Engineering, Hunan UniversityThe School of Artificial Intelligence, Henan UniversityThe Department of Computer Science, State University of New York at New PaltzAbstract Adaptive traffic signal control is the core of the intelligent transportation system (ITS), which can effectively reduce the pressure on traffic congestion and improve travel efficiency. Methods based on deep Q-leaning network (DQN) have become the mainstream to solve single-intersection traffic signal control. However, most of them neglect the important difference of samples and the dependence of traffic states, and cannot quickly respond to randomly changing traffic flows. In this paper, we propose a new single-intersection traffic signal control method (Pri-DDQN) based on reinforcement learning and model the traffic environment as a reinforcement learning environment, and the agent chooses the best action to schedule the traffic flow at the intersection based on the real-time traffic states. With the goal of minimizing the waiting time and queue length at intersections, we use double DQN to train the agent, incorporate traffic state and reward into the loss function, and update the target network parameters asynchronously, to improve the agent’s learning ability. We try to use the power function to dynamically change the exploration rate to accelerate convergence. In addition, we introduce a priority-based dynamic experience replay mechanism to increase the sampling rate of important samples. The results show that Pri-DDQN achieves better performance, compared to the best baseline, it reduces the average queue length is reduced by 13.41%, and the average waiting time by 32.33% at the intersection.https://doi.org/10.1007/s40747-024-01651-5Adaptive traffic signal control (ATSC)Double DQNDecay $$\varepsilon $$ ε -greedyPriority-based experience replayReinforcement learning (RL) |
| spellingShingle | Yanliu Zheng Juan Luo Han Gao Yi Zhou Keqin Li Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent Complex & Intelligent Systems Adaptive traffic signal control (ATSC) Double DQN Decay $$\varepsilon $$ ε -greedy Priority-based experience replay Reinforcement learning (RL) |
| title | Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent |
| title_full | Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent |
| title_fullStr | Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent |
| title_full_unstemmed | Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent |
| title_short | Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent |
| title_sort | pri ddqn learning adaptive traffic signal control strategy through a hybrid agent |
| topic | Adaptive traffic signal control (ATSC) Double DQN Decay $$\varepsilon $$ ε -greedy Priority-based experience replay Reinforcement learning (RL) |
| url | https://doi.org/10.1007/s40747-024-01651-5 |
| work_keys_str_mv | AT yanliuzheng priddqnlearningadaptivetrafficsignalcontrolstrategythroughahybridagent AT juanluo priddqnlearningadaptivetrafficsignalcontrolstrategythroughahybridagent AT hangao priddqnlearningadaptivetrafficsignalcontrolstrategythroughahybridagent AT yizhou priddqnlearningadaptivetrafficsignalcontrolstrategythroughahybridagent AT keqinli priddqnlearningadaptivetrafficsignalcontrolstrategythroughahybridagent |