Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting
As an essential part of intelligent transportation system (ITS), traffic forecasting has provided crucial role for traffic management and risk assessment. However, complex spatial–temporal dependencies, heterogeneity, dynamicity, and periodicity of traffic data influence the traffic forecasting perf...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-01-01
|
| Series: | IET Signal Processing |
| Online Access: | http://dx.doi.org/10.1049/2024/8639981 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832546174519738368 |
|---|---|
| author | Yongpeng Yang Zhenzhen Yang Zhen Yang |
| author_facet | Yongpeng Yang Zhenzhen Yang Zhen Yang |
| author_sort | Yongpeng Yang |
| collection | DOAJ |
| description | As an essential part of intelligent transportation system (ITS), traffic forecasting has provided crucial role for traffic management and risk assessment. However, complex spatial–temporal dependencies, heterogeneity, dynamicity, and periodicity of traffic data influence the traffic forecasting performance. Consequently, we propose a novel effective gated spatial–temporal merged transformer (GSTMT) inspired by multimask and dual branch for accurate traffic forecasting in this paper. Specifically, we first conduct a concatenation of gated spatial static mask transformer (GSSMT) and gated spatial dynamic mask transformer (GSDMT) with residual network. The GSSMT and GSDMT evolve from the traditional transformer by making preferable modifications that include gated linear unit (GLU), multimask mechanism including static mask matrix (SMM) and dynamic mask matrix (DMM), and spatial attention (SA). Among them, GLU is to promote the performance of capturing spatial dependency, dynamicity, and heterogeneity due to advanced performance for controlling information flow through layers. Additionally, by developing multimask mechanism including two novel SMM and DMM, the proposed GSTMT can precisely model the static and dynamic spatial structure for effectively highlighting static dependency and dynamicity. And SA is injected for enhancing the ability of capturing spatial dependency of GSSMT and GSDMT. Secondly, we develop a dual-branch gated temporal transformer (DBGTT) for capturing temporal dependency, heterogeneity, dynamicity, and periodicity via incorporating the GLU and mixed time series decomposition (MTD) into traditional transformer. Similarly, we also introduce the GLU for empowering DBGTT with capability of capturing temporal dependency, dynamicity, and heterogeneity. In addition, MTD, which brings dual-branch mechanism, can enhance the DBGTT for capturing more detailed temporal information via exploiting global and periodic profile of traffic data. At last, some experiments, which are performed on several real-world traffic datasets, demonstrate the better results over classic traffic forecasting methods. |
| format | Article |
| id | doaj-art-06ca53a1df934c6e96564d4977471c72 |
| institution | Kabale University |
| issn | 1751-9683 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Signal Processing |
| spelling | doaj-art-06ca53a1df934c6e96564d4977471c722025-02-03T07:23:36ZengWileyIET Signal Processing1751-96832024-01-01202410.1049/2024/8639981Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic ForecastingYongpeng Yang0Zhenzhen Yang1Zhen Yang2Key Laboratory of Ministry of Education in Broadband Wireless Communication and Sensor Network TechnologyKey Laboratory of Ministry of Education in Broadband Wireless Communication and Sensor Network TechnologyKey Laboratory of Ministry of Education in Broadband Wireless Communication and Sensor Network TechnologyAs an essential part of intelligent transportation system (ITS), traffic forecasting has provided crucial role for traffic management and risk assessment. However, complex spatial–temporal dependencies, heterogeneity, dynamicity, and periodicity of traffic data influence the traffic forecasting performance. Consequently, we propose a novel effective gated spatial–temporal merged transformer (GSTMT) inspired by multimask and dual branch for accurate traffic forecasting in this paper. Specifically, we first conduct a concatenation of gated spatial static mask transformer (GSSMT) and gated spatial dynamic mask transformer (GSDMT) with residual network. The GSSMT and GSDMT evolve from the traditional transformer by making preferable modifications that include gated linear unit (GLU), multimask mechanism including static mask matrix (SMM) and dynamic mask matrix (DMM), and spatial attention (SA). Among them, GLU is to promote the performance of capturing spatial dependency, dynamicity, and heterogeneity due to advanced performance for controlling information flow through layers. Additionally, by developing multimask mechanism including two novel SMM and DMM, the proposed GSTMT can precisely model the static and dynamic spatial structure for effectively highlighting static dependency and dynamicity. And SA is injected for enhancing the ability of capturing spatial dependency of GSSMT and GSDMT. Secondly, we develop a dual-branch gated temporal transformer (DBGTT) for capturing temporal dependency, heterogeneity, dynamicity, and periodicity via incorporating the GLU and mixed time series decomposition (MTD) into traditional transformer. Similarly, we also introduce the GLU for empowering DBGTT with capability of capturing temporal dependency, dynamicity, and heterogeneity. In addition, MTD, which brings dual-branch mechanism, can enhance the DBGTT for capturing more detailed temporal information via exploiting global and periodic profile of traffic data. At last, some experiments, which are performed on several real-world traffic datasets, demonstrate the better results over classic traffic forecasting methods.http://dx.doi.org/10.1049/2024/8639981 |
| spellingShingle | Yongpeng Yang Zhenzhen Yang Zhen Yang Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting IET Signal Processing |
| title | Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting |
| title_full | Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting |
| title_fullStr | Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting |
| title_full_unstemmed | Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting |
| title_short | Gated Spatial–Temporal Merged Transformer Inspired by Multimask and Dual Branch for Traffic Forecasting |
| title_sort | gated spatial temporal merged transformer inspired by multimask and dual branch for traffic forecasting |
| url | http://dx.doi.org/10.1049/2024/8639981 |
| work_keys_str_mv | AT yongpengyang gatedspatialtemporalmergedtransformerinspiredbymultimaskanddualbranchfortrafficforecasting AT zhenzhenyang gatedspatialtemporalmergedtransformerinspiredbymultimaskanddualbranchfortrafficforecasting AT zhenyang gatedspatialtemporalmergedtransformerinspiredbymultimaskanddualbranchfortrafficforecasting |