STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection
Convolutional neural networks and attention mechanisms have greatly benefited remote sensing change detection (RSCD) because of their outstanding discriminative ability. Existent RSCD methods often follow a paradigm of using a noninteractive Siamese neural network for multitemporal feature extractio...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10815617/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590426369949696 |
|---|---|
| author | Xiaowen Ma Zhenkai Wu Mengting Ma Mengjiao Zhao Fan Yang Zhenhong Du Wei Zhang |
| author_facet | Xiaowen Ma Zhenkai Wu Mengting Ma Mengjiao Zhao Fan Yang Zhenhong Du Wei Zhang |
| author_sort | Xiaowen Ma |
| collection | DOAJ |
| description | Convolutional neural networks and attention mechanisms have greatly benefited remote sensing change detection (RSCD) because of their outstanding discriminative ability. Existent RSCD methods often follow a paradigm of using a noninteractive Siamese neural network for multitemporal feature extraction and change detection heads for feature fusion and change representation. However, this paradigm lacks the contemplation of the characteristics of RSCD in temporal and spatial dimensions, and causes the drawback on spatial–temporal interaction that hinders high-quality feature extraction. To address this problem, we present a spatial–temporal interaction Transformer architecture for multitemporal feature extraction, which is the first general backbone network specifically designed for RSCD. In addition, we propose a parameter-free multifrequency token mixer to integrate frequency-domain features that provide spectral information for RSCD. Experimental results on three datasets validate the effectiveness of the proposed method, which can outperform the state-of-the-art methods and achieve the most satisfactory efficiency-accuracy tradeoff. |
| format | Article |
| id | doaj-art-cb88eafcfe2044a7b96e19a49e56aad4 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-cb88eafcfe2044a7b96e19a49e56aad42025-01-24T00:00:57ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01183735374510.1109/JSTARS.2024.352232910815617STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change DetectionXiaowen Ma0https://orcid.org/0000-0001-5031-2641Zhenkai Wu1https://orcid.org/0009-0000-0613-0584Mengting Ma2https://orcid.org/0000-0002-6897-3576Mengjiao Zhao3https://orcid.org/0009-0006-1814-2404Fan Yang4https://orcid.org/0009-0002-1292-0894Zhenhong Du5https://orcid.org/0000-0001-9449-0415Wei Zhang6https://orcid.org/0000-0002-4424-079XSchool of Software Technology, Zhejiang University, Hangzhou, ChinaSchool of Software Technology, Zhejiang University, Hangzhou, ChinaSchool of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaSchool of Software Technology, Zhejiang University, Hangzhou, ChinaSchool of Software Technology, Zhejiang University, Hangzhou, ChinaSchool of Earth Sciences, Zhejiang University, Hangzhou, ChinaSchool of Software Technology, Zhejiang University, Hangzhou, ChinaConvolutional neural networks and attention mechanisms have greatly benefited remote sensing change detection (RSCD) because of their outstanding discriminative ability. Existent RSCD methods often follow a paradigm of using a noninteractive Siamese neural network for multitemporal feature extraction and change detection heads for feature fusion and change representation. However, this paradigm lacks the contemplation of the characteristics of RSCD in temporal and spatial dimensions, and causes the drawback on spatial–temporal interaction that hinders high-quality feature extraction. To address this problem, we present a spatial–temporal interaction Transformer architecture for multitemporal feature extraction, which is the first general backbone network specifically designed for RSCD. In addition, we propose a parameter-free multifrequency token mixer to integrate frequency-domain features that provide spectral information for RSCD. Experimental results on three datasets validate the effectiveness of the proposed method, which can outperform the state-of-the-art methods and achieve the most satisfactory efficiency-accuracy tradeoff.https://ieeexplore.ieee.org/document/10815617/Change detectioncross-spatial interactioncross-temporal interactionmultifrequency |
| spellingShingle | Xiaowen Ma Zhenkai Wu Mengting Ma Mengjiao Zhao Fan Yang Zhenhong Du Wei Zhang STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Change detection cross-spatial interaction cross-temporal interaction multifrequency |
| title | STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection |
| title_full | STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection |
| title_fullStr | STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection |
| title_full_unstemmed | STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection |
| title_short | STeInFormer: Spatial–Temporal Interaction Transformer Architecture for Remote Sensing Change Detection |
| title_sort | steinformer spatial x2013 temporal interaction transformer architecture for remote sensing change detection |
| topic | Change detection cross-spatial interaction cross-temporal interaction multifrequency |
| url | https://ieeexplore.ieee.org/document/10815617/ |
| work_keys_str_mv | AT xiaowenma steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection AT zhenkaiwu steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection AT mengtingma steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection AT mengjiaozhao steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection AT fanyang steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection AT zhenhongdu steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection AT weizhang steinformerspatialx2013temporalinteractiontransformerarchitectureforremotesensingchangedetection |