Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening
Pansharpening aims to reconstruct high-resolution multispectral (HRMS) images from low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) data. In recent years, deep learning development has spurred the evolution of data-driven pansharpening methods, which have demonstrate...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10947514/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850155077021466624 |
|---|---|
| author | Lihua Jian Jiabo Liu Lihui Chen Di Zhang Gemine Vivone Xichuan Zhou |
| author_facet | Lihua Jian Jiabo Liu Lihui Chen Di Zhang Gemine Vivone Xichuan Zhou |
| author_sort | Lihua Jian |
| collection | DOAJ |
| description | Pansharpening aims to reconstruct high-resolution multispectral (HRMS) images from low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) data. In recent years, deep learning development has spurred the evolution of data-driven pansharpening methods, which have demonstrated superior performance compared to traditional approaches. However, most existing methods face challenges in preserving spectral information, struggle to effectively interact and aggregate MS and PAN image data, and fail to adaptively coordinate the local textures of high-resolution PAN with the global characteristics of low-resolution MS, leading to suboptimal spectral and spatial fidelity in the generated HRMS images. This article introduces a feature interaction and adaptive fusion network (FIAFN) with spectral modulation for pansharpening to address these issues. The FIAFN is designed to maintain spectral integrity by incorporating spectral modulation while employing feature interaction attention (FIA) modules and a feature adaptive selection attention (FASA) module to fuse complementary information effectively. Specifically, FIA integrates a large receptive field-selective kernel with efficient multiscale attention (EMA) to extract both global information from MS images and multiscale detail information from PAN images. In addition, a residual structure-based self-guided spatial-channel adaptive convolution is introduced to accommodate diverse features within FASA adaptively. Comprehensive experiments demonstrate that our proposed FIAFN achieves superior performance compared to recent methods on the QuickBird (QB), WorldView-3 (WV3), and WorldView-2 (WV2) datasets. |
| format | Article |
| id | doaj-art-c55df870b86546adb3dffebefbcd1e9c |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-c55df870b86546adb3dffebefbcd1e9c2025-08-20T02:25:03ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118104581047010.1109/JSTARS.2025.355716510947514Feature Interaction and Adaptive Fusion Network With Spectral Modulation for PansharpeningLihua Jian0https://orcid.org/0000-0002-1922-566XJiabo Liu1Lihui Chen2https://orcid.org/0000-0002-0948-1600Di Zhang3https://orcid.org/0000-0001-7190-1621Gemine Vivone4https://orcid.org/0000-0001-9542-0638Xichuan Zhou5https://orcid.org/0000-0002-3304-3045School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, ChinaSchool of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, ChinaSchool of Microelectronics and Communication Engineering, Chongqing University, Chongqing, ChinaSchool of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, ChinaNational Research Council, Institute of Methodologies for Environmental Analysis, CNR-IMAA, Tito, ItalySchool of Microelectronics and Communication Engineering, Chongqing University, Chongqing, ChinaPansharpening aims to reconstruct high-resolution multispectral (HRMS) images from low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) data. In recent years, deep learning development has spurred the evolution of data-driven pansharpening methods, which have demonstrated superior performance compared to traditional approaches. However, most existing methods face challenges in preserving spectral information, struggle to effectively interact and aggregate MS and PAN image data, and fail to adaptively coordinate the local textures of high-resolution PAN with the global characteristics of low-resolution MS, leading to suboptimal spectral and spatial fidelity in the generated HRMS images. This article introduces a feature interaction and adaptive fusion network (FIAFN) with spectral modulation for pansharpening to address these issues. The FIAFN is designed to maintain spectral integrity by incorporating spectral modulation while employing feature interaction attention (FIA) modules and a feature adaptive selection attention (FASA) module to fuse complementary information effectively. Specifically, FIA integrates a large receptive field-selective kernel with efficient multiscale attention (EMA) to extract both global information from MS images and multiscale detail information from PAN images. In addition, a residual structure-based self-guided spatial-channel adaptive convolution is introduced to accommodate diverse features within FASA adaptively. Comprehensive experiments demonstrate that our proposed FIAFN achieves superior performance compared to recent methods on the QuickBird (QB), WorldView-3 (WV3), and WorldView-2 (WV2) datasets.https://ieeexplore.ieee.org/document/10947514/Feature interaction and fusionimage enhancementimage fusionpansharpeningremote sensing |
| spellingShingle | Lihua Jian Jiabo Liu Lihui Chen Di Zhang Gemine Vivone Xichuan Zhou Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Feature interaction and fusion image enhancement image fusion pansharpening remote sensing |
| title | Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening |
| title_full | Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening |
| title_fullStr | Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening |
| title_full_unstemmed | Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening |
| title_short | Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening |
| title_sort | feature interaction and adaptive fusion network with spectral modulation for pansharpening |
| topic | Feature interaction and fusion image enhancement image fusion pansharpening remote sensing |
| url | https://ieeexplore.ieee.org/document/10947514/ |
| work_keys_str_mv | AT lihuajian featureinteractionandadaptivefusionnetworkwithspectralmodulationforpansharpening AT jiaboliu featureinteractionandadaptivefusionnetworkwithspectralmodulationforpansharpening AT lihuichen featureinteractionandadaptivefusionnetworkwithspectralmodulationforpansharpening AT dizhang featureinteractionandadaptivefusionnetworkwithspectralmodulationforpansharpening AT geminevivone featureinteractionandadaptivefusionnetworkwithspectralmodulationforpansharpening AT xichuanzhou featureinteractionandadaptivefusionnetworkwithspectralmodulationforpansharpening |