Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph
Deep learning (DL)-based pan-sharpening methods have become mainstream due to their exceptional performance. However, the lack of ground truth for supervised learning forces most DL-based methods to use pseudoground truth multispectral images, limiting learning potential and constraining the model&a...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10818769/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590434904309760 |
|---|---|
| author | Yunxuan Tang Xue Wang Peng Liu Tong Li |
| author_facet | Yunxuan Tang Xue Wang Peng Liu Tong Li |
| author_sort | Yunxuan Tang |
| collection | DOAJ |
| description | Deep learning (DL)-based pan-sharpening methods have become mainstream due to their exceptional performance. However, the lack of ground truth for supervised learning forces most DL-based methods to use pseudoground truth multispectral images, limiting learning potential and constraining the model's solution space. Unsupervised methods often neglect mutual learning across modalities, leading to insufficient spatial details in pan-sharpened images. To address these issues, this study proposes a fusion-decomposition pan-sharpening model based on interactive learning of representation graphs. This model considers both the compression process from source images to fused results and the decomposition process back to the source images. It aims to leveraging feature consistency between these processes to enhance the spatial and spectral consistency learned by the fusion network in a data-driven manner. Specifically, the fusion network incorporates the meticulously designed representational graph interaction module and the graph interaction fusion module. These modules construct a representational graph structure for cross-modal feature communication, generating a global representation that guides the cross-modal semantic aggregation of multispectral and panchromatic data. In the decomposition network, the spatial structure perception module and the spectral feature extraction module, designed based on the attributes of the source image features, enable the network to better perceive and reconstruct multispectral and panchromatic data from the fused result. This, in turn, enhances the fusion network's perception of spectral information and spatial structure. Qualitative and quantitative results on the IKONOS, GaoFen-2, WorldView-2, and WorldView-3 datasets validate the effectiveness of the proposed method in comparison to other state-of-the-art methods. |
| format | Article |
| id | doaj-art-eddfede2a58749bab8304142f8d1d4ea |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-eddfede2a58749bab8304142f8d1d4ea2025-01-24T00:01:00ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01183812382610.1109/JSTARS.2024.352438610818769Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation GraphYunxuan Tang0https://orcid.org/0009-0003-5289-4122Xue Wang1https://orcid.org/0000-0001-6674-8140Peng Liu2https://orcid.org/0000-0002-3283-454XTong Li3https://orcid.org/0000-0002-3257-213XSchool of Information Science and Engineering, Yunnan University, Kunming, ChinaSchool of Information Science and Engineering, Yunnan University, Kunming, ChinaSchool of Information Science and Engineering, Yunnan University, Kunming, ChinaCollege of Big Data, Yunnan Agricultural University, Kunming, ChinaDeep learning (DL)-based pan-sharpening methods have become mainstream due to their exceptional performance. However, the lack of ground truth for supervised learning forces most DL-based methods to use pseudoground truth multispectral images, limiting learning potential and constraining the model's solution space. Unsupervised methods often neglect mutual learning across modalities, leading to insufficient spatial details in pan-sharpened images. To address these issues, this study proposes a fusion-decomposition pan-sharpening model based on interactive learning of representation graphs. This model considers both the compression process from source images to fused results and the decomposition process back to the source images. It aims to leveraging feature consistency between these processes to enhance the spatial and spectral consistency learned by the fusion network in a data-driven manner. Specifically, the fusion network incorporates the meticulously designed representational graph interaction module and the graph interaction fusion module. These modules construct a representational graph structure for cross-modal feature communication, generating a global representation that guides the cross-modal semantic aggregation of multispectral and panchromatic data. In the decomposition network, the spatial structure perception module and the spectral feature extraction module, designed based on the attributes of the source image features, enable the network to better perceive and reconstruct multispectral and panchromatic data from the fused result. This, in turn, enhances the fusion network's perception of spectral information and spatial structure. Qualitative and quantitative results on the IKONOS, GaoFen-2, WorldView-2, and WorldView-3 datasets validate the effectiveness of the proposed method in comparison to other state-of-the-art methods.https://ieeexplore.ieee.org/document/10818769/Deep learning (DL)fusion-decompositiongraph neural network (GNN)pan-sharpening |
| spellingShingle | Yunxuan Tang Xue Wang Peng Liu Tong Li Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Deep learning (DL) fusion-decomposition graph neural network (GNN) pan-sharpening |
| title | Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph |
| title_full | Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph |
| title_fullStr | Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph |
| title_full_unstemmed | Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph |
| title_short | Fusion-Decomposition Pan-Sharpening Network With Interactive Learning of Representation Graph |
| title_sort | fusion decomposition pan sharpening network with interactive learning of representation graph |
| topic | Deep learning (DL) fusion-decomposition graph neural network (GNN) pan-sharpening |
| url | https://ieeexplore.ieee.org/document/10818769/ |
| work_keys_str_mv | AT yunxuantang fusiondecompositionpansharpeningnetworkwithinteractivelearningofrepresentationgraph AT xuewang fusiondecompositionpansharpeningnetworkwithinteractivelearningofrepresentationgraph AT pengliu fusiondecompositionpansharpeningnetworkwithinteractivelearningofrepresentationgraph AT tongli fusiondecompositionpansharpeningnetworkwithinteractivelearningofrepresentationgraph |