Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision
Remote sensing scene knowledge graphs symbolically describe the concepts of scenes and reveal their interrelations, highlighting robust knowledge modeling and inference capabilities in zero-shot remote sensing scene classification tasks. However, current graphs rely heavily on expert manual interpre...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10771687/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832592922464223232 |
|---|---|
| author | Chao Wang Jiajun Yang Tanvir Ahmed Yang Zhao Tong Zhang Bing Sun Tao Xie Jie Wang Tianyu Chen |
| author_facet | Chao Wang Jiajun Yang Tanvir Ahmed Yang Zhao Tong Zhang Bing Sun Tao Xie Jie Wang Tianyu Chen |
| author_sort | Chao Wang |
| collection | DOAJ |
| description | Remote sensing scene knowledge graphs symbolically describe the concepts of scenes and reveal their interrelations, highlighting robust knowledge modeling and inference capabilities in zero-shot remote sensing scene classification tasks. However, current graphs rely heavily on expert manual interpretation, making them susceptible to human biases and difficult to expand. They also lack quantitative description of the degree of spatial relationship associations and fail to adequately represent semantic information. Additionally, they often overlook discriminative local landscapes, which is crucial for accurate scene classification. To address these limitations, this article proposes the “zero-shot remote sensing scene classification via automatic knowledge graph and dual-branch semantic correlation supervision (AKG-DBSS).” This method starts with the scenes and automates the construction of a knowledge graph, “scene-landscape-ground objects” (ASLG-KG), by analyzing the composition and spatial distribution of ground objects in local regions. On this basis, the dual-branch semantic correlation supervised zero-shot remote sensing scene classification network (DBSS) supervises the mapping of semantic features to the visual space via both global and local branches, ensuring the visual space reflects the correlation structure of the semantic space. Extensive experiments on the UCM, AID, and NWPU datasets demonstrate that AKG-DBSS achieves class average accuracy and overall accuracy of up to 98%, and 59.56%, respectively, for the classification of unseen class scenes, with standard deviations below 6.91%, significantly outperforming four other advanced comparative methods. Furthermore, additional experiments prove that ASLG-KG and DBSS are feasible, necessary, and effective, with an accuracy improvement in overall accuracy of over 8.98%. |
| format | Article |
| id | doaj-art-5161c32d60ba49fea6b0ad1aa9b9177c |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-5161c32d60ba49fea6b0ad1aa9b9177c2025-01-21T00:00:47ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01183300331410.1109/JSTARS.2024.350593510771687Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation SupervisionChao Wang0https://orcid.org/0009-0005-5856-4153Jiajun Yang1https://orcid.org/0009-0003-3572-7859Tanvir Ahmed2https://orcid.org/0009-0003-4298-8459Yang Zhao3Tong Zhang4https://orcid.org/0009-0000-9330-9389Bing Sun5Tao Xie6https://orcid.org/0000-0002-2070-8343Jie Wang7https://orcid.org/0000-0002-1933-1593Tianyu Chen8https://orcid.org/0009-0009-1352-7825School of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Reading, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaSchool of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, ChinaRemote sensing scene knowledge graphs symbolically describe the concepts of scenes and reveal their interrelations, highlighting robust knowledge modeling and inference capabilities in zero-shot remote sensing scene classification tasks. However, current graphs rely heavily on expert manual interpretation, making them susceptible to human biases and difficult to expand. They also lack quantitative description of the degree of spatial relationship associations and fail to adequately represent semantic information. Additionally, they often overlook discriminative local landscapes, which is crucial for accurate scene classification. To address these limitations, this article proposes the “zero-shot remote sensing scene classification via automatic knowledge graph and dual-branch semantic correlation supervision (AKG-DBSS).” This method starts with the scenes and automates the construction of a knowledge graph, “scene-landscape-ground objects” (ASLG-KG), by analyzing the composition and spatial distribution of ground objects in local regions. On this basis, the dual-branch semantic correlation supervised zero-shot remote sensing scene classification network (DBSS) supervises the mapping of semantic features to the visual space via both global and local branches, ensuring the visual space reflects the correlation structure of the semantic space. Extensive experiments on the UCM, AID, and NWPU datasets demonstrate that AKG-DBSS achieves class average accuracy and overall accuracy of up to 98%, and 59.56%, respectively, for the classification of unseen class scenes, with standard deviations below 6.91%, significantly outperforming four other advanced comparative methods. Furthermore, additional experiments prove that ASLG-KG and DBSS are feasible, necessary, and effective, with an accuracy improvement in overall accuracy of over 8.98%.https://ieeexplore.ieee.org/document/10771687/Automatic “scene-landscape-ground objects” knowledge graphdual-branch semantic correlation supervision (DBSS)remote sensing scene classificationzero-shot learning |
| spellingShingle | Chao Wang Jiajun Yang Tanvir Ahmed Yang Zhao Tong Zhang Bing Sun Tao Xie Jie Wang Tianyu Chen Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Automatic “scene-landscape-ground objects” knowledge graph dual-branch semantic correlation supervision (DBSS) remote sensing scene classification zero-shot learning |
| title | Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision |
| title_full | Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision |
| title_fullStr | Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision |
| title_full_unstemmed | Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision |
| title_short | Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision |
| title_sort | zero shot remote sensing scene classification based on automatic knowledge graph and dual branch semantic correlation supervision |
| topic | Automatic “scene-landscape-ground objects” knowledge graph dual-branch semantic correlation supervision (DBSS) remote sensing scene classification zero-shot learning |
| url | https://ieeexplore.ieee.org/document/10771687/ |
| work_keys_str_mv | AT chaowang zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT jiajunyang zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT tanvirahmed zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT yangzhao zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT tongzhang zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT bingsun zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT taoxie zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT jiewang zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision AT tianyuchen zeroshotremotesensingsceneclassificationbasedonautomaticknowledgegraphanddualbranchsemanticcorrelationsupervision |