GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation
Synthetic aperture radar (SAR) image to optical image translation not only assists information interpretability, but also fills the gaps in optical applications due to weather and light limitations. However, several studies have pointed out that specialized methods heavily struggle to deliver images...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10816574/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832592935194984448 |
|---|---|
| author | Hongbo Liang Xuezhi Yang Xiangyu Yang Jinjin Luo Jiajia Zhu |
| author_facet | Hongbo Liang Xuezhi Yang Xiangyu Yang Jinjin Luo Jiajia Zhu |
| author_sort | Hongbo Liang |
| collection | DOAJ |
| description | Synthetic aperture radar (SAR) image to optical image translation not only assists information interpretability, but also fills the gaps in optical applications due to weather and light limitations. However, several studies have pointed out that specialized methods heavily struggle to deliver images with widely varying optical imaging styles, thus, resulting in poor image translation with disharmonious and repetitive artifacts. Another critical issue attributes to the scarcity of geographical prior knowledge. The generator always attempts to produce images within a narrow scope of the data space, which severely restricts the semantic correspondence between SAR content and optical styles. In this article, we introduce a novel tokenization, namely geographical imaging tokenizer (GIT), which captures imaging style of ground materials in the optical image. Based on the GIT, we propose a geographical feature tokenization transformer framework (GFTT) that discovers the consensus between SAR and optical images. In addition, we leverage a self-supervisory task to encourage the transformer to learn meaningful semantic correspondence from local and global style patterns. Finally, we utilize the noise-contrastive estimation loss to maximize mutual information between the input and translated image. Through qualitative and quantitative experimental evaluations, we verify the reliability of the proposed GIT that aligns with authentic expressions of the optical observation scenario, and indicates the superiority of GFTT in contrast to the state-of-the-art algorithms. |
| format | Article |
| id | doaj-art-7eb47e082054478187543e04c818f3bc |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-7eb47e082054478187543e04c818f3bc2025-01-21T00:00:40ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01182975298910.1109/JSTARS.2024.352327410816574GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image TranslationHongbo Liang0https://orcid.org/0000-0002-6046-9340Xuezhi Yang1https://orcid.org/0000-0002-2303-4858Xiangyu Yang2Jinjin Luo3Jiajia Zhu4https://orcid.org/0009-0008-9914-275XAnhui Key Laboratory of Industry Safety and Emergency Technology, Hefei, ChinaAnhui Key Laboratory of Industry Safety and Emergency Technology, Hefei, ChinaAnhui Key Laboratory of Industry Safety and Emergency Technology, Hefei, ChinaAnhui Key Laboratory of Industry Safety and Emergency Technology, Hefei, ChinaAnhui Key Laboratory of Industry Safety and Emergency Technology, Hefei, ChinaSynthetic aperture radar (SAR) image to optical image translation not only assists information interpretability, but also fills the gaps in optical applications due to weather and light limitations. However, several studies have pointed out that specialized methods heavily struggle to deliver images with widely varying optical imaging styles, thus, resulting in poor image translation with disharmonious and repetitive artifacts. Another critical issue attributes to the scarcity of geographical prior knowledge. The generator always attempts to produce images within a narrow scope of the data space, which severely restricts the semantic correspondence between SAR content and optical styles. In this article, we introduce a novel tokenization, namely geographical imaging tokenizer (GIT), which captures imaging style of ground materials in the optical image. Based on the GIT, we propose a geographical feature tokenization transformer framework (GFTT) that discovers the consensus between SAR and optical images. In addition, we leverage a self-supervisory task to encourage the transformer to learn meaningful semantic correspondence from local and global style patterns. Finally, we utilize the noise-contrastive estimation loss to maximize mutual information between the input and translated image. Through qualitative and quantitative experimental evaluations, we verify the reliability of the proposed GIT that aligns with authentic expressions of the optical observation scenario, and indicates the superiority of GFTT in contrast to the state-of-the-art algorithms.https://ieeexplore.ieee.org/document/10816574/Geographical imaging tokenizer (GIT)noise-contrastive estimation (NCE)self-supervisory tasksynthetic aperture radar (SAR)-to-optical (S2O) image translationtransformer |
| spellingShingle | Hongbo Liang Xuezhi Yang Xiangyu Yang Jinjin Luo Jiajia Zhu GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Geographical imaging tokenizer (GIT) noise-contrastive estimation (NCE) self-supervisory task synthetic aperture radar (SAR)-to-optical (S2O) image translation transformer |
| title | GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation |
| title_full | GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation |
| title_fullStr | GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation |
| title_full_unstemmed | GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation |
| title_short | GFTT: Geographical Feature Tokenization Transformer for SAR-to-Optical Image Translation |
| title_sort | gftt geographical feature tokenization transformer for sar to optical image translation |
| topic | Geographical imaging tokenizer (GIT) noise-contrastive estimation (NCE) self-supervisory task synthetic aperture radar (SAR)-to-optical (S2O) image translation transformer |
| url | https://ieeexplore.ieee.org/document/10816574/ |
| work_keys_str_mv | AT hongboliang gfttgeographicalfeaturetokenizationtransformerforsartoopticalimagetranslation AT xuezhiyang gfttgeographicalfeaturetokenizationtransformerforsartoopticalimagetranslation AT xiangyuyang gfttgeographicalfeaturetokenizationtransformerforsartoopticalimagetranslation AT jinjinluo gfttgeographicalfeaturetokenizationtransformerforsartoopticalimagetranslation AT jiajiazhu gfttgeographicalfeaturetokenizationtransformerforsartoopticalimagetranslation |