Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing
Recent advancements in generative AI, particularly diffusion-based image editing, have enabled the transformation of images into highly realistic scenes using only text instructions. This technology offers significant potential for generating diverse synthetic datasets to evaluate model robustness i...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10870179/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850177752198545408 |
|---|---|
| author | Naufal Suryanto Andro Aprila Adiputra Ahmada Yusril Kadiptya Thi-Thu-Huong Le Derry Pratama Yongsu Kim Howon Kim |
| author_facet | Naufal Suryanto Andro Aprila Adiputra Ahmada Yusril Kadiptya Thi-Thu-Huong Le Derry Pratama Yongsu Kim Howon Kim |
| author_sort | Naufal Suryanto |
| collection | DOAJ |
| description | Recent advancements in generative AI, particularly diffusion-based image editing, have enabled the transformation of images into highly realistic scenes using only text instructions. This technology offers significant potential for generating diverse synthetic datasets to evaluate model robustness in real-world applications, especially in the era of autonomous driving. In this paper, we introduce Cityscape-Adverse, a benchmark that employs diffusion-based image editing to simulate eight adverse conditions, including variations in weather, lighting, and seasons, while preserving the original semantic labels. We evaluate the reliability of diffusion-based models in generating realistic scene modifications and assess the performance of state-of-the-art CNN and Transformer-based semantic segmentation models under these challenging conditions. Additionally, we analyze which modifications have the greatest impact on model performance and explore how training on synthetic datasets can improve robustness in real-world adverse scenarios. Our results demonstrate that all tested models, particularly CNN-based architectures, experienced significant performance degradation under extreme conditions, while Transformer-based models exhibited greater resilience. We verify that models trained on Cityscape-Adverse show significantly enhanced resilience when applied to unseen domains. Code and datasets will be released at <uri>https://github.com/naufalso/cityscape-adverse</uri>. |
| format | Article |
| id | doaj-art-43af9a6e7d1e47dcb3cd9e0a1b9a3d93 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-43af9a6e7d1e47dcb3cd9e0a1b9a3d932025-08-20T02:18:55ZengIEEEIEEE Access2169-35362025-01-0113699216994010.1109/ACCESS.2025.353798110870179Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image EditingNaufal Suryanto0https://orcid.org/0000-0002-0396-5938Andro Aprila Adiputra1Ahmada Yusril Kadiptya2https://orcid.org/0009-0008-8612-9812Thi-Thu-Huong Le3https://orcid.org/0000-0002-8366-9396Derry Pratama4https://orcid.org/0000-0003-2246-3017Yongsu Kim5Howon Kim6https://orcid.org/0000-0001-8475-7294IoT Research Center, Pusan National University, Busan, Republic of KoreaSchool of Computer Science and Engineering, Pusan National University, Busan, Republic of KoreaSchool of Computer Science and Engineering, Pusan National University, Busan, Republic of KoreaBlockchain Platform Research Center, Pusan National University, Busan, Republic of KoreaSchool of Computer Science and Engineering, Pusan National University, Busan, Republic of KoreaSmartM2M, Busan, Republic of KoreaSchool of Computer Science and Engineering, Pusan National University, Busan, Republic of KoreaRecent advancements in generative AI, particularly diffusion-based image editing, have enabled the transformation of images into highly realistic scenes using only text instructions. This technology offers significant potential for generating diverse synthetic datasets to evaluate model robustness in real-world applications, especially in the era of autonomous driving. In this paper, we introduce Cityscape-Adverse, a benchmark that employs diffusion-based image editing to simulate eight adverse conditions, including variations in weather, lighting, and seasons, while preserving the original semantic labels. We evaluate the reliability of diffusion-based models in generating realistic scene modifications and assess the performance of state-of-the-art CNN and Transformer-based semantic segmentation models under these challenging conditions. Additionally, we analyze which modifications have the greatest impact on model performance and explore how training on synthetic datasets can improve robustness in real-world adverse scenarios. Our results demonstrate that all tested models, particularly CNN-based architectures, experienced significant performance degradation under extreme conditions, while Transformer-based models exhibited greater resilience. We verify that models trained on Cityscape-Adverse show significantly enhanced resilience when applied to unseen domains. Code and datasets will be released at <uri>https://github.com/naufalso/cityscape-adverse</uri>.https://ieeexplore.ieee.org/document/10870179/Adverse conditionsbenchmarkdatasetdiffusion-based image editinggenerative AImodel robustness |
| spellingShingle | Naufal Suryanto Andro Aprila Adiputra Ahmada Yusril Kadiptya Thi-Thu-Huong Le Derry Pratama Yongsu Kim Howon Kim Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing IEEE Access Adverse conditions benchmark dataset diffusion-based image editing generative AI model robustness |
| title | Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing |
| title_full | Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing |
| title_fullStr | Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing |
| title_full_unstemmed | Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing |
| title_short | Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing |
| title_sort | cityscape adverse benchmarking robustness of semantic segmentation with realistic scene modifications via diffusion based image editing |
| topic | Adverse conditions benchmark dataset diffusion-based image editing generative AI model robustness |
| url | https://ieeexplore.ieee.org/document/10870179/ |
| work_keys_str_mv | AT naufalsuryanto cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting AT androaprilaadiputra cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting AT ahmadayusrilkadiptya cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting AT thithuhuongle cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting AT derrypratama cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting AT yongsukim cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting AT howonkim cityscapeadversebenchmarkingrobustnessofsemanticsegmentationwithrealisticscenemodificationsviadiffusionbasedimageediting |