Rain removal method for single image of dual-branch joint network based on sparse transformer
Abstract In response to image degradation caused by rain during image acquisition, this paper proposes a rain removal method for single image of dual-branch joint network based on a sparse Transformer (DBSTNet). The developed model comprises a rain removal subnet and a background recovery subnet. Th...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-12-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01711-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571190905929728 |
|---|---|
| author | Fangfang Qin Zongpu Jia Xiaoyan Pang Shan Zhao |
| author_facet | Fangfang Qin Zongpu Jia Xiaoyan Pang Shan Zhao |
| author_sort | Fangfang Qin |
| collection | DOAJ |
| description | Abstract In response to image degradation caused by rain during image acquisition, this paper proposes a rain removal method for single image of dual-branch joint network based on a sparse Transformer (DBSTNet). The developed model comprises a rain removal subnet and a background recovery subnet. The former extracts rain trace information utilizing a rain removal strategy, while the latter employs this information to restore background details. Furthermore, a U-shaped encoder-decoder branch (UEDB) focuses on local features to mitigate the impact of rainwater on background detail textures. UEDB incorporates a feature refinement unit to maximize the contribution of the channel attention mechanism in recovering local detail features. Additionally, since tokens with low relevance in the Transformer may influence image recovery, this study introduces a residual sparse Transformer branch (RSTB) to overcome the limitations of the Convolutional Neural Network’s (CNN’s) receptive field. Indeed, RSTB preserves the most valuable self-attention values for the aggregation of features, facilitating high-quality image reconstruction from a global perspective. Finally, the parallel dual-branch joint module, composed of RSTB and UEDB branches, effectively captures the local context and global structure, culminating in a clear background image. Experimental validation on synthetic and real datasets demonstrates that rain removal images exhibit richer detail information, significantly improving the overall visual effect. |
| format | Article |
| id | doaj-art-b143d60a3b9d48b4b59b558c4114e8f5 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-b143d60a3b9d48b4b59b558c4114e8f52025-02-02T12:49:43ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-12-0111111910.1007/s40747-024-01711-wRain removal method for single image of dual-branch joint network based on sparse transformerFangfang Qin0Zongpu Jia1Xiaoyan Pang2Shan Zhao3School of Software, Henan Polytechnic UniversitySchool of Software, Henan Polytechnic UniversitySchool of Software, Henan Polytechnic UniversitySchool of Software, Henan Polytechnic UniversityAbstract In response to image degradation caused by rain during image acquisition, this paper proposes a rain removal method for single image of dual-branch joint network based on a sparse Transformer (DBSTNet). The developed model comprises a rain removal subnet and a background recovery subnet. The former extracts rain trace information utilizing a rain removal strategy, while the latter employs this information to restore background details. Furthermore, a U-shaped encoder-decoder branch (UEDB) focuses on local features to mitigate the impact of rainwater on background detail textures. UEDB incorporates a feature refinement unit to maximize the contribution of the channel attention mechanism in recovering local detail features. Additionally, since tokens with low relevance in the Transformer may influence image recovery, this study introduces a residual sparse Transformer branch (RSTB) to overcome the limitations of the Convolutional Neural Network’s (CNN’s) receptive field. Indeed, RSTB preserves the most valuable self-attention values for the aggregation of features, facilitating high-quality image reconstruction from a global perspective. Finally, the parallel dual-branch joint module, composed of RSTB and UEDB branches, effectively captures the local context and global structure, culminating in a clear background image. Experimental validation on synthetic and real datasets demonstrates that rain removal images exhibit richer detail information, significantly improving the overall visual effect.https://doi.org/10.1007/s40747-024-01711-wImage derainingSelf-attentionDual-branchSparse transformerDeep learning |
| spellingShingle | Fangfang Qin Zongpu Jia Xiaoyan Pang Shan Zhao Rain removal method for single image of dual-branch joint network based on sparse transformer Complex & Intelligent Systems Image deraining Self-attention Dual-branch Sparse transformer Deep learning |
| title | Rain removal method for single image of dual-branch joint network based on sparse transformer |
| title_full | Rain removal method for single image of dual-branch joint network based on sparse transformer |
| title_fullStr | Rain removal method for single image of dual-branch joint network based on sparse transformer |
| title_full_unstemmed | Rain removal method for single image of dual-branch joint network based on sparse transformer |
| title_short | Rain removal method for single image of dual-branch joint network based on sparse transformer |
| title_sort | rain removal method for single image of dual branch joint network based on sparse transformer |
| topic | Image deraining Self-attention Dual-branch Sparse transformer Deep learning |
| url | https://doi.org/10.1007/s40747-024-01711-w |
| work_keys_str_mv | AT fangfangqin rainremovalmethodforsingleimageofdualbranchjointnetworkbasedonsparsetransformer AT zongpujia rainremovalmethodforsingleimageofdualbranchjointnetworkbasedonsparsetransformer AT xiaoyanpang rainremovalmethodforsingleimageofdualbranchjointnetworkbasedonsparsetransformer AT shanzhao rainremovalmethodforsingleimageofdualbranchjointnetworkbasedonsparsetransformer |