Tell Me What You See: Text-Guided Real-World Image Denoising
Image reconstruction from noisy sensor measurements is challenging and many methods have been proposed for it. Yet, most approaches focus on learning robust natural image priors while modeling the scene’s noise statistics. In extremely low-light conditions, these methods often remain insu...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Open Journal of Signal Processing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11078899/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849246294231482368 |
|---|---|
| author | Erez Yosef Raja Giryes |
| author_facet | Erez Yosef Raja Giryes |
| author_sort | Erez Yosef |
| collection | DOAJ |
| description | Image reconstruction from noisy sensor measurements is challenging and many methods have been proposed for it. Yet, most approaches focus on learning robust natural image priors while modeling the scene’s noise statistics. In extremely low-light conditions, these methods often remain insufficient. Additional information is needed, such as multiple captures or, as suggested here, scene description. As an alternative, we propose using a text-based description of the scene as an additional prior, something the photographer can easily provide. Inspired by the remarkable success of text-guided diffusion models in image generation, we show that adding image caption information significantly improves image denoising and reconstruction for both synthetic and real-world images. All code and data will be made publicly available upon publication. |
| format | Article |
| id | doaj-art-ecccd4bcaf944b40a83a33aca09dfefb |
| institution | Kabale University |
| issn | 2644-1322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Signal Processing |
| spelling | doaj-art-ecccd4bcaf944b40a83a33aca09dfefb2025-08-20T03:58:32ZengIEEEIEEE Open Journal of Signal Processing2644-13222025-01-01689089910.1109/OJSP.2025.358871511078899Tell Me What You See: Text-Guided Real-World Image DenoisingErez Yosef0https://orcid.org/0000-0003-1503-189XRaja Giryes1https://orcid.org/0000-0002-2830-0297Tel-Aviv University, Tel Aviv, IsraelTel-Aviv University, Tel Aviv, IsraelImage reconstruction from noisy sensor measurements is challenging and many methods have been proposed for it. Yet, most approaches focus on learning robust natural image priors while modeling the scene’s noise statistics. In extremely low-light conditions, these methods often remain insufficient. Additional information is needed, such as multiple captures or, as suggested here, scene description. As an alternative, we propose using a text-based description of the scene as an additional prior, something the photographer can easily provide. Inspired by the remarkable success of text-guided diffusion models in image generation, we show that adding image caption information significantly improves image denoising and reconstruction for both synthetic and real-world images. All code and data will be made publicly available upon publication.https://ieeexplore.ieee.org/document/11078899/Computational imagingdeep learningartificial intelligence |
| spellingShingle | Erez Yosef Raja Giryes Tell Me What You See: Text-Guided Real-World Image Denoising IEEE Open Journal of Signal Processing Computational imaging deep learning artificial intelligence |
| title | Tell Me What You See: Text-Guided Real-World Image Denoising |
| title_full | Tell Me What You See: Text-Guided Real-World Image Denoising |
| title_fullStr | Tell Me What You See: Text-Guided Real-World Image Denoising |
| title_full_unstemmed | Tell Me What You See: Text-Guided Real-World Image Denoising |
| title_short | Tell Me What You See: Text-Guided Real-World Image Denoising |
| title_sort | tell me what you see text guided real world image denoising |
| topic | Computational imaging deep learning artificial intelligence |
| url | https://ieeexplore.ieee.org/document/11078899/ |
| work_keys_str_mv | AT erezyosef tellmewhatyouseetextguidedrealworldimagedenoising AT rajagiryes tellmewhatyouseetextguidedrealworldimagedenoising |