Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression
Multimodal medical imaging, which involves the simultaneous acquisition of different modalities, enhances diagnostic accuracy and provides comprehensive visualization of anatomy and physiology. However, this significantly increases data size, posing storage and transmission challenges. Standard imag...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Open Journal of Signal Processing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10978054/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850272384972488704 |
|---|---|
| author | Daniel S. Nicolau Lucas A. Thomaz Luis M. N. Tavora Sergio M. M. Faria |
| author_facet | Daniel S. Nicolau Lucas A. Thomaz Luis M. N. Tavora Sergio M. M. Faria |
| author_sort | Daniel S. Nicolau |
| collection | DOAJ |
| description | Multimodal medical imaging, which involves the simultaneous acquisition of different modalities, enhances diagnostic accuracy and provides comprehensive visualization of anatomy and physiology. However, this significantly increases data size, posing storage and transmission challenges. Standard image codecs fail to properly exploit cross-modality redundancies, limiting coding efficiency. In this paper, a novel approach is proposed to enhance the compression gain and to reduce the computational complexity of a lossless cross-modality coding scheme for multimodal image pairs. The scheme uses a deep learning-based approach with Image-to-Image translation based on a Generative Adversarial Network architecture to generate an estimated image of one modality from its cross-modal pair. Two different approaches for inter-modal prediction are considered: one using the original and the estimated images for the inter-prediction scheme and another considering a weighted sum of both images. Subsequently, a decider based on a Convolutional Neural Network is employed to estimate the best coding approach to be selected among the two alternatives, before the coding step. A novel loss function that considers the decision accuracy and the compression gain of the chosen prediction approach is applied to improve the decision-making task. The experimental results on PET-CT and PET-MRI datasets demonstrate that the proposed approach improves by 11.76% and 4.61% the compression efficiency when compared with the single modality intra-coding of the Versatile Video Coding. Additionally, this approach allows to reduce the computational complexity by almost half in comparison to selecting the most compression-efficient after testing both schemes. |
| format | Article |
| id | doaj-art-e64498c377cd4e20a819cdc07d8a6fc1 |
| institution | OA Journals |
| issn | 2644-1322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Signal Processing |
| spelling | doaj-art-e64498c377cd4e20a819cdc07d8a6fc12025-08-20T01:51:49ZengIEEEIEEE Open Journal of Signal Processing2644-13222025-01-01648949710.1109/OJSP.2025.356483010978054Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging CompressionDaniel S. Nicolau0https://orcid.org/0009-0005-9095-8478Lucas A. Thomaz1https://orcid.org/0000-0002-1004-7772Luis M. N. Tavora2https://orcid.org/0000-0002-8580-1979Sergio M. M. Faria3https://orcid.org/0000-0002-0993-9124Instituto de Telecomunicações, Leiria, PortugalInstituto de Telecomunicações, Leiria, PortugalInstituto de Telecomunicações, Leiria, PortugalInstituto de Telecomunicações, Leiria, PortugalMultimodal medical imaging, which involves the simultaneous acquisition of different modalities, enhances diagnostic accuracy and provides comprehensive visualization of anatomy and physiology. However, this significantly increases data size, posing storage and transmission challenges. Standard image codecs fail to properly exploit cross-modality redundancies, limiting coding efficiency. In this paper, a novel approach is proposed to enhance the compression gain and to reduce the computational complexity of a lossless cross-modality coding scheme for multimodal image pairs. The scheme uses a deep learning-based approach with Image-to-Image translation based on a Generative Adversarial Network architecture to generate an estimated image of one modality from its cross-modal pair. Two different approaches for inter-modal prediction are considered: one using the original and the estimated images for the inter-prediction scheme and another considering a weighted sum of both images. Subsequently, a decider based on a Convolutional Neural Network is employed to estimate the best coding approach to be selected among the two alternatives, before the coding step. A novel loss function that considers the decision accuracy and the compression gain of the chosen prediction approach is applied to improve the decision-making task. The experimental results on PET-CT and PET-MRI datasets demonstrate that the proposed approach improves by 11.76% and 4.61% the compression efficiency when compared with the single modality intra-coding of the Versatile Video Coding. Additionally, this approach allows to reduce the computational complexity by almost half in comparison to selecting the most compression-efficient after testing both schemes.https://ieeexplore.ieee.org/document/10978054/Generative predictive codinglearning-based predictionlossless image codingmultimodal medical imagingversatile video coding |
| spellingShingle | Daniel S. Nicolau Lucas A. Thomaz Luis M. N. Tavora Sergio M. M. Faria Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression IEEE Open Journal of Signal Processing Generative predictive coding learning-based prediction lossless image coding multimodal medical imaging versatile video coding |
| title | Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression |
| title_full | Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression |
| title_fullStr | Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression |
| title_full_unstemmed | Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression |
| title_short | Enhancing Learning-Based Cross-Modality Prediction for Lossless Medical Imaging Compression |
| title_sort | enhancing learning based cross modality prediction for lossless medical imaging compression |
| topic | Generative predictive coding learning-based prediction lossless image coding multimodal medical imaging versatile video coding |
| url | https://ieeexplore.ieee.org/document/10978054/ |
| work_keys_str_mv | AT danielsnicolau enhancinglearningbasedcrossmodalitypredictionforlosslessmedicalimagingcompression AT lucasathomaz enhancinglearningbasedcrossmodalitypredictionforlosslessmedicalimagingcompression AT luismntavora enhancinglearningbasedcrossmodalitypredictionforlosslessmedicalimagingcompression AT sergiommfaria enhancinglearningbasedcrossmodalitypredictionforlosslessmedicalimagingcompression |