A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer
Kirsten rat sarcoma viral oncogene homolog (namely KRAS) is a key biomarker for prognostic analysis and targeted therapy of colorectal cancer. Recently, the advancement of machine learning, especially deep learning, has greatly promoted the development of KRAS mutation detection from tumor phenotype...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2024-09-01
|
| Series: | Big Data Mining and Analytics |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/BDMA.2024.9020012 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832543050804494336 |
|---|---|
| author | Zhilong Lv Rui Yan Yuexiao Lin Lin Gao Fa Zhang Ying Wang |
| author_facet | Zhilong Lv Rui Yan Yuexiao Lin Lin Gao Fa Zhang Ying Wang |
| author_sort | Zhilong Lv |
| collection | DOAJ |
| description | Kirsten rat sarcoma viral oncogene homolog (namely KRAS) is a key biomarker for prognostic analysis and targeted therapy of colorectal cancer. Recently, the advancement of machine learning, especially deep learning, has greatly promoted the development of KRAS mutation detection from tumor phenotype data, such as pathology slides or radiology images. However, there are still two major problems in existing studies: inadequate single-modal feature learning and lack of multimodal phenotypic feature fusion. In this paper, we propose a Disentangled Representation-based Multimodal Fusion framework integrating Pathomics and Radiomics (DRMF-PaRa) for KRAS mutation detection. Specifically, the DRMF-PaRa model consists of three parts: (1) the pathomics learning module, which introduces a tissue-guided Transformer model to extract more comprehensive and targeted pathological features; (2) the radiomics learning module, which captures the generic hand-crafted radiomics features and the task-specific deep radiomics features; (3) the disentangled representation-based multimodal fusion module, which learns factorized subspaces for each modality and provides a holistic view of the two heterogeneous phenotypic features. The proposed model is developed and evaluated on a multi modality dataset of 111 colorectal cancer patients with whole slide images and contrast-enhanced CT. The experimental results demonstrate the superiority of the proposed DRMF-PaRa model with an accuracy of 0.876 and an AUC of 0.865 for KRAS mutation detection. |
| format | Article |
| id | doaj-art-e5318c9013be456dab7a3a654e626975 |
| institution | Kabale University |
| issn | 2096-0654 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | Big Data Mining and Analytics |
| spelling | doaj-art-e5318c9013be456dab7a3a654e6269752025-02-03T11:53:24ZengTsinghua University PressBig Data Mining and Analytics2096-06542024-09-017359060210.26599/BDMA.2024.9020012A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal CancerZhilong Lv0Rui Yan1Yuexiao Lin2Lin Gao3Fa Zhang4Ying Wang5School of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaSchool of Biomedical Engineering, University of Science and Technology of China, Hefei 230026, ChinaDepartment of General Surgery, Beijing Chaoyang Hospital, Capital Medical University, Beijing 100020, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing 100081, ChinaDepartment of Pathology, Beijing Chaoyang Hospital, Capital Medical University, Beijing 100020, ChinaKirsten rat sarcoma viral oncogene homolog (namely KRAS) is a key biomarker for prognostic analysis and targeted therapy of colorectal cancer. Recently, the advancement of machine learning, especially deep learning, has greatly promoted the development of KRAS mutation detection from tumor phenotype data, such as pathology slides or radiology images. However, there are still two major problems in existing studies: inadequate single-modal feature learning and lack of multimodal phenotypic feature fusion. In this paper, we propose a Disentangled Representation-based Multimodal Fusion framework integrating Pathomics and Radiomics (DRMF-PaRa) for KRAS mutation detection. Specifically, the DRMF-PaRa model consists of three parts: (1) the pathomics learning module, which introduces a tissue-guided Transformer model to extract more comprehensive and targeted pathological features; (2) the radiomics learning module, which captures the generic hand-crafted radiomics features and the task-specific deep radiomics features; (3) the disentangled representation-based multimodal fusion module, which learns factorized subspaces for each modality and provides a holistic view of the two heterogeneous phenotypic features. The proposed model is developed and evaluated on a multi modality dataset of 111 colorectal cancer patients with whole slide images and contrast-enhanced CT. The experimental results demonstrate the superiority of the proposed DRMF-PaRa model with an accuracy of 0.876 and an AUC of 0.865 for KRAS mutation detection.https://www.sciopen.com/article/10.26599/BDMA.2024.9020012kras mutation detectionmultimodal feature fusionpathomicsradiomics |
| spellingShingle | Zhilong Lv Rui Yan Yuexiao Lin Lin Gao Fa Zhang Ying Wang A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer Big Data Mining and Analytics kras mutation detection multimodal feature fusion pathomics radiomics |
| title | A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer |
| title_full | A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer |
| title_fullStr | A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer |
| title_full_unstemmed | A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer |
| title_short | A Disentangled Representation-Based Multimodal Fusion Framework Integrating Pathomics and Radiomics for KRAS Mutation Detection in Colorectal Cancer |
| title_sort | disentangled representation based multimodal fusion framework integrating pathomics and radiomics for kras mutation detection in colorectal cancer |
| topic | kras mutation detection multimodal feature fusion pathomics radiomics |
| url | https://www.sciopen.com/article/10.26599/BDMA.2024.9020012 |
| work_keys_str_mv | AT zhilonglv adisentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT ruiyan adisentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT yuexiaolin adisentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT lingao adisentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT fazhang adisentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT yingwang adisentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT zhilonglv disentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT ruiyan disentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT yuexiaolin disentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT lingao disentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT fazhang disentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer AT yingwang disentangledrepresentationbasedmultimodalfusionframeworkintegratingpathomicsandradiomicsforkrasmutationdetectionincolorectalcancer |