Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review
<i>Background</i>: Electronic health records (EHR) are now widely available in healthcare institutions to document the medical history of patients as they interact with healthcare services. In particular, routine care EHR data are collected for a large number of patients.These data span...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/16/1/54 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588347440103424 |
|---|---|
| author | Zina Ben-Miled Jacob A. Shebesh Jing Su Paul R. Dexter Randall W. Grout Malaz A. Boustani |
| author_facet | Zina Ben-Miled Jacob A. Shebesh Jing Su Paul R. Dexter Randall W. Grout Malaz A. Boustani |
| author_sort | Zina Ben-Miled |
| collection | DOAJ |
| description | <i>Background</i>: Electronic health records (EHR) are now widely available in healthcare institutions to document the medical history of patients as they interact with healthcare services. In particular, routine care EHR data are collected for a large number of patients.These data span multiple heterogeneous elements (i.e., demographics, diagnosis, medications, clinical notes, vital signs, and laboratory results) which contain semantic, concept, and temporal information. Recent advances in generative learning techniques were able to leverage the fusion of multiple routine care EHR data elements to enhance clinical decision support. <i>Objective</i>: A scoping review of the proposed techniques including fusion architectures, input data elements, and application areas is needed to synthesize variances and identify research gaps that can promote re-use of these techniques for new clinical outcomes. <i>Design</i>: A comprehensive literature search was conducted using Google Scholar to identify high impact fusion architectures over multi-modal routine care EHR data during the period 2018 to 2023. The guidelines from the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping review were followed. The findings were derived from the selected studies using a thematic and comparative analysis. <i>Results</i>: The scoping review revealed the lack of standard definition for EHR data elements as they are transformed into input modalities. These definitions ignore one or more key characteristics of the data including source, encoding scheme, and concept level. Moreover, in order to adapt to emergent generative learning techniques, the classification of fusion architectures should distinguish fusion from learning and take into consideration that learning can concurrently happen in all three layers of new fusion architectures (i.e., encoding, representation, and decision). These aspects constitute the first step towards a streamlined approach to the design of multi-modal fusion architectures for routine care EHR data. In addition, current pretrained encoding models are inconsistent in their handling of temporal and semantic information thereby hindering their re-use for different applications and clinical settings. <i>Conclusions</i>: Current routine care EHR fusion architectures mostly follow a design-by-example methodology. Guidelines are needed for the design of efficient multi-modal models for a broad range of healthcare applications. In addition to promoting re-use, these guidelines need to outline best practices for combining multiple modalities while leveraging transfer learning and co-learning as well as semantic and temporal encoding. |
| format | Article |
| id | doaj-art-6c53026226c2415f9fc2fc84f8112d16 |
| institution | Kabale University |
| issn | 2078-2489 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj-art-6c53026226c2415f9fc2fc84f8112d162025-01-24T13:35:17ZengMDPI AGInformation2078-24892025-01-011615410.3390/info16010054Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping ReviewZina Ben-Miled0Jacob A. Shebesh1Jing Su2Paul R. Dexter3Randall W. Grout4Malaz A. Boustani5Phillip M. Drayer Department of Electrical and Computer Engineering, Lamar University, Cherry Building, Beaumont, TX 77705, USADepartment of Electrical and Computer Engineering, School of Engineering and Technology, Indiana University Purdue University at Indianapolis, 723 W. Michigan Street, Indianapolis, IN 46202, USAIndiana University School of Medicine, Indiana University, 340 W 10th St, Indianapolis, IN 46202, USAIndiana University School of Medicine, Indiana University, 340 W 10th St, Indianapolis, IN 46202, USAIndiana University School of Medicine, Indiana University, 340 W 10th St, Indianapolis, IN 46202, USAIndiana University School of Medicine, Indiana University, 340 W 10th St, Indianapolis, IN 46202, USA<i>Background</i>: Electronic health records (EHR) are now widely available in healthcare institutions to document the medical history of patients as they interact with healthcare services. In particular, routine care EHR data are collected for a large number of patients.These data span multiple heterogeneous elements (i.e., demographics, diagnosis, medications, clinical notes, vital signs, and laboratory results) which contain semantic, concept, and temporal information. Recent advances in generative learning techniques were able to leverage the fusion of multiple routine care EHR data elements to enhance clinical decision support. <i>Objective</i>: A scoping review of the proposed techniques including fusion architectures, input data elements, and application areas is needed to synthesize variances and identify research gaps that can promote re-use of these techniques for new clinical outcomes. <i>Design</i>: A comprehensive literature search was conducted using Google Scholar to identify high impact fusion architectures over multi-modal routine care EHR data during the period 2018 to 2023. The guidelines from the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping review were followed. The findings were derived from the selected studies using a thematic and comparative analysis. <i>Results</i>: The scoping review revealed the lack of standard definition for EHR data elements as they are transformed into input modalities. These definitions ignore one or more key characteristics of the data including source, encoding scheme, and concept level. Moreover, in order to adapt to emergent generative learning techniques, the classification of fusion architectures should distinguish fusion from learning and take into consideration that learning can concurrently happen in all three layers of new fusion architectures (i.e., encoding, representation, and decision). These aspects constitute the first step towards a streamlined approach to the design of multi-modal fusion architectures for routine care EHR data. In addition, current pretrained encoding models are inconsistent in their handling of temporal and semantic information thereby hindering their re-use for different applications and clinical settings. <i>Conclusions</i>: Current routine care EHR fusion architectures mostly follow a design-by-example methodology. Guidelines are needed for the design of efficient multi-modal models for a broad range of healthcare applications. In addition to promoting re-use, these guidelines need to outline best practices for combining multiple modalities while leveraging transfer learning and co-learning as well as semantic and temporal encoding.https://www.mdpi.com/2078-2489/16/1/54multi-modal fusionelectronic health recordsmachine learningtransformersmodality |
| spellingShingle | Zina Ben-Miled Jacob A. Shebesh Jing Su Paul R. Dexter Randall W. Grout Malaz A. Boustani Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review Information multi-modal fusion electronic health records machine learning transformers modality |
| title | Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review |
| title_full | Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review |
| title_fullStr | Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review |
| title_full_unstemmed | Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review |
| title_short | Multi-Modal Fusion of Routine Care Electronic Health Records (EHR): A Scoping Review |
| title_sort | multi modal fusion of routine care electronic health records ehr a scoping review |
| topic | multi-modal fusion electronic health records machine learning transformers modality |
| url | https://www.mdpi.com/2078-2489/16/1/54 |
| work_keys_str_mv | AT zinabenmiled multimodalfusionofroutinecareelectronichealthrecordsehrascopingreview AT jacobashebesh multimodalfusionofroutinecareelectronichealthrecordsehrascopingreview AT jingsu multimodalfusionofroutinecareelectronichealthrecordsehrascopingreview AT paulrdexter multimodalfusionofroutinecareelectronichealthrecordsehrascopingreview AT randallwgrout multimodalfusionofroutinecareelectronichealthrecordsehrascopingreview AT malazaboustani multimodalfusionofroutinecareelectronichealthrecordsehrascopingreview |