Multimodal machine learning enables AI chatbot to diagnose ophthalmic diseases and provide high-quality medical responses

Multimodal machine learning enables AI chatbot to diagnose ophthalmic diseases and provide high-quality medical responses

Abstract Chatbot-based multimodal AI holds promise for collecting medical histories and diagnosing ophthalmic diseases using textual and imaging data. This study developed and evaluated the ChatGPT-powered Intelligent Ophthalmic Multimodal Interactive Diagnostic System (IOMIDS) to enable patient sel...

Full description

Saved in:
Bibliographic Details
Main Authors: Ruiqi Ma, Qian Cheng, Jing Yao, Zhiyu Peng, Mingxu Yan, Jie Lu, Jingjing Liao, Lejin Tian, Wenjun Shu, Yunqiu Zhang, Jinghan Wang, Pengfei Jiang, Weiyi Xia, Xiaofeng Li, Lu Gan, Yue Zhao, Jiang Zhu, Bing Qin, Qin Jiang, Xiawei Wang, Xintong Lin, Haifeng Chen, Weifang Zhu, Dehui Xiang, Baoqing Nie, Jingtao Wang, Jie Guo, Kang Xue, Hongguang Cui, Jinwei Cheng, Xiangjia Zhu, Jiaxu Hong, Fei Shi, Rui Zhang, Xinjian Chen, Chen Zhao
Format: Article
Language:English
Published: Nature Portfolio 2025-01-01
Series:npj Digital Medicine
Online Access:https://doi.org/10.1038/s41746-025-01461-0
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Abstract Chatbot-based multimodal AI holds promise for collecting medical histories and diagnosing ophthalmic diseases using textual and imaging data. This study developed and evaluated the ChatGPT-powered Intelligent Ophthalmic Multimodal Interactive Diagnostic System (IOMIDS) to enable patient self-diagnosis and self-triage. IOMIDS included a text model and three multimodal models (text + slit-lamp, text + smartphone, text + slit-lamp + smartphone). The performance was evaluated through a two-stage cross-sectional study across three medical centers involving 10 subspecialties and 50 diseases. Using 15640 data entries, IOMIDS actively collected and analyzed medical history alongside slit-lamp and/or smartphone images. The text + smartphone model showed the highest diagnostic accuracy (internal: 79.6%, external: 81.1%), while other multimodal models underperformed or matched the text model (internal: 69.6%, external: 72.5%). Moreover, triage accuracy was consistent across models. Multimodal approaches enhanced response quality and reduced misinformation. This proof-of-concept study highlights the potential of chatbot-based multimodal AI for self-diagnosis and self-triage. (The clinical trial was registered on June 26, 2023, on ClinicalTrials.gov under the registration number NCT05930444.).
ISSN:2398-6352

Similar Items