Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study
Objective To develop and validate a real-world screening, guideline-based deep learning (DL) system for referable diabetic retinopathy (DR) detection.Design This is a multicentre platform development study based on retrospective, cross-sectional data sets. Images were labelled by two-level certifica...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2022-07-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/12/7/e060155.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832576039570636800 |
|---|---|
| author | Chi Pui Pang Weiqi Chen Mingzhi Zhang Tsz Kin Ng Yi Zheng Guihua Zhang Jian-Wei Lin Ji Wang Jie Ji Ling-Ping Cen Peiwen Xie Yongqun Xiong Hanfu Wu Dongjie Li |
| author_facet | Chi Pui Pang Weiqi Chen Mingzhi Zhang Tsz Kin Ng Yi Zheng Guihua Zhang Jian-Wei Lin Ji Wang Jie Ji Ling-Ping Cen Peiwen Xie Yongqun Xiong Hanfu Wu Dongjie Li |
| author_sort | Chi Pui Pang |
| collection | DOAJ |
| description | Objective To develop and validate a real-world screening, guideline-based deep learning (DL) system for referable diabetic retinopathy (DR) detection.Design This is a multicentre platform development study based on retrospective, cross-sectional data sets. Images were labelled by two-level certificated graders as the ground truth. According to the UK DR screening guideline, a DL model based on colour retinal images with five-dimensional classifiers, namely image quality, retinopathy, maculopathy gradability, maculopathy and photocoagulation, was developed. Referable decisions were generated by integrating the output of all classifiers and reported at the image, eye and patient level. The performance of the DL was compared with DR experts.Setting DR screening programmes from three hospitals and the Lifeline Express Diabetic Retinopathy Screening Program in China.Participants 83 465 images of 39 836 eyes from 21 716 patients were annotated, of which 53 211 images were used as the development set and 30 254 images were used as the external validation set, split based on centre and period.Main outcomes Accuracy, F1 score, sensitivity, specificity, area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), Cohen’s unweighted κ and Gwet’s AC1 were calculated to evaluate the performance of the DL algorithm.Results In the external validation set, the five classifiers achieved an accuracy of 0.915–0.980, F1 score of 0.682–0.966, sensitivity of 0.917–0.978, specificity of 0.907–0.981, AUROC of 0.9639–0.9944 and AUPRC of 0.7504–0.9949. Referable DR at three levels was detected with an accuracy of 0.918–0.967, F1 score of 0.822–0.918, sensitivity of 0.970–0.971, specificity of 0.905–0.967, AUROC of 0.9848–0.9931 and AUPRC of 0.9527–0.9760. With reference to the ground truth, the DL system showed comparable performance (Cohen’s κ: 0.86–0.93; Gwet’s AC1: 0.89–0.94) with three DR experts (Cohen’s κ: 0.89–0.96; Gwet’s AC1: 0.91–0.97) in detecting referable lesions.Conclusions The automatic DL system for detection of referable DR based on the UK guideline could achieve high accuracy in multidimensional classifications. It is suitable for large-scale, real-world DR screening. |
| format | Article |
| id | doaj-art-34c5f4aa537543b380e9185baa8fe073 |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2022-07-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-34c5f4aa537543b380e9185baa8fe0732025-01-31T13:25:10ZengBMJ Publishing GroupBMJ Open2044-60552022-07-0112710.1136/bmjopen-2021-060155Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective studyChi Pui Pang0Weiqi Chen1Mingzhi Zhang2Tsz Kin Ng3Yi Zheng4Guihua Zhang5Jian-Wei Lin6Ji Wang7Jie Ji8Ling-Ping Cen9Peiwen Xie10Yongqun Xiong11Hanfu Wu12Dongjie Li13Department of Ophthalmology and Visual Sciences, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong, Hong KongJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, China1 Department of Oncology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, China3Department of Clinical Pharmacy, Key Laboratory of Chemical Biology (Ministry of Education), School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, ChinaJoint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong, Shantou, ChinaDepartment of Respiratory and Critical Care Medicine, Clinical Research Center for Respiratory Diseases, West China Hospital, Sichuan University, Chengdu, ChinaThe big data center, Shantou University Medical College, Shantou, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, ChinaJoint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong, Shantou, ChinaObjective To develop and validate a real-world screening, guideline-based deep learning (DL) system for referable diabetic retinopathy (DR) detection.Design This is a multicentre platform development study based on retrospective, cross-sectional data sets. Images were labelled by two-level certificated graders as the ground truth. According to the UK DR screening guideline, a DL model based on colour retinal images with five-dimensional classifiers, namely image quality, retinopathy, maculopathy gradability, maculopathy and photocoagulation, was developed. Referable decisions were generated by integrating the output of all classifiers and reported at the image, eye and patient level. The performance of the DL was compared with DR experts.Setting DR screening programmes from three hospitals and the Lifeline Express Diabetic Retinopathy Screening Program in China.Participants 83 465 images of 39 836 eyes from 21 716 patients were annotated, of which 53 211 images were used as the development set and 30 254 images were used as the external validation set, split based on centre and period.Main outcomes Accuracy, F1 score, sensitivity, specificity, area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), Cohen’s unweighted κ and Gwet’s AC1 were calculated to evaluate the performance of the DL algorithm.Results In the external validation set, the five classifiers achieved an accuracy of 0.915–0.980, F1 score of 0.682–0.966, sensitivity of 0.917–0.978, specificity of 0.907–0.981, AUROC of 0.9639–0.9944 and AUPRC of 0.7504–0.9949. Referable DR at three levels was detected with an accuracy of 0.918–0.967, F1 score of 0.822–0.918, sensitivity of 0.970–0.971, specificity of 0.905–0.967, AUROC of 0.9848–0.9931 and AUPRC of 0.9527–0.9760. With reference to the ground truth, the DL system showed comparable performance (Cohen’s κ: 0.86–0.93; Gwet’s AC1: 0.89–0.94) with three DR experts (Cohen’s κ: 0.89–0.96; Gwet’s AC1: 0.91–0.97) in detecting referable lesions.Conclusions The automatic DL system for detection of referable DR based on the UK guideline could achieve high accuracy in multidimensional classifications. It is suitable for large-scale, real-world DR screening.https://bmjopen.bmj.com/content/12/7/e060155.full |
| spellingShingle | Chi Pui Pang Weiqi Chen Mingzhi Zhang Tsz Kin Ng Yi Zheng Guihua Zhang Jian-Wei Lin Ji Wang Jie Ji Ling-Ping Cen Peiwen Xie Yongqun Xiong Hanfu Wu Dongjie Li Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study BMJ Open |
| title | Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study |
| title_full | Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study |
| title_fullStr | Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study |
| title_full_unstemmed | Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study |
| title_short | Automated multidimensional deep learning platform for referable diabetic retinopathy detection: a multicentre, retrospective study |
| title_sort | automated multidimensional deep learning platform for referable diabetic retinopathy detection a multicentre retrospective study |
| url | https://bmjopen.bmj.com/content/12/7/e060155.full |
| work_keys_str_mv | AT chipuipang automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT weiqichen automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT mingzhizhang automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT tszkinng automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT yizheng automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT guihuazhang automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT jianweilin automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT jiwang automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT jieji automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT lingpingcen automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT peiwenxie automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT yongqunxiong automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT hanfuwu automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy AT dongjieli automatedmultidimensionaldeeplearningplatformforreferablediabeticretinopathydetectionamulticentreretrospectivestudy |