Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers
Depth cameras can provide an effective, noncontact, and privacy-preserving means to monitor patients in the Neonatal Intensive Care Unit (NICU). Clinical interventions and routine care events can disrupt video-based patient monitoring. Automatically detecting these periods can decrease the time requ...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/23/7753 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849220378163937280 |
|---|---|
| author | Zein Hajj-Ali Yasmina Souley Dosso Kim Greenwood JoAnn Harrold James R. Green |
| author_facet | Zein Hajj-Ali Yasmina Souley Dosso Kim Greenwood JoAnn Harrold James R. Green |
| author_sort | Zein Hajj-Ali |
| collection | DOAJ |
| description | Depth cameras can provide an effective, noncontact, and privacy-preserving means to monitor patients in the Neonatal Intensive Care Unit (NICU). Clinical interventions and routine care events can disrupt video-based patient monitoring. Automatically detecting these periods can decrease the time required for hand-annotating recordings, which is needed for system development. Moreover, the automatic detection can be used in the future for real-time or retrospective intervention event classification. An intervention detection method based solely on depth data was developed using a vision transformer (ViT) model utilizing real-world data from patients in the NICU. Multiple design parameters were investigated, including encoding of depth data and perspective transform to account for nonoptimal camera placement. The best-performing model utilized ∼85 M trainable parameters, leveraged both perspective transform and HHA (Horizontal disparity, Height above ground, and Angle with gravity) encoding, and achieved a sensitivity of 85.6%, a precision of 89.8%, and an F1-Score of 87.6%. |
| format | Article |
| id | doaj-art-61f71e33873d4e26a47f3ed1398301b4 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-61f71e33873d4e26a47f3ed1398301b42024-12-13T16:32:39ZengMDPI AGSensors1424-82202024-12-012423775310.3390/s24237753Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision TransformersZein Hajj-Ali0Yasmina Souley Dosso1Kim Greenwood2JoAnn Harrold3James R. Green4Systems and Computer Engineering, Carleton University, Ottawa, ON K1S 5B6, CanadaSystems and Computer Engineering, Carleton University, Ottawa, ON K1S 5B6, CanadaClinical Engineering, Children’s Hospital of Eastern Ontario, Ottawa, ON K1H 8L1, CanadaNeonatology, Children’s Hospital of Eastern Ontario, Ottawa, ON K1H 8L1, CanadaSystems and Computer Engineering, Carleton University, Ottawa, ON K1S 5B6, CanadaDepth cameras can provide an effective, noncontact, and privacy-preserving means to monitor patients in the Neonatal Intensive Care Unit (NICU). Clinical interventions and routine care events can disrupt video-based patient monitoring. Automatically detecting these periods can decrease the time required for hand-annotating recordings, which is needed for system development. Moreover, the automatic detection can be used in the future for real-time or retrospective intervention event classification. An intervention detection method based solely on depth data was developed using a vision transformer (ViT) model utilizing real-world data from patients in the NICU. Multiple design parameters were investigated, including encoding of depth data and perspective transform to account for nonoptimal camera placement. The best-performing model utilized ∼85 M trainable parameters, leveraged both perspective transform and HHA (Horizontal disparity, Height above ground, and Angle with gravity) encoding, and achieved a sensitivity of 85.6%, a precision of 89.8%, and an F1-Score of 87.6%.https://www.mdpi.com/1424-8220/24/23/7753depth cameraneonatal patient monitoringNICUtransformervision transformerViT |
| spellingShingle | Zein Hajj-Ali Yasmina Souley Dosso Kim Greenwood JoAnn Harrold James R. Green Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers Sensors depth camera neonatal patient monitoring NICU transformer vision transformer ViT |
| title | Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers |
| title_full | Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers |
| title_fullStr | Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers |
| title_full_unstemmed | Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers |
| title_short | Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers |
| title_sort | depth based intervention detection in the neonatal intensive care unit using vision transformers |
| topic | depth camera neonatal patient monitoring NICU transformer vision transformer ViT |
| url | https://www.mdpi.com/1424-8220/24/23/7753 |
| work_keys_str_mv | AT zeinhajjali depthbasedinterventiondetectionintheneonatalintensivecareunitusingvisiontransformers AT yasminasouleydosso depthbasedinterventiondetectionintheneonatalintensivecareunitusingvisiontransformers AT kimgreenwood depthbasedinterventiondetectionintheneonatalintensivecareunitusingvisiontransformers AT joannharrold depthbasedinterventiondetectionintheneonatalintensivecareunitusingvisiontransformers AT jamesrgreen depthbasedinterventiondetectionintheneonatalintensivecareunitusingvisiontransformers |