Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions
Blink detection is considered a useful indicator both for clinical conditions and drowsiness state. In this work, we propose and compare deep learning architectures for the task of detecting blinks in video frame sequences. The first step is the training and application of an eye detector that extra...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/11/1/27 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588250938605568 |
|---|---|
| author | George Nousias Konstantinos K. Delibasis Georgios Labiris |
| author_facet | George Nousias Konstantinos K. Delibasis Georgios Labiris |
| author_sort | George Nousias |
| collection | DOAJ |
| description | Blink detection is considered a useful indicator both for clinical conditions and drowsiness state. In this work, we propose and compare deep learning architectures for the task of detecting blinks in video frame sequences. The first step is the training and application of an eye detector that extracts the eye regions from each video frame. The cropped eye regions are organized as three-dimensional (3D) input with the third dimension spanning time of 300 ms. Two different 3D convolutional neural networks are utilized (a simple 3D CNN and 3D ResNet), as well as a 3D autoencoder combined with a classifier coupled to the latent space. Finally, we propose the usage of a frame prediction accumulator combined with morphological processing and watershed segmentation to detect blinks and determine their start and stop frame in previously unseen videos. The proposed framework was trained on ten (9) different participants and tested on five (8) different ones, with a total of 162,400 frames and 1172 blinks for each eye. The start and end frame of each blink in the dataset has been annotate by specialized ophthalmologist. Quantitative comparison with state-of-the-art blink detection methodologies provide favorable results for the proposed neural architectures coupled with the prediction accumulator, with the 3D ResNet being the best as well as the fastest performer. |
| format | Article |
| id | doaj-art-0a96ad84735546dbb26c274ebd9773e2 |
| institution | Kabale University |
| issn | 2313-433X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj-art-0a96ad84735546dbb26c274ebd9773e22025-01-24T13:36:19ZengMDPI AGJournal of Imaging2313-433X2025-01-011112710.3390/jimaging11010027Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame PredictionsGeorge Nousias0Konstantinos K. Delibasis1Georgios Labiris2Department of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, GreeceDepartment of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, GreeceDepartment of Ophthalmology, General University Hospital of Alexandroupolis, 68131 Alexandroupolis, GreeceBlink detection is considered a useful indicator both for clinical conditions and drowsiness state. In this work, we propose and compare deep learning architectures for the task of detecting blinks in video frame sequences. The first step is the training and application of an eye detector that extracts the eye regions from each video frame. The cropped eye regions are organized as three-dimensional (3D) input with the third dimension spanning time of 300 ms. Two different 3D convolutional neural networks are utilized (a simple 3D CNN and 3D ResNet), as well as a 3D autoencoder combined with a classifier coupled to the latent space. Finally, we propose the usage of a frame prediction accumulator combined with morphological processing and watershed segmentation to detect blinks and determine their start and stop frame in previously unseen videos. The proposed framework was trained on ten (9) different participants and tested on five (8) different ones, with a total of 162,400 frames and 1172 blinks for each eye. The start and end frame of each blink in the dataset has been annotate by specialized ophthalmologist. Quantitative comparison with state-of-the-art blink detection methodologies provide favorable results for the proposed neural architectures coupled with the prediction accumulator, with the 3D ResNet being the best as well as the fastest performer.https://www.mdpi.com/2313-433X/11/1/27blink detection3D CNN3D autoencoder3D ResNetprediction accumulatorsignal analysis |
| spellingShingle | George Nousias Konstantinos K. Delibasis Georgios Labiris Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions Journal of Imaging blink detection 3D CNN 3D autoencoder 3D ResNet prediction accumulator signal analysis |
| title | Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions |
| title_full | Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions |
| title_fullStr | Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions |
| title_full_unstemmed | Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions |
| title_short | Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions |
| title_sort | blink detection using 3d convolutional neural architectures and analysis of accumulated frame predictions |
| topic | blink detection 3D CNN 3D autoencoder 3D ResNet prediction accumulator signal analysis |
| url | https://www.mdpi.com/2313-433X/11/1/27 |
| work_keys_str_mv | AT georgenousias blinkdetectionusing3dconvolutionalneuralarchitecturesandanalysisofaccumulatedframepredictions AT konstantinoskdelibasis blinkdetectionusing3dconvolutionalneuralarchitecturesandanalysisofaccumulatedframepredictions AT georgioslabiris blinkdetectionusing3dconvolutionalneuralarchitecturesandanalysisofaccumulatedframepredictions |