Early stroke behavior detection based on improved video masked autoencoders for potential patients
Abstract Stroke is the prevalent cerebrovascular disease characterized by significant incidence and disability rates. To enhance the early perceive and detection of potential stroke patients, the early stroke behavior detection based on improved Video Masked Autoencoders (VideoMAE) for potential pat...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01610-0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571189337260032 |
|---|---|
| author | Meng Wang Guanci Yang Kexin Luo Yang Li Ling He |
| author_facet | Meng Wang Guanci Yang Kexin Luo Yang Li Ling He |
| author_sort | Meng Wang |
| collection | DOAJ |
| description | Abstract Stroke is the prevalent cerebrovascular disease characterized by significant incidence and disability rates. To enhance the early perceive and detection of potential stroke patients, the early stroke behavior detection based on improved Video Masked Autoencoders (VideoMAE) for potential patients (EPBR-PS) is proposed. The proposed method begins with novel time interval-based sampling strategy, capturing video frame sequences enriched with sparse motion features. On the basis of establishing the masking mechanism for adjacent frames and pixel blocks within these sequences, The EPBR-PS employes pipeline mask strategy to extract spatiotemporal features effectively. Then, the local convolution attention mechanism is designed to capture local dynamic feature information, and central to the EPBR-PS is the integration of local convolutional attention mechanism with VideoMAE's multi-head attention mechanism. This integration facilitates the simultaneous leveraging of global high-level semantics and local dynamic feature information. Dual attention mechanism-based method for the fusion of these global and local features is proposed. After that, the optimal parameters of EPBR-PS were determined through the experiment of learning rate and fusion weights of different features. On the NTU-ST dataset, comparative analysis with eight models demonstrated the superiority of EPBR-PS, evidenced by the average recognition accuracy of 89.61%, surpassing that 1.67% over the benchmark VideoMAE. On the HMDB51 dataset, EPBR-PS has Top1 of 71.31%, which is 0.73% higher than that of the VideoMAE, providing the viable behavior detection for perception early signs of potential stroke in the home environment. This code is available at https://github.com/wang-325/EPBR-PS/ . |
| format | Article |
| id | doaj-art-a8d9ee27e37b4e74bbaef1675aef5dd3 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-a8d9ee27e37b4e74bbaef1675aef5dd32025-02-02T12:49:15ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111410.1007/s40747-024-01610-0Early stroke behavior detection based on improved video masked autoencoders for potential patientsMeng Wang0Guanci Yang1Kexin Luo2Yang Li3Ling He4Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou UniversityKey Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou UniversityKey Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou UniversityKey Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou UniversityKey Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou UniversityAbstract Stroke is the prevalent cerebrovascular disease characterized by significant incidence and disability rates. To enhance the early perceive and detection of potential stroke patients, the early stroke behavior detection based on improved Video Masked Autoencoders (VideoMAE) for potential patients (EPBR-PS) is proposed. The proposed method begins with novel time interval-based sampling strategy, capturing video frame sequences enriched with sparse motion features. On the basis of establishing the masking mechanism for adjacent frames and pixel blocks within these sequences, The EPBR-PS employes pipeline mask strategy to extract spatiotemporal features effectively. Then, the local convolution attention mechanism is designed to capture local dynamic feature information, and central to the EPBR-PS is the integration of local convolutional attention mechanism with VideoMAE's multi-head attention mechanism. This integration facilitates the simultaneous leveraging of global high-level semantics and local dynamic feature information. Dual attention mechanism-based method for the fusion of these global and local features is proposed. After that, the optimal parameters of EPBR-PS were determined through the experiment of learning rate and fusion weights of different features. On the NTU-ST dataset, comparative analysis with eight models demonstrated the superiority of EPBR-PS, evidenced by the average recognition accuracy of 89.61%, surpassing that 1.67% over the benchmark VideoMAE. On the HMDB51 dataset, EPBR-PS has Top1 of 71.31%, which is 0.73% higher than that of the VideoMAE, providing the viable behavior detection for perception early signs of potential stroke in the home environment. This code is available at https://github.com/wang-325/EPBR-PS/ .https://doi.org/10.1007/s40747-024-01610-0Stroke behavior recognitionComputer visionSpatiotemporal feature extractionEarly stroke perception |
| spellingShingle | Meng Wang Guanci Yang Kexin Luo Yang Li Ling He Early stroke behavior detection based on improved video masked autoencoders for potential patients Complex & Intelligent Systems Stroke behavior recognition Computer vision Spatiotemporal feature extraction Early stroke perception |
| title | Early stroke behavior detection based on improved video masked autoencoders for potential patients |
| title_full | Early stroke behavior detection based on improved video masked autoencoders for potential patients |
| title_fullStr | Early stroke behavior detection based on improved video masked autoencoders for potential patients |
| title_full_unstemmed | Early stroke behavior detection based on improved video masked autoencoders for potential patients |
| title_short | Early stroke behavior detection based on improved video masked autoencoders for potential patients |
| title_sort | early stroke behavior detection based on improved video masked autoencoders for potential patients |
| topic | Stroke behavior recognition Computer vision Spatiotemporal feature extraction Early stroke perception |
| url | https://doi.org/10.1007/s40747-024-01610-0 |
| work_keys_str_mv | AT mengwang earlystrokebehaviordetectionbasedonimprovedvideomaskedautoencodersforpotentialpatients AT guanciyang earlystrokebehaviordetectionbasedonimprovedvideomaskedautoencodersforpotentialpatients AT kexinluo earlystrokebehaviordetectionbasedonimprovedvideomaskedautoencodersforpotentialpatients AT yangli earlystrokebehaviordetectionbasedonimprovedvideomaskedautoencodersforpotentialpatients AT linghe earlystrokebehaviordetectionbasedonimprovedvideomaskedautoencodersforpotentialpatients |