A Multimodal Information Fusion Model for Robot Action Recognition with Time Series
The current robotics field, led by a new generation of information technology, is moving into a new stage of human-machine collaborative operation. Unlike traditional robots that need to use isolation rails to maintain a certain safety distance from people, the new generation of human-machine collab...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-01-01
|
| Series: | Journal of Electrical and Computer Engineering |
| Online Access: | http://dx.doi.org/10.1155/2022/7270412 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832562212039819264 |
|---|---|
| author | Xiaozhi Zhang Hongyan Li Mengjie Qian |
| author_facet | Xiaozhi Zhang Hongyan Li Mengjie Qian |
| author_sort | Xiaozhi Zhang |
| collection | DOAJ |
| description | The current robotics field, led by a new generation of information technology, is moving into a new stage of human-machine collaborative operation. Unlike traditional robots that need to use isolation rails to maintain a certain safety distance from people, the new generation of human-machine collaboration systems can work side by side with humans without spatial obstruction, giving full play to the expertise of people and machines through an intelligent assignment of operational tasks and improving work patterns to achieve increased efficiency. The robot’s efficient and accurate recognition of human movements has become a key factor in measuring robot performance. Usually, the data for action recognition is video data, and video data is time-series data. Time series describe the response results of a certain system at different times. Therefore, the study of time series can be used to recognize the structural characteristics of the system and reveal its operation law. As a result, this paper proposes a time series-based action recognition model with multimodal information fusion and applies it to a robot to realize friendly human-robot interaction. Multifeatures can characterize data information comprehensively, and in this study, the spatial flow and motion flow features of the dataset are extracted separately, and each feature is input into a bidirectional long and short-term memory network (BiLSTM). A confidence fusion method was used to obtain the final action recognition results. Experiment results on the publicly available datasets NTU-RGB + D and MSR Action 3D show that the method proposed in this paper can improve action recognition accuracy. |
| format | Article |
| id | doaj-art-b5e0570e7a5b4584acff9398c6486f52 |
| institution | Kabale University |
| issn | 2090-0155 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Journal of Electrical and Computer Engineering |
| spelling | doaj-art-b5e0570e7a5b4584acff9398c6486f522025-02-03T01:23:09ZengWileyJournal of Electrical and Computer Engineering2090-01552022-01-01202210.1155/2022/7270412A Multimodal Information Fusion Model for Robot Action Recognition with Time SeriesXiaozhi Zhang0Hongyan Li1Mengjie Qian2Information Engineering DepartmentInformation Engineering DepartmentInformation Engineering DepartmentThe current robotics field, led by a new generation of information technology, is moving into a new stage of human-machine collaborative operation. Unlike traditional robots that need to use isolation rails to maintain a certain safety distance from people, the new generation of human-machine collaboration systems can work side by side with humans without spatial obstruction, giving full play to the expertise of people and machines through an intelligent assignment of operational tasks and improving work patterns to achieve increased efficiency. The robot’s efficient and accurate recognition of human movements has become a key factor in measuring robot performance. Usually, the data for action recognition is video data, and video data is time-series data. Time series describe the response results of a certain system at different times. Therefore, the study of time series can be used to recognize the structural characteristics of the system and reveal its operation law. As a result, this paper proposes a time series-based action recognition model with multimodal information fusion and applies it to a robot to realize friendly human-robot interaction. Multifeatures can characterize data information comprehensively, and in this study, the spatial flow and motion flow features of the dataset are extracted separately, and each feature is input into a bidirectional long and short-term memory network (BiLSTM). A confidence fusion method was used to obtain the final action recognition results. Experiment results on the publicly available datasets NTU-RGB + D and MSR Action 3D show that the method proposed in this paper can improve action recognition accuracy.http://dx.doi.org/10.1155/2022/7270412 |
| spellingShingle | Xiaozhi Zhang Hongyan Li Mengjie Qian A Multimodal Information Fusion Model for Robot Action Recognition with Time Series Journal of Electrical and Computer Engineering |
| title | A Multimodal Information Fusion Model for Robot Action Recognition with Time Series |
| title_full | A Multimodal Information Fusion Model for Robot Action Recognition with Time Series |
| title_fullStr | A Multimodal Information Fusion Model for Robot Action Recognition with Time Series |
| title_full_unstemmed | A Multimodal Information Fusion Model for Robot Action Recognition with Time Series |
| title_short | A Multimodal Information Fusion Model for Robot Action Recognition with Time Series |
| title_sort | multimodal information fusion model for robot action recognition with time series |
| url | http://dx.doi.org/10.1155/2022/7270412 |
| work_keys_str_mv | AT xiaozhizhang amultimodalinformationfusionmodelforrobotactionrecognitionwithtimeseries AT hongyanli amultimodalinformationfusionmodelforrobotactionrecognitionwithtimeseries AT mengjieqian amultimodalinformationfusionmodelforrobotactionrecognitionwithtimeseries AT xiaozhizhang multimodalinformationfusionmodelforrobotactionrecognitionwithtimeseries AT hongyanli multimodalinformationfusionmodelforrobotactionrecognitionwithtimeseries AT mengjieqian multimodalinformationfusionmodelforrobotactionrecognitionwithtimeseries |