Study on video action recognition based on augment negative example multi-granularity discrimination model
An augment negative example discrimination paradigm based on contrastive learning was proposed to improve the model’s fine-grained discrimination ability of video actions. The most challenging video-text negative pairs was generated, forming an augmented negative example set for each video sample. B...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Journal on Communications
2024-12-01
|
| Series: | Tongxin xuebao |
| Subjects: | |
| Online Access: | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2024268/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832595464899264512 |
|---|---|
| author | LIU Liangzhen YANG Yang XIA Yingjie KUANG Li |
| author_facet | LIU Liangzhen YANG Yang XIA Yingjie KUANG Li |
| author_sort | LIU Liangzhen |
| collection | DOAJ |
| description | An augment negative example discrimination paradigm based on contrastive learning was proposed to improve the model’s fine-grained discrimination ability of video actions. The most challenging video-text negative pairs was generated, forming an augmented negative example set for each video sample. Based on this paradigm, a multi-granularity discrimination model for video action recognition was proposed to further distinguish between positive and negative examples. In this model, video features were extracted by the video representation module guided by textual positive examples, while self-correlation relationships between positive and negative semantics were established by the semantic discriminator equipped with a self-attention mechanism. Meanwhile, a coarse-grained distinction between the video modality and the augmented negative example set was achieved, while a fine-grained distinction between positive examples and the augmented negative example set within the text modality was also accomplished. Experimental results demonstrate that the augment negative set improves the model’s recognition ability on fine-grained class labels, and the multi-granularity discrimination model outperforms current state-of-the-art methods on the Kinetics-400, HMDB51 and UCF101 datasets. |
| format | Article |
| id | doaj-art-4f0050310c034a7a8aabf7d02714e1df |
| institution | Kabale University |
| issn | 1000-436X |
| language | zho |
| publishDate | 2024-12-01 |
| publisher | Editorial Department of Journal on Communications |
| record_format | Article |
| series | Tongxin xuebao |
| spelling | doaj-art-4f0050310c034a7a8aabf7d02714e1df2025-01-18T19:00:07ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2024-12-0145284380268822Study on video action recognition based on augment negative example multi-granularity discrimination modelLIU LiangzhenYANG YangXIA YingjieKUANG LiAn augment negative example discrimination paradigm based on contrastive learning was proposed to improve the model’s fine-grained discrimination ability of video actions. The most challenging video-text negative pairs was generated, forming an augmented negative example set for each video sample. Based on this paradigm, a multi-granularity discrimination model for video action recognition was proposed to further distinguish between positive and negative examples. In this model, video features were extracted by the video representation module guided by textual positive examples, while self-correlation relationships between positive and negative semantics were established by the semantic discriminator equipped with a self-attention mechanism. Meanwhile, a coarse-grained distinction between the video modality and the augmented negative example set was achieved, while a fine-grained distinction between positive examples and the augmented negative example set within the text modality was also accomplished. Experimental results demonstrate that the augment negative set improves the model’s recognition ability on fine-grained class labels, and the multi-granularity discrimination model outperforms current state-of-the-art methods on the Kinetics-400, HMDB51 and UCF101 datasets.http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2024268/contrastive learningaugmented negative examplesparadigmvideo action recognition |
| spellingShingle | LIU Liangzhen YANG Yang XIA Yingjie KUANG Li Study on video action recognition based on augment negative example multi-granularity discrimination model Tongxin xuebao contrastive learning augmented negative examples paradigm video action recognition |
| title | Study on video action recognition based on augment negative example multi-granularity discrimination model |
| title_full | Study on video action recognition based on augment negative example multi-granularity discrimination model |
| title_fullStr | Study on video action recognition based on augment negative example multi-granularity discrimination model |
| title_full_unstemmed | Study on video action recognition based on augment negative example multi-granularity discrimination model |
| title_short | Study on video action recognition based on augment negative example multi-granularity discrimination model |
| title_sort | study on video action recognition based on augment negative example multi granularity discrimination model |
| topic | contrastive learning augmented negative examples paradigm video action recognition |
| url | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2024268/ |
| work_keys_str_mv | AT liuliangzhen studyonvideoactionrecognitionbasedonaugmentnegativeexamplemultigranularitydiscriminationmodel AT yangyang studyonvideoactionrecognitionbasedonaugmentnegativeexamplemultigranularitydiscriminationmodel AT xiayingjie studyonvideoactionrecognitionbasedonaugmentnegativeexamplemultigranularitydiscriminationmodel AT kuangli studyonvideoactionrecognitionbasedonaugmentnegativeexamplemultigranularitydiscriminationmodel |