Attention-based interactive multi-level feature fusion for named entity recognition
Abstract Named Entity Recognition (NER) is an essential component of numerous Natural Language Processing (NLP) systems, with the aim of identifying and classifying entities that have specific meanings in raw text, such as person (PER), location (LOC), and organization (ORG). Recently, Deep Neural N...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-86718-0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585774146519040 |
|---|---|
| author | Yiwu Xu Yun Chen |
| author_facet | Yiwu Xu Yun Chen |
| author_sort | Yiwu Xu |
| collection | DOAJ |
| description | Abstract Named Entity Recognition (NER) is an essential component of numerous Natural Language Processing (NLP) systems, with the aim of identifying and classifying entities that have specific meanings in raw text, such as person (PER), location (LOC), and organization (ORG). Recently, Deep Neural Networks (DNNs) have been extensively applied to NER tasks owing to the rapid development of deep learning technology. However, despite their advancements, these models fail to take full advantage of the multi-level features (e.g., lexical phrases, keywords, capitalization, suffixes, etc.) of entities and the dependencies between different features. To address this issue, we propose a novel attention-based interactive multi-level feature fusion (AIMFF) framework, which aims to improve NER by obtaining multi-level features from different perspectives and interactively capturing the dependencies between different features. Our model is composed of four parts: the input, feature extraction, feature fusion, and sequence-labeling layers. First, we generate the original word- and character-level embeddings in the input layer. Then, we incorporate four parallel components to capture global word-level, local word-level, global character-level, and local character-level features in the feature extraction layer to enrich word embeddings with comprehensive multi-level semantic features. Next, we adopt cross-attention in the feature fusion layer to fuse features by exploiting the interaction between word- and character-level features. Finally, the fused features are fed into the sequence labeling layer to predict the word labels. We conducted generous comparative experiments on three datasets, and the experimental results showed that our model achieved better performance than several state-of-the-art models. |
| format | Article |
| id | doaj-art-5b5cf4384b7e449b8eeed43170cf8fdb |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-5b5cf4384b7e449b8eeed43170cf8fdb2025-01-26T12:29:38ZengNature PortfolioScientific Reports2045-23222025-01-0115111610.1038/s41598-025-86718-0Attention-based interactive multi-level feature fusion for named entity recognitionYiwu Xu0Yun Chen1Guangzhou Institute of Science and TechnologyNanfang College GuangzhouAbstract Named Entity Recognition (NER) is an essential component of numerous Natural Language Processing (NLP) systems, with the aim of identifying and classifying entities that have specific meanings in raw text, such as person (PER), location (LOC), and organization (ORG). Recently, Deep Neural Networks (DNNs) have been extensively applied to NER tasks owing to the rapid development of deep learning technology. However, despite their advancements, these models fail to take full advantage of the multi-level features (e.g., lexical phrases, keywords, capitalization, suffixes, etc.) of entities and the dependencies between different features. To address this issue, we propose a novel attention-based interactive multi-level feature fusion (AIMFF) framework, which aims to improve NER by obtaining multi-level features from different perspectives and interactively capturing the dependencies between different features. Our model is composed of four parts: the input, feature extraction, feature fusion, and sequence-labeling layers. First, we generate the original word- and character-level embeddings in the input layer. Then, we incorporate four parallel components to capture global word-level, local word-level, global character-level, and local character-level features in the feature extraction layer to enrich word embeddings with comprehensive multi-level semantic features. Next, we adopt cross-attention in the feature fusion layer to fuse features by exploiting the interaction between word- and character-level features. Finally, the fused features are fed into the sequence labeling layer to predict the word labels. We conducted generous comparative experiments on three datasets, and the experimental results showed that our model achieved better performance than several state-of-the-art models.https://doi.org/10.1038/s41598-025-86718-0Named entity recognitionMulti-level featuresCross-attentionFeature fusion |
| spellingShingle | Yiwu Xu Yun Chen Attention-based interactive multi-level feature fusion for named entity recognition Scientific Reports Named entity recognition Multi-level features Cross-attention Feature fusion |
| title | Attention-based interactive multi-level feature fusion for named entity recognition |
| title_full | Attention-based interactive multi-level feature fusion for named entity recognition |
| title_fullStr | Attention-based interactive multi-level feature fusion for named entity recognition |
| title_full_unstemmed | Attention-based interactive multi-level feature fusion for named entity recognition |
| title_short | Attention-based interactive multi-level feature fusion for named entity recognition |
| title_sort | attention based interactive multi level feature fusion for named entity recognition |
| topic | Named entity recognition Multi-level features Cross-attention Feature fusion |
| url | https://doi.org/10.1038/s41598-025-86718-0 |
| work_keys_str_mv | AT yiwuxu attentionbasedinteractivemultilevelfeaturefusionfornamedentityrecognition AT yunchen attentionbasedinteractivemultilevelfeaturefusionfornamedentityrecognition |