Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT
Abstract Sentiment analysis (SA) as a research field has gained popularity among the researcher throughout the globe over the past 10 years. Deep neural networks (DNN) and word vector models are employed nowadays and perform well in sentiment analysis. Among the different deep neural networks utiliz...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01631-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571151658778624 |
|---|---|
| author | Lal Khan Atika Qazi Hsien-Tsung Chang Mousa Alhajlah Awais Mahmood |
| author_facet | Lal Khan Atika Qazi Hsien-Tsung Chang Mousa Alhajlah Awais Mahmood |
| author_sort | Lal Khan |
| collection | DOAJ |
| description | Abstract Sentiment analysis (SA) as a research field has gained popularity among the researcher throughout the globe over the past 10 years. Deep neural networks (DNN) and word vector models are employed nowadays and perform well in sentiment analysis. Among the different deep neural networks utilized for SA globally, Bi-directional long short-term memory (Bi-LSTM), BERT, and CNN models have received much attention. Even though these models can process a wide range of text types, Because DNNs treat different features the same, using these models in the feature learning phase of a DNN model leads to the creation of a feature space with very high dimensionality. We suggest an attention-based, stacked, two-layer CNN-Bi-LSTM DNN to overcome these glitches. After local feature extraction, by applying stacked two-layer Bi-LSTM, our proposed model extracted coming and outgoing sequences by seeing sequential data streams in backward and forward directions. The output of the stacked two-layer Bi-LSTM is supplied to the attention layer to assign various words with varying values. A second Bi-LSTM layer is constructed atop the initial layer in the suggested network to increase performance. Various experiments have been conducted to evaluate the effectiveness of our proposed model on two Urdu sentiment analysis datasets named as UCSA-21 and UCSA, and an accuracies of 83.12% and 78.91% achieved, respectively. |
| format | Article |
| id | doaj-art-27cd5430a23749a4b2249c2e73231f7d |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-27cd5430a23749a4b2249c2e73231f7d2025-02-02T12:49:01ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111410.1007/s40747-024-01631-9Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERTLal Khan0Atika Qazi1Hsien-Tsung Chang2Mousa Alhajlah3Awais Mahmood4Department of Computer Science, IBADAT International University IslamabadCentre for Lifelong Learning, Universiti Brunei DarussalamBachelor Program in Artificial Intelligence, Chang Gung UniversityComputer Science and Information Systems Department, Applied Computer Science College, King Saud UniversityComputer Science and Information Systems Department, Applied Computer Science College, King Saud UniversityAbstract Sentiment analysis (SA) as a research field has gained popularity among the researcher throughout the globe over the past 10 years. Deep neural networks (DNN) and word vector models are employed nowadays and perform well in sentiment analysis. Among the different deep neural networks utilized for SA globally, Bi-directional long short-term memory (Bi-LSTM), BERT, and CNN models have received much attention. Even though these models can process a wide range of text types, Because DNNs treat different features the same, using these models in the feature learning phase of a DNN model leads to the creation of a feature space with very high dimensionality. We suggest an attention-based, stacked, two-layer CNN-Bi-LSTM DNN to overcome these glitches. After local feature extraction, by applying stacked two-layer Bi-LSTM, our proposed model extracted coming and outgoing sequences by seeing sequential data streams in backward and forward directions. The output of the stacked two-layer Bi-LSTM is supplied to the attention layer to assign various words with varying values. A second Bi-LSTM layer is constructed atop the initial layer in the suggested network to increase performance. Various experiments have been conducted to evaluate the effectiveness of our proposed model on two Urdu sentiment analysis datasets named as UCSA-21 and UCSA, and an accuracies of 83.12% and 78.91% achieved, respectively.https://doi.org/10.1007/s40747-024-01631-9Sentiment analysisWord embeddingMachine learningDeep learningBi-LSTMCNN |
| spellingShingle | Lal Khan Atika Qazi Hsien-Tsung Chang Mousa Alhajlah Awais Mahmood Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT Complex & Intelligent Systems Sentiment analysis Word embedding Machine learning Deep learning Bi-LSTM CNN |
| title | Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT |
| title_full | Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT |
| title_fullStr | Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT |
| title_full_unstemmed | Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT |
| title_short | Empowering Urdu sentiment analysis: an attention-based stacked CNN-Bi-LSTM DNN with multilingual BERT |
| title_sort | empowering urdu sentiment analysis an attention based stacked cnn bi lstm dnn with multilingual bert |
| topic | Sentiment analysis Word embedding Machine learning Deep learning Bi-LSTM CNN |
| url | https://doi.org/10.1007/s40747-024-01631-9 |
| work_keys_str_mv | AT lalkhan empoweringurdusentimentanalysisanattentionbasedstackedcnnbilstmdnnwithmultilingualbert AT atikaqazi empoweringurdusentimentanalysisanattentionbasedstackedcnnbilstmdnnwithmultilingualbert AT hsientsungchang empoweringurdusentimentanalysisanattentionbasedstackedcnnbilstmdnnwithmultilingualbert AT mousaalhajlah empoweringurdusentimentanalysisanattentionbasedstackedcnnbilstmdnnwithmultilingualbert AT awaismahmood empoweringurdusentimentanalysisanattentionbasedstackedcnnbilstmdnnwithmultilingualbert |