An Experimental Study on Dynamic Lifelong Learning With GPT for Mitigating Catastrophic Forgetting in Aspect-Based Sentiment Analysis
GPT is widely recognized as one of the most versatile and powerful large language models, excelling across diverse domains. However, its significant computational demands often render it economically unfeasible for individuals and small businesses, underscoring the need for efficient, domain-specifi...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10909082/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | GPT is widely recognized as one of the most versatile and powerful large language models, excelling across diverse domains. However, its significant computational demands often render it economically unfeasible for individuals and small businesses, underscoring the need for efficient, domain-specific alternatives. In Aspect-Based Sentiment Analysis (ABSA), existing models are typically optimized for single domains, facing challenges in performing effectively across multiple domains. A key issue, known as “catastrophic forgetting,” arises when models trained on one domain lose previously learned knowledge upon exposure to new domain data. This leads to two significant problems: limited cross-domain generalization and difficulty in retaining prior knowledge while learning domain-specific information. To address these challenges, we introduce the Dynamic Lifelong Learning Aspect-Based Sentiment Analysis GPT (DllaGPT), a model designed to handle multiple domains while mitigating catastrophic forgetting. Leveraging datasets from four ABSA domains—Laptops, Restaurants, Tweets, and Finance—this study fine-tunes a pretrained GPT model from HuggingFace sequentially across domains. DllaGPT employs a mechanism to retain real data from earlier domains during new domain training, effectively preserving prior knowledge. Experimental results highlight that DllaGPT achieves an average accuracy of 0.85 and a Backward Transfer (BWT) score of -0.09 across the four domains, showcasing its high accuracy and robust lifelong learning capabilities. |
|---|---|
| ISSN: | 2169-3536 |