Measuring Catastrophic Forgetting in Cross-Lingual Classification: Transfer Paradigms and Tuning Strategies
Cross-lingual transfer leverages knowledge from a resource-rich source language, commonly English, to enhance performance in less-resourced target languages. Two widely used strategies are: Cross-Lingual Validation (CLV), which involves training on the source language and validating on the target la...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10892119/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Cross-lingual transfer leverages knowledge from a resource-rich source language, commonly English, to enhance performance in less-resourced target languages. Two widely used strategies are: Cross-Lingual Validation (CLV), which involves training on the source language and validating on the target language, and Intermediate Training (IT), where models are first fine-tuned on the source language and then further trained on the target language. While both strategies have been studied, their effects on encoder-based models for classification tasks remain underexplored. In this paper, we systematically compare these strategies across six multilingual classification tasks, evaluating downstream performance, catastrophic forgetting, and both zero-shot and full-shot scenarios. Additionally, we contrast parameter-efficient adapter methods with full-parameter fine-tuning. Our results show that IT generally performs better in the target language, whereas CLV more effectively preserves source-language knowledge across multiple cross-lingual transfers. These findings underscore the trade-offs between optimizing target performance and mitigating catastrophic forgetting. |
|---|---|
| ISSN: | 2169-3536 |