TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora
In large language models (LLMs), the translation quality has limitations in the translation when translated into different languages. This study compares Chinese allusions in human and machine translated corpora translated by OpenAI GPT-3.5, Volctrans, and human translated texts. The framework innov...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2025-01-01
|
| Series: | Automatika |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/00051144.2024.2447652 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | In large language models (LLMs), the translation quality has limitations in the translation when translated into different languages. This study compares Chinese allusions in human and machine translated corpora translated by OpenAI GPT-3.5, Volctrans, and human translated texts. The framework innovatively combines two automated evaluation metrics, BLEU and METEOR, with a translation quality assessment method derived from Fuzzy Mathematics and Optimality Theory. The findings of the study indicate that the GPT-3.5 translated version exhibits higher quality than the Volctrans version when evaluated by a machine. Similarly, human evaluations indicate that among the three versions, Volctrans is of the lowest quality, while the human version exceeds the GPT-3.5 version in terms of quality. Thus, the study further reveals that Volctrans version is deemed to have the lowest translation quality from both human and machine evaluation perspectives. Finally, this study not only introduces but also validates a novel framework for assessing machine translation quality. |
|---|---|
| ISSN: | 0005-1144 1848-3380 |