TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora
In large language models (LLMs), the translation quality has limitations in the translation when translated into different languages. This study compares Chinese allusions in human and machine translated corpora translated by OpenAI GPT-3.5, Volctrans, and human translated texts. The framework innov...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2025-01-01
|
| Series: | Automatika |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/00051144.2024.2447652 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832582074618347520 |
|---|---|
| author | Li Yating Muhammad Afzaal Xiao Shanshan Dina Abdel Salam El-Dakhs |
| author_facet | Li Yating Muhammad Afzaal Xiao Shanshan Dina Abdel Salam El-Dakhs |
| author_sort | Li Yating |
| collection | DOAJ |
| description | In large language models (LLMs), the translation quality has limitations in the translation when translated into different languages. This study compares Chinese allusions in human and machine translated corpora translated by OpenAI GPT-3.5, Volctrans, and human translated texts. The framework innovatively combines two automated evaluation metrics, BLEU and METEOR, with a translation quality assessment method derived from Fuzzy Mathematics and Optimality Theory. The findings of the study indicate that the GPT-3.5 translated version exhibits higher quality than the Volctrans version when evaluated by a machine. Similarly, human evaluations indicate that among the three versions, Volctrans is of the lowest quality, while the human version exceeds the GPT-3.5 version in terms of quality. Thus, the study further reveals that Volctrans version is deemed to have the lowest translation quality from both human and machine evaluation perspectives. Finally, this study not only introduces but also validates a novel framework for assessing machine translation quality. |
| format | Article |
| id | doaj-art-0cd8d1e2d6cd4ccbad154c4925d4cc7a |
| institution | Kabale University |
| issn | 0005-1144 1848-3380 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Automatika |
| spelling | doaj-art-0cd8d1e2d6cd4ccbad154c4925d4cc7a2025-01-30T05:18:09ZengTaylor & Francis GroupAutomatika0005-11441848-33802025-01-016619110210.1080/00051144.2024.2447652TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corporaLi Yating0Muhammad Afzaal1Xiao Shanshan2Dina Abdel Salam El-Dakhs3School of English Studies, Shanghai International Studies University, Shanghai, People’s Republic of ChinaInstitute of Language Sciences, Shanghai International Studies University, Shanghai, People’s Republic of ChinaSchool of English Studies, Shanghai International Studies University, Shanghai, People’s Republic of ChinaCollege of Humanities, Prince Sultan University, Riyadh, Saudi ArabiaIn large language models (LLMs), the translation quality has limitations in the translation when translated into different languages. This study compares Chinese allusions in human and machine translated corpora translated by OpenAI GPT-3.5, Volctrans, and human translated texts. The framework innovatively combines two automated evaluation metrics, BLEU and METEOR, with a translation quality assessment method derived from Fuzzy Mathematics and Optimality Theory. The findings of the study indicate that the GPT-3.5 translated version exhibits higher quality than the Volctrans version when evaluated by a machine. Similarly, human evaluations indicate that among the three versions, Volctrans is of the lowest quality, while the human version exceeds the GPT-3.5 version in terms of quality. Thus, the study further reveals that Volctrans version is deemed to have the lowest translation quality from both human and machine evaluation perspectives. Finally, this study not only introduces but also validates a novel framework for assessing machine translation quality.https://www.tandfonline.com/doi/10.1080/00051144.2024.2447652LLMsmultilingual corporatranslation quality assessmentmachine translationallusions |
| spellingShingle | Li Yating Muhammad Afzaal Xiao Shanshan Dina Abdel Salam El-Dakhs TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora Automatika LLMs multilingual corpora translation quality assessment machine translation allusions |
| title | TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora |
| title_full | TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora |
| title_fullStr | TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora |
| title_full_unstemmed | TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora |
| title_short | TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora |
| title_sort | tqfll a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora |
| topic | LLMs multilingual corpora translation quality assessment machine translation allusions |
| url | https://www.tandfonline.com/doi/10.1080/00051144.2024.2447652 |
| work_keys_str_mv | AT liyating tqfllanovelunifiedanalyticsframeworkfortranslationqualityframeworkforlargelanguagemodelandhumantranslationofallusionsinmultilingualcorpora AT muhammadafzaal tqfllanovelunifiedanalyticsframeworkfortranslationqualityframeworkforlargelanguagemodelandhumantranslationofallusionsinmultilingualcorpora AT xiaoshanshan tqfllanovelunifiedanalyticsframeworkfortranslationqualityframeworkforlargelanguagemodelandhumantranslationofallusionsinmultilingualcorpora AT dinaabdelsalameldakhs tqfllanovelunifiedanalyticsframeworkfortranslationqualityframeworkforlargelanguagemodelandhumantranslationofallusionsinmultilingualcorpora |