Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation
With the rapid development of machine translation (MT), the MT evaluation becomes very important to timely tell us whether the MT system makes any progress. The conventional MT evaluation methods tend to calculate the similarity between hypothesis translations offered by automatic translation system...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2014-01-01
|
| Series: | The Scientific World Journal |
| Online Access: | http://dx.doi.org/10.1155/2014/760301 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832555864110661632 |
|---|---|
| author | Aaron L.-F. Han Derek F. Wong Lidia S. Chao Liangye He Yi Lu |
| author_facet | Aaron L.-F. Han Derek F. Wong Lidia S. Chao Liangye He Yi Lu |
| author_sort | Aaron L.-F. Han |
| collection | DOAJ |
| description | With the rapid development of machine translation (MT), the MT evaluation becomes very important to timely tell us whether the MT system makes any progress. The conventional MT evaluation methods tend to calculate the similarity between hypothesis translations offered by automatic translation systems and reference translations offered by professional translators. There are several weaknesses in existing evaluation metrics. Firstly, the designed incomprehensive factors result in language-bias problem, which means they perform well on some special language pairs but weak on other language pairs. Secondly, they tend to use no linguistic features or too many linguistic features, of which no usage of linguistic feature draws a lot of criticism from the linguists and too many linguistic features make the model weak in repeatability. Thirdly, the employed reference translations are very expensive and sometimes not available in the practice. In this paper, the authors propose an unsupervised MT evaluation metric using universal part-of-speech tagset without relying on reference translations. The authors also explore the performances of the designed metric on traditional supervised evaluation tasks. Both the supervised and unsupervised experiments show that the designed methods yield higher correlation scores with human judgments. |
| format | Article |
| id | doaj-art-6ddd4e7914b343309cb6e41c3895fad6 |
| institution | Kabale University |
| issn | 2356-6140 1537-744X |
| language | English |
| publishDate | 2014-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | The Scientific World Journal |
| spelling | doaj-art-6ddd4e7914b343309cb6e41c3895fad62025-02-03T05:46:54ZengWileyThe Scientific World Journal2356-61401537-744X2014-01-01201410.1155/2014/760301760301Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised EvaluationAaron L.-F. Han0Derek F. Wong1Lidia S. Chao2Liangye He3Yi Lu4Natural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information Science, University of Macau, MacauNatural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information Science, University of Macau, MacauNatural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information Science, University of Macau, MacauNatural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information Science, University of Macau, MacauNatural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information Science, University of Macau, MacauWith the rapid development of machine translation (MT), the MT evaluation becomes very important to timely tell us whether the MT system makes any progress. The conventional MT evaluation methods tend to calculate the similarity between hypothesis translations offered by automatic translation systems and reference translations offered by professional translators. There are several weaknesses in existing evaluation metrics. Firstly, the designed incomprehensive factors result in language-bias problem, which means they perform well on some special language pairs but weak on other language pairs. Secondly, they tend to use no linguistic features or too many linguistic features, of which no usage of linguistic feature draws a lot of criticism from the linguists and too many linguistic features make the model weak in repeatability. Thirdly, the employed reference translations are very expensive and sometimes not available in the practice. In this paper, the authors propose an unsupervised MT evaluation metric using universal part-of-speech tagset without relying on reference translations. The authors also explore the performances of the designed metric on traditional supervised evaluation tasks. Both the supervised and unsupervised experiments show that the designed methods yield higher correlation scores with human judgments.http://dx.doi.org/10.1155/2014/760301 |
| spellingShingle | Aaron L.-F. Han Derek F. Wong Lidia S. Chao Liangye He Yi Lu Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation The Scientific World Journal |
| title | Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation |
| title_full | Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation |
| title_fullStr | Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation |
| title_full_unstemmed | Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation |
| title_short | Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation |
| title_sort | unsupervised quality estimation model for english to german translation and its application in extensive supervised evaluation |
| url | http://dx.doi.org/10.1155/2014/760301 |
| work_keys_str_mv | AT aaronlfhan unsupervisedqualityestimationmodelforenglishtogermantranslationanditsapplicationinextensivesupervisedevaluation AT derekfwong unsupervisedqualityestimationmodelforenglishtogermantranslationanditsapplicationinextensivesupervisedevaluation AT lidiaschao unsupervisedqualityestimationmodelforenglishtogermantranslationanditsapplicationinextensivesupervisedevaluation AT liangyehe unsupervisedqualityestimationmodelforenglishtogermantranslationanditsapplicationinextensivesupervisedevaluation AT yilu unsupervisedqualityestimationmodelforenglishtogermantranslationanditsapplicationinextensivesupervisedevaluation |