Barriers and enabling factors for error analysis in NLG research
Earlier research has shown that few studies in Natural Language Generation (NLG) evaluate their system outputs using an error analysis, despite known limitations of automatic evaluation metrics and human ratings. This position paper takes the stance that error analyses should be encouraged, and dis...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Linköping University Electronic Press
2023-02-01
|
| Series: | Northern European Journal of Language Technology |
| Online Access: | https://nejlt.ep.liu.se/article/view/4529 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591228904931328 |
|---|---|
| author | Emiel van Miltenburg Miruna Clinciu Ondřej Dušek Dimitra Gkatzia Stephanie Inglis Leo Leppänen Saad Mahamood Stephanie Schoch Craig Thomson Luou Wen |
| author_facet | Emiel van Miltenburg Miruna Clinciu Ondřej Dušek Dimitra Gkatzia Stephanie Inglis Leo Leppänen Saad Mahamood Stephanie Schoch Craig Thomson Luou Wen |
| author_sort | Emiel van Miltenburg |
| collection | DOAJ |
| description |
Earlier research has shown that few studies in Natural Language Generation (NLG) evaluate their system outputs using an error analysis, despite known limitations of automatic evaluation metrics and human ratings. This position paper takes the stance that error analyses should be encouraged, and discusses several ways to do so. This paper is based on our shared experience as authors as well as a survey we distributed as a means of public consultation. We provide an overview of existing barriers to carrying out error analyses, and propose changes to improve error reporting in the NLG literature.
|
| format | Article |
| id | doaj-art-4dfdf66418db42f18d4ff01d7f884db3 |
| institution | Kabale University |
| issn | 2000-1533 |
| language | English |
| publishDate | 2023-02-01 |
| publisher | Linköping University Electronic Press |
| record_format | Article |
| series | Northern European Journal of Language Technology |
| spelling | doaj-art-4dfdf66418db42f18d4ff01d7f884db32025-01-22T15:25:16ZengLinköping University Electronic PressNorthern European Journal of Language Technology2000-15332023-02-019110.3384/nejlt.2000-1533.2023.4529Barriers and enabling factors for error analysis in NLG researchEmiel van Miltenburg0Miruna Clinciu1Ondřej Dušek2Dimitra Gkatzia3Stephanie Inglis4Leo Leppänen5Saad Mahamood6Stephanie Schoch7Craig Thomson8Luou Wen9Tilburg UniversityHeriot-Watt University and the University of EdinburghCharles University, PragueEdinburgh~Napier~University, UKArria NLG / University of AberdeenUniversity of Helsinkitrivago N.V.University of Virginia, USAUniversity of AberdeenIndependent researcher Earlier research has shown that few studies in Natural Language Generation (NLG) evaluate their system outputs using an error analysis, despite known limitations of automatic evaluation metrics and human ratings. This position paper takes the stance that error analyses should be encouraged, and discusses several ways to do so. This paper is based on our shared experience as authors as well as a survey we distributed as a means of public consultation. We provide an overview of existing barriers to carrying out error analyses, and propose changes to improve error reporting in the NLG literature. https://nejlt.ep.liu.se/article/view/4529 |
| spellingShingle | Emiel van Miltenburg Miruna Clinciu Ondřej Dušek Dimitra Gkatzia Stephanie Inglis Leo Leppänen Saad Mahamood Stephanie Schoch Craig Thomson Luou Wen Barriers and enabling factors for error analysis in NLG research Northern European Journal of Language Technology |
| title | Barriers and enabling factors for error analysis in NLG research |
| title_full | Barriers and enabling factors for error analysis in NLG research |
| title_fullStr | Barriers and enabling factors for error analysis in NLG research |
| title_full_unstemmed | Barriers and enabling factors for error analysis in NLG research |
| title_short | Barriers and enabling factors for error analysis in NLG research |
| title_sort | barriers and enabling factors for error analysis in nlg research |
| url | https://nejlt.ep.liu.se/article/view/4529 |
| work_keys_str_mv | AT emielvanmiltenburg barriersandenablingfactorsforerroranalysisinnlgresearch AT mirunaclinciu barriersandenablingfactorsforerroranalysisinnlgresearch AT ondrejdusek barriersandenablingfactorsforerroranalysisinnlgresearch AT dimitragkatzia barriersandenablingfactorsforerroranalysisinnlgresearch AT stephanieinglis barriersandenablingfactorsforerroranalysisinnlgresearch AT leoleppanen barriersandenablingfactorsforerroranalysisinnlgresearch AT saadmahamood barriersandenablingfactorsforerroranalysisinnlgresearch AT stephanieschoch barriersandenablingfactorsforerroranalysisinnlgresearch AT craigthomson barriersandenablingfactorsforerroranalysisinnlgresearch AT luouwen barriersandenablingfactorsforerroranalysisinnlgresearch |