Rethinking round-trip translation for machine translation evaluation

Terry Yue Zhuo, Qiongkai Xu*, Xuanli He, Trevor Cohn

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

1 Citation (Scopus)

Abstract

Automatic evaluation methods for translation often require model training, and thus the availability of parallel corpora limits their applicability to low-resource settings. Round-trip translation is a potential workaround, which can reframe bilingual evaluation into a much simpler monolingual task. Early results from the era of statistical machine translation (SMT) raised fundamental concerns about the utility of this approach, based on poor correlation with human translation quality judgments. In this paper, we revisit this technique with modern neural translation (NMT) and show that round-trip translation does allow for accurate automatic evaluation without the need for reference translations. These opposite findings can be explained through the copy mechanism in SMT that is absent in NMT. We demonstrate that round-trip translation benefits multiple machine translation evaluation tasks: i) predicting forward translation scores; ii) improving the performance of a quality estimation model; and iii) identifying adversarial competitors in shared tasks via cross-system verification.
Original languageEnglish
Title of host publicationFindings of the Association for Computational Linguistics
Subtitle of host publicationACL 2023
EditorsAnna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Place of PublicationKerrville, TX
PublisherAssociation for Computational Linguistics
Pages319-337
Number of pages19
ISBN (Electronic)9781959429623
DOIs
Publication statusPublished - 2023
Externally publishedYes
EventAnnual Meeting of the Association for Computational Linguistics (61st : 2023) - Toronto, Canada
Duration: 9 Jul 202314 Jul 2023
Conference number: 61st
https://2023.aclweb.org/

Conference

ConferenceAnnual Meeting of the Association for Computational Linguistics (61st : 2023)
Abbreviated titleACL'23
Country/TerritoryCanada
CityToronto
Period9/07/2314/07/23
Internet address

Fingerprint

Dive into the research topics of 'Rethinking round-trip translation for machine translation evaluation'. Together they form a unique fingerprint.

Cite this