Investigating neural machine translation quality: An xCOMET-XL-based analysis across Katharina Reiss's text typologies
Investigating neural machine translation quality: An xCOMET-XL-based analysis across Katharina Reiss's text typologies

Summary/Abstract: This study examines AI-driven neural methods that reduce human involvement in machine translation evaluation. Traditional evaluation approaches face challenges related to consistency, scalability, and evaluator subjectivity. To address these limitations, the study employs xCOMET-XL, which evaluates translation quality within a shared semantic space. Within the scope of this study, fifteen Turkish source texts, classified according to Katharina Reiss's typology as informative, expressive, and operative, were translated into English using Google Translate and DeepL. The translations were evaluated using xCOMET-XL, followed by human evaluation. The originality of the study lies in its comparative design examining the alignment between xCOMET-XL scores and human judgments. After comparing results, alignment between the methods was analysed. Findings signal a relatively high degree of alignment with human judgments in certain texts, highlighting neural metrics' potential as a fast, scalable, and systematic alternative. The study contributes to Translation Studies by supporting alternative translation evaluation approaches.

• Details
• Contents