Sense and Sensitivity: Exploring how Neural Machine Translation Systems Handle Slurs

Abstract

The rise of streaming platforms such as Netflix and HBO has brought a surge in audiovisual content to be translated. While the translation industry at large have adopted machine translation (MT) as a tool to meet the rising demands, the subtitling industry has been reluctant to embrace this trend. One possible reason is that MT tends to perform badly on the kind of content found in audiovisual material, such as sensitive and offensive language. Meanwhile, research in the intersection of MT and human translation is sparse, and the differing approaches and terminology used within the two fields seem to impede interdisciplinary collaborations. In this work I explore how approaches borrowed from descriptive translation studies (DTS) can be adapted for analysis of MT output. I apply a taxonomy inspired by those used for analysis of human translation to output from three different MT systems in order to investigate how they translate slurs from Swedish to English. In contrast to conventional approaches to MT evaluation which focus on overall quality, the approach used here aims to descriptively capture the ways a translation can relate to its source text on a lexical-semantic level. The results provide some preliminary insights into the kinds of semantic divergence that can be introduced in the translation process when using MT to translate slurs. These can be seen as pointers to areas for future investigations. For example, some instances of gender bias were identified, where systems tended to translate terms denoting individuals of any gender into terms specifically denoting men. While further revision may be needed for the taxonomy to better fit the material to be analysed, this initial investigation suggests that DTS approaches could be well suited as the basis for more fine-grained analyses of MT output.

NOTE: This thesis deals with sensitive and offensive language and includes, by necessity, several instances of such language as examples.