Towards a cognitive audiovisual translatology: subtitles and embodied cognition

This chapter presents the findings of a study to investigate the impact of subtitles on the processing of audiovisual texts in terms of levels of self-reported engagement with the text. It also presents a methodology for investigating the neural processing of subtitles using electroencephalography (EEG) and psychometrics. By establishing the beta coherence between prefrontal and posterior regions, we can determine whether the effect of subtitles can be captured through EEG, and how this is related to psychological immersion. We therefore use EEG to investigate the cognitive processing of subtitled film in combination with subjective psychometrics. Initial validation of this methodology shows that adding same language subtitles results in statistically significantly higher levels of immersion and enjoyment, lending support to a view that subtitles facilitate (dis)embodied cognition. We propose these as the first steps towards a cognitive audiovisual translatology.