The impact of monolingual and bilingual subtitles on visual attention, cognitive load, and comprehension

With the proliferation and global dissemination of audiovisual products, subtitles have been widely used as a cost-effective tool to minimise language barriers for audiences of diverse cultural and linguistic backgrounds. However, the effectiveness of subtitles is still a topic of much debate and subject to various conditions, such as the context of use, the subtitle type, and the relationship between the language of the soundtrack and that of the subtitles. Drawing on an analysis of eye movements and a self-reported questionnaire, this study compares the impact of bilingual subtitles to that of monolingual subtitles in terms of viewers’ visual attention distribution, cognitive load, and overall comprehension of video content. Twenty Chinese (L1) native speakers watched four videos with English (L2) audio, each in a different condition: with Chinese subtitles (interlingual/L1 subtitles), with English subtitles (intralingual/L2 subtitles), with both Chinese and English subtitles (bilingual subtitles), and without subtitles. Our results indicate that viewers’ visual attention allocation to L1 subtitles was more stable than to L2 subtitles and less sensitive to the increased visual competition in the bilingual condition, which, we argue, can be attributed to the language dominance of their native language. Bilingual subtitles as a combination of intralingual and interlingual subtitles did not appear to induce more cognitive load or produce more cognitive gain than monolingual subtitles. Compared with the no subtitles condition, however, we found bilingual subtitles to be more beneficial as they provided linguistic support to make the video easier to comprehend and facilitate the learning process.