Measuring the impact of subtitles on cognitive load: Eye tracking and dynamic audiovisual texts

In educational design literature, it is often taken as fact that subtitles increase cognitive load (CL). This paper investigates this assumption experimentally by comparing various measures of CL when students watch a recorded academic lecture with or without subtitles. Since the measurement of cognitive load is by no means a simple matter, we first provide an overview of the different measurement techniques based on causality and objectivity. We measure CL by means of eye tracking (pupil dilation), electroencephalography (EEG), self-reported ratings of mental effort, frustration, comprehension effort and engagement, as well as performance measures (comprehension test). Our findings seem to indicate that the subtitled condition in fact created lower CL in terms of percentage change in pupil diameter (PCPD) for the stimulus, approaching significance. In the subtitled condition PCPD also correlates significantly with participants' self-reported comprehension effort levels (their perception of how easy or difficult it was to understand the lecture). The EEG data, in turn, shows a significantly higher level of frustration for the unsubtitled condition. Negative emotional states could be caused by situations of higher CL (or cognitive overload) leading to learner frustration and dissatisfaction with learning activities and own performance [16]. It could therefore be reasoned that participants had a higher CL in the absence of subtitles. The self-reported frustration levels correlate with the frustration measured by the EEG as well as the self-reported engagement levels for the subtitled group. We also found a significant correlation between the self-reported engagement levels and both the short- and long term comprehension for the unsubtitled condition but not for the subtitled condition. There was no significant difference in either short-term or long-term performance measures between the two groups, which seems to suggest that subtitles at the very least, do not result in cognitive overload.