Measuring the impact of subtitles on cognitive load: Eye tracking and dynamic audiovisual texts

Jan Louis Kruger, Esté Hefer, Gordon Matthew

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

78 Citations (Scopus)


In educational design literature, it is often taken as fact that subtitles increase cognitive load (CL). This paper investigates this assumption experimentally by comparing various measures of CL when students watch a recorded academic lecture with or without subtitles. Since the measurement of cognitive load is by no means a simple matter, we first provide an overview of the different measurement techniques based on causality and objectivity. We measure CL by means of eye tracking (pupil dilation), electroencephalography (EEG), self-reported ratings of mental effort, frustration, comprehension effort and engagement, as well as performance measures (comprehension test). Our findings seem to indicate that the subtitled condition in fact created lower CL in terms of percentage change in pupil diameter (PCPD) for the stimulus, approaching significance. In the subtitled condition PCPD also correlates significantly with participants' self-reported comprehension effort levels (their perception of how easy or difficult it was to understand the lecture). The EEG data, in turn, shows a significantly higher level of frustration for the unsubtitled condition. Negative emotional states could be caused by situations of higher CL (or cognitive overload) leading to learner frustration and dissatisfaction with learning activities and own performance [16]. It could therefore be reasoned that participants had a higher CL in the absence of subtitles. The self-reported frustration levels correlate with the frustration measured by the EEG as well as the self-reported engagement levels for the subtitled group. We also found a significant correlation between the self-reported engagement levels and both the short- and long term comprehension for the unsubtitled condition but not for the subtitled condition. There was no significant difference in either short-term or long-term performance measures between the two groups, which seems to suggest that subtitles at the very least, do not result in cognitive overload.

Original languageEnglish
Title of host publicationProceedings of the 2013 Conference on Eye Tracking South Africa, ETSA 2013
Place of PublicationNew York
Number of pages5
ISBN (Print)9781450321105
Publication statusPublished - 2013
Externally publishedYes
Event2013 Conference on Eye Tracking South Africa, ETSA 2013 - Cape Town, South Africa
Duration: 29 Aug 201331 Aug 2013


Other2013 Conference on Eye Tracking South Africa, ETSA 2013
Country/TerritorySouth Africa
CityCape Town


Dive into the research topics of 'Measuring the impact of subtitles on cognitive load: Eye tracking and dynamic audiovisual texts'. Together they form a unique fingerprint.

Cite this