Do not freak me out! The impact of lip movement and appearance on knowledge gain and confidence

Amal Abdulrahman, Katherine Hopman, Deborah Richards*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
44 Downloads (Pure)

Abstract

Virtual agents (VAs) have been used effectively for psychoeducation. However, getting the VA’s design right is critical to ensure the user experience does not become a barrier to receiving and responding to the intended message. The study reported in this paper seeks to help first-year psychology students to develop knowledge and confidence to recommend emotion regulation strategies. In previous work, we received negative feedback concerning the VA’s lip-syncing, including creepiness and visual overload, in the case of stroke patients. We seek to test the impact of the removal of lip-syncing on the perception of the VA and its ability to achieve its intended outcomes, also considering the influence of the visual features of the avatar. We conducted a 2 (lip-sync/no lip-sync) × 2 (human-like/cartoon-like) experimental design and measured participants’ perception of the VA in terms of eeriness, user experience, knowledge gain and participants’ confidence to practice their knowledge. While participants showed a tendency to prefer the cartoon look over the human look and the absence of lip-syncing over its presence, all groups reported no significant increase in knowledge but significant increases in confidence in their knowledge and ability to recommend the learnt strategies to others, concluding that realism and lip-syncing did not influence the intended outcomes. Thus, in future designs, we will allow the user to switch off the lip-sync function if they prefer. Further, our findings suggest that lip-syncing should not be a standard animation included with VAs, as is currently the case.

Original languageEnglish
Article number22
Pages (from-to)1-15
Number of pages15
JournalMultimodal Technologies and Interaction
Volume8
Issue number3
DOIs
Publication statusPublished - Mar 2024

Bibliographical note

Copyright the Author(s) 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Keywords

  • virtual coach
  • lip-sync
  • realism
  • behaviour change

Fingerprint

Dive into the research topics of 'Do not freak me out! The impact of lip movement and appearance on knowledge gain and confidence'. Together they form a unique fingerprint.

Cite this