Measuring user experience in conversational interfaces: a comparison of six questionnaires

A. Baki Kocaballi, Liliana Laranjo da Silva, Enrico Coiera

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

56 Citations (Scopus)

Abstract

User experience (UX) has become an important aspect in the evaluation of interactive systems. In parallel, conversational interfaces have been increasingly used in many work and everyday settings. Although there have been various methods developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This study reviews the six main questionnaires for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure various UX dimensions. We found that (i) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (ii) two questionnaires assessed affect, and one assessed frustration dimension; and, (iii) enchantment, playfulness and motivation dimensions have not been covered sufficiently by any questionnaires. We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension.
Original languageEnglish
Title of host publicationProceedings of 32nd BCS Human Computer Interaction Conference (HCI 2018)
EditorsRaymond Bond, Maurice Mulvenna, Jonathan Wallace, Michaela Black
Place of PublicationBelfast
PublisherBCS, The Chartered Institute for IT
Pages1-12
Number of pages12
DOIs
Publication statusPublished - Nov 2018
Event32nd International BCS Human Computer Interaction Conference (HCI 2018) - Belfast, Ireland
Duration: 4 Jul 20186 Jul 2018

Conference

Conference32nd International BCS Human Computer Interaction Conference (HCI 2018)
Country/TerritoryIreland
CityBelfast
Period4/07/186/07/18

Keywords

  • User Experience
  • conversational agents
  • dialogue systems
  • evaluation
  • voice interfaces
  • standardised questionnaires

Fingerprint

Dive into the research topics of 'Measuring user experience in conversational interfaces: a comparison of six questionnaires'. Together they form a unique fingerprint.

Cite this