Understanding and measuring user experience in conversational interfaces

Research output: Contribution to journalArticlepeer-review

43 Citations (Scopus)


Although various methods have been developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This paper reviews the understandings of user experience (UX) in conversational interfaces literature and examines the six questionnaires commonly used for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure different UX dimensions in that context. The method to examine the questionnaires involved developing an assessment framework for main UX dimensions with relevant attributes and coding the items in the questionnaires according to the framework. The results show that (i) the understandings of UX notably differed in literature; (ii) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (iii) while the dimension of affect was covered by two questionnaires, playfulness, motivation, and frustration dimensions were covered by one questionnaire only. The largest coverage of UX dimensions has been provided by the Subjective Assessment of Speech System Interfaces (SASSI). We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension.
Original languageEnglish
Pages (from-to)192-207
Number of pages16
JournalInteracting with Computers
Issue number2
Publication statusPublished - Mar 2019


  • Human computer interaction
  • user experience
  • Natural language interfaces
  • Interaction design
  • conversational agents


Dive into the research topics of 'Understanding and measuring user experience in conversational interfaces'. Together they form a unique fingerprint.

Cite this