Abstract
| Language | English |
|---|---|
| Pages | 192-207 |
| Number of pages | 16 |
| Journal | Interacting with Computers |
| Volume | 31 |
| Issue number | 2 |
| Early online date | 13 May 2019 |
| DOIs | |
| Publication status | Published - 2019 |
Keywords
- Human computer interaction
- user experience
- Natural language interfaces
- Interaction design
- conversational agents
Cite this
}
Understanding and measuring user experience in conversational interfaces. / Kocaballi, Ahmet Baki; Laranjo, Liliana; Coiera, Enrico.
In: Interacting with Computers, Vol. 31, No. 2, 2019, p. 192-207.Research output: Contribution to journal › Article › Research › peer-review
TY - JOUR
T1 - Understanding and measuring user experience in conversational interfaces
AU - Kocaballi, Ahmet Baki
AU - Laranjo, Liliana
AU - Coiera, Enrico
PY - 2019
Y1 - 2019
N2 - Although various methods have been developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This paper reviews the understandings of user experience (UX) in conversational interfaces literature and examines the six questionnaires commonly used for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure different UX dimensions in that context. The method to examine the questionnaires involved developing an assessment framework for main UX dimensions with relevant attributes and coding the items in the questionnaires according to the framework. The results show that (i) the understandings of UX notably differed in literature; (ii) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (iii) while the dimension of affect was covered by two questionnaires, playfulness, motivation, and frustration dimensions were covered by one questionnaire only. The largest coverage of UX dimensions has been provided by the Subjective Assessment of Speech System Interfaces (SASSI). We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension.
AB - Although various methods have been developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This paper reviews the understandings of user experience (UX) in conversational interfaces literature and examines the six questionnaires commonly used for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure different UX dimensions in that context. The method to examine the questionnaires involved developing an assessment framework for main UX dimensions with relevant attributes and coding the items in the questionnaires according to the framework. The results show that (i) the understandings of UX notably differed in literature; (ii) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (iii) while the dimension of affect was covered by two questionnaires, playfulness, motivation, and frustration dimensions were covered by one questionnaire only. The largest coverage of UX dimensions has been provided by the Subjective Assessment of Speech System Interfaces (SASSI). We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension.
KW - Human computer interaction
KW - user experience
KW - Natural language interfaces
KW - Interaction design
KW - conversational agents
U2 - 10.1093/iwc/iwz015
DO - 10.1093/iwc/iwz015
M3 - Article
VL - 31
SP - 192
EP - 207
JO - Interacting with Computers
T2 - Interacting with Computers
JF - Interacting with Computers
SN - 0953-5438
IS - 2
ER -