TY - GEN
T1 - What are we measuring anyway?
T2 - 19th ACM International Conference on Intelligent Virtual Agents, IVA 2019
AU - Fitrianie, Siska
AU - Bruijnes, Merijn
AU - Richards, Deborah
AU - Abdulrahman, Amal
AU - Brinkman, Willem-Paul
PY - 2019/7/1
Y1 - 2019/7/1
N2 - Research into artificial social agents aims at constructing these agents and at establishing an empirically grounded understanding of them, their interaction with humans, and howthey can ultimately deliver certain outcomes in areas such as health, entertainment, and education. Key for establishing such understanding is the community's ability to describe and replicate their observations on how users perceive and interact with their agents. In this paper, we address this ability by examining questionnaires and their constructs used in empirical studies reported in the intelligent virtual agent conference proceedings from 2013 to 2018. The literature survey shows the identification of 189 constructs used in 89 questionnaires thatwere reported across 81 papers.We found unexpectedly little repeated use of questionnaires as the vast majority of questionnaires (more than 76%) were only reported in a single paper. We expect that this finding will motivate joint effort by the IVA community towards creating a unified measurement instrument.
AB - Research into artificial social agents aims at constructing these agents and at establishing an empirically grounded understanding of them, their interaction with humans, and howthey can ultimately deliver certain outcomes in areas such as health, entertainment, and education. Key for establishing such understanding is the community's ability to describe and replicate their observations on how users perceive and interact with their agents. In this paper, we address this ability by examining questionnaires and their constructs used in empirical studies reported in the intelligent virtual agent conference proceedings from 2013 to 2018. The literature survey shows the identification of 189 constructs used in 89 questionnaires thatwere reported across 81 papers.We found unexpectedly little repeated use of questionnaires as the vast majority of questionnaires (more than 76%) were only reported in a single paper. We expect that this finding will motivate joint effort by the IVA community towards creating a unified measurement instrument.
KW - artificial social agent
KW - evaluation instrument
KW - measurement construct
KW - questionnaire
KW - user study
UR - http://www.scopus.com/inward/record.url?scp=85069726771&partnerID=8YFLogxK
U2 - 10.1145/3308532.3329421
DO - 10.1145/3308532.3329421
M3 - Conference proceeding contribution
T3 - IVA 2019 - Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents
SP - 159
EP - 161
BT - IVA '19 Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents
PB - Association for Computing Machinery (ACM)
CY - New York, NY, USA
Y2 - 2 July 2019 through 5 July 2019
ER -