TY - GEN
T1 - The 19 unifying questionnaire constructs of artificial social agents
T2 - 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
AU - Fitrianie, Siska
AU - Bruijnes, Merijn
AU - Richards, Deborah
AU - Bönsch, Andrea
AU - Brinkman, Willem-Paul
PY - 2020/10
Y1 - 2020/10
N2 - In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 80 researchers worldwide, researching the IVA community interests and practises in evaluating human interaction with an artificial social agent (ASA). The effort is driven by previous IVA workshops and plenary IVA discussions related to the methodological crisis on the evaluation of ASAs. A previous literature review showed a continuous practise of creating new questionnaires instead of reusing validated questionnaires. We address this issue by examining questionnaire measurement constructs used in empirical studies between 2013 to 2018 published in the IVA conference. We identified 189 constructs used in 89 questionnaires that are reported across 81 studies. Although these constructs have different names, they often measure the same thing. In this paper, we, therefore, present a unifying set of 19 constructs that captures more than 80% of the 189 constructs initially identified. We established this set in two steps. First, 49 researchers classified the constructs in broad theoretically based categories. Next, 23 researchers grouped the constructs in each category on their similarity. The resulting 19 groups form a unifying set of constructs, which will be the basis for the future questionnaire instrument of human-ASA interaction.
AB - In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 80 researchers worldwide, researching the IVA community interests and practises in evaluating human interaction with an artificial social agent (ASA). The effort is driven by previous IVA workshops and plenary IVA discussions related to the methodological crisis on the evaluation of ASAs. A previous literature review showed a continuous practise of creating new questionnaires instead of reusing validated questionnaires. We address this issue by examining questionnaire measurement constructs used in empirical studies between 2013 to 2018 published in the IVA conference. We identified 189 constructs used in 89 questionnaires that are reported across 81 studies. Although these constructs have different names, they often measure the same thing. In this paper, we, therefore, present a unifying set of 19 constructs that captures more than 80% of the 189 constructs initially identified. We established this set in two steps. First, 49 researchers classified the constructs in broad theoretically based categories. Next, 23 researchers grouped the constructs in each category on their similarity. The resulting 19 groups form a unifying set of constructs, which will be the basis for the future questionnaire instrument of human-ASA interaction.
KW - Artificial social agent
KW - evaluation instrument
KW - user study
KW - questionnaire
KW - measurement construct
UR - http://www.scopus.com/inward/record.url?scp=85096969234&partnerID=8YFLogxK
U2 - 10.1145/3383652.3423873
DO - 10.1145/3383652.3423873
M3 - Conference proceeding contribution
AN - SCOPUS:85096969234
SN - 9781450375863
T3 - Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
SP - 1
EP - 8
BT - Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
PB - Association for Computing Machinery, Inc
CY - New York, NY
Y2 - 20 October 2020 through 22 October 2020
ER -