The ethical acceptability of artificial social agents

Artificial social agents (ASAs) are computer-based autonomous entities who interact with humans in a range of social roles, including advising, coaching, and consumer support in education and health. While there are many discussions around the ethical use of Artificial Intelligence in general, such as the AI4People Ethical Framework, the ethical ramifications that ASAs will have on human relationships as we share not only personal data but our inner most thoughts and feelings, is less explored. We conducted a study with 199 student participants exposed to Sam, an ASA which acts as a personal guide to support the student during their studies. During the interaction, Sam elicits private information, takes decisions for the user and speaks of its own study experiences. Our results indicate that (loss of) autonomy raised the strongest ethical concerns. These results confirm the importance of informed consent, transparency and accountability of ASAs and question the ethics of false memories and emotion sharing.