To be or not to be: respecting when Socially Interactive Agents should or should not conform to expectations

Socially Interactive Agents (SIAs) seek to engage humans at a socio-emotional level to play roles typically played by other humans. In this paper, we raise the ethical concerns of SIA dependency (emotional attachment), deception (of humanness) and divulgation (of sensitive disclosures) and discrimination (unequal access or fit), where the latter may make some individuals more vulnerable to the other harms. Further, we consider how some roles, such as a coach, may require the SIA to meet the user's expectations, while other roles, such as empathy training, may need the SIA to challenge the user's expectations, revealing a dilemma that SIAs need to be discriminating, without being discriminatory. To design SIAs to avoid these harms, we offer a position paper grounded in two decades of SIA research that contributes the RESPECT framework and its operational design processes: Risk management, Empowerment, Safety, Personalisation, Empathy, Context awareness, and Technology appropriateness.