Should AI-enabled medical devices be explainable?

Rita Matulionyte*, Paul Nolan, Farah Magrabi, Amin Beheshti

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

8 Citations (Scopus)

Abstract

Despite its exponential growth, artificial intelligence (AI) in healthcare faces various challenges. One of them is a lack of explainability of AI medical devices, which arguably leads to insufficient trust in AI technologies, quality, and accountability and liability issues. The aim of this paper is to examine whether, why and to what extent AI explainability should be demanded with relation to AI-enabled medical devices and their outputs. Relying on a critical analysis of interdisciplinary literature on this topic and an empirical study, we conclude that the role of AI explainability in the medical AI context is a limited one. If narrowly defined, AI explainability principle is capable of addressing only a limited range of challenges associated with AI and is likely to reach fewer goals than sometimes expected. The study shows that, instead of technical explainability of medical AI devices, most stakeholders need more transparency around its development and quality assurance process.

Original languageEnglish
Pages (from-to)151-180
Number of pages30
JournalInternational Journal of Law and Information Technology
Volume30
Issue number2
Early online date6 Oct 2022
DOIs
Publication statusPublished - 2022

Keywords

  • artificial intelligence
  • healthcare
  • explainability
  • transparency
  • medical devices
  • clinical validation
  • informed consent
  • liability

Fingerprint

Dive into the research topics of 'Should AI-enabled medical devices be explainable?'. Together they form a unique fingerprint.

Cite this