Continuous learning as a threat to care

Purpose: To highlight the risk that the use of adaptive machine learning systems may result in clinicians spending more time on computers and less time caring for patients.

Methods: Philosophical and ethical reflection on issues identified in the literature on human-computer interaction.

Results: Adaptive machine learning systems learn from data generated by their use. Managing the process by which these systems improve is likely to require that clinicians pay more attention to them than they do to other software. The call for “explainable AI” will create a new “hermeneutic burden” for clinicians when it comes to the use of these systems.

Conclusions: Adaptive machine learning systems are likely to exacerbate problems associated with computers becoming the “third party in the room” during clinical consultations. A key challenge is to ensure that the application of adaptive learning enhances, rather than damages, the relationship between clinician and patient.