Application of a deep learning system to detect papilledema on nonmydriatic ocular fundus photographs in an emergency department

Purpose: The Fundus photography vs Ophthalmoscopy Trial Outcomes in the Emergency Department (FOTO-ED) studies showed that ED providers poorly recognized funduscopic findings in patients in the ED. We tested a modified version of the Brain and Optic Nerve Study Artificial Intelligence (BONSAI) deep learning system on nonmydriatic fundus photographs from the FOTO-ED studies to determine if the deep learning system could have improved the detection of papilledema had it been available to ED providers as a real-time diagnostic aid. Design: Retrospective secondary analysis of a cohort of patients included in the FOTO-ED studies. Methods: The testing data set included 1608 photographs obtained from 828 patients in the FOTO-ED studies. Photographs were reclassified according to the optic disc classification system used by the deep learning system (“normal optic discs,” “papilledema,” and “other optic disc abnormalities”). The system's performance was evaluated by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity using a 1-vs-rest strategy, with reference to expert neuro-ophthalmologists. Results: The BONSAI deep learning system successfully distinguished normal from abnormal optic discs (AUC 0.92 [95% confidence interval {CI} 0.90-0.93]; sensitivity 75.6% [73.7%-77.5%] and specificity 89.6% [86.3%-92.8%]), and papilledema from normal and others (AUC 0.97 [0.95-0.99]; sensitivity 84.0% [75.0%-92.6%] and specificity 98.9% [98.5%-99.4%]). Six patients with missed papilledema in 1 eye were correctly identified by the deep learning system as having papilledema in the other eye. Conclusions: The BONSAI deep learning system was able to reliably identify papilledema and normal optic discs on nonmydriatic photographs obtained in the FOTO-ED studies. Our deep learning system has excellent potential as a diagnostic aid in EDs and non-ophthalmology clinics equipped with nonmydriatic fundus cameras.