Abstract
Introduction: Many studies display significant heterogeneity in the reliability of artificial intelligence (AI) assessment of minimally invasive surgical (MIS) skills. Our objective is to investigate whether AI systems utilising standardised objective metrics (SOMs) as the basis of skill assessment can provide a clearer understanding of the current state of such technology.
Methods: We systematically searched Medline, Embase, Scopus, CENTRAL and Web of Science from March 2023 to September 2023. Results were compiled as a narrative review.
Results: Twenty-four citations were analysed. Overall accuracy of AI systems in predicting overall SOM score of a procedure ranged from 63% to 100%. The most frequently used SOM by AI algorithms were Objective Structured Assessment of Technical Skills (OSATS) (8/24) and Global Evaluative Assessment of Robotic Skills (GEARS) (8/24).
Conclusions: Stratifying for AI studies which employed SOMs to assess surgical skill did not reduce heterogeneity of reported reliability. Our study identifies key issues within the current literature, which, once addressed, could allow more meaningful comparisons between studies.
Methods: We systematically searched Medline, Embase, Scopus, CENTRAL and Web of Science from March 2023 to September 2023. Results were compiled as a narrative review.
Results: Twenty-four citations were analysed. Overall accuracy of AI systems in predicting overall SOM score of a procedure ranged from 63% to 100%. The most frequently used SOM by AI algorithms were Objective Structured Assessment of Technical Skills (OSATS) (8/24) and Global Evaluative Assessment of Robotic Skills (GEARS) (8/24).
Conclusions: Stratifying for AI studies which employed SOMs to assess surgical skill did not reduce heterogeneity of reported reliability. Our study identifies key issues within the current literature, which, once addressed, could allow more meaningful comparisons between studies.
| Original language | English |
|---|---|
| Article number | 116074 |
| Pages (from-to) | 1-11 |
| Number of pages | 11 |
| Journal | American Journal of Surgery |
| Volume | 241 |
| DOIs | |
| Publication status | Accepted/In press - 6 Nov 2024 |
Bibliographical note
Copyright the Author(s) 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.Keywords
- Artificial Intelligence
- Surgical education
- Minimally invasive surgery
- Standardised objective metrics
- Remote learning
- Artificial intelligence