A systematic review of large language models in medical specialties: applications, challenges and future directions

This systematic review evaluates recent literature from January 2021 to March 2024 on large language model (LLM) applications across diverse medical specialties. Searching PubMed, Web of Science, and Scopus, we included 84 studies. LLMs were applied to tasks such as clinical natural language processing, medical decision support, education, and aiding diagnostic processes. While studies reported benefits such as improved efficiency and, in some specific NLP tasks, high accuracy above 90%, significant challenges persist concerning reliability, ethical implications, and performance consistency, with accuracy in broader diagnostic support applications showing substantial variability, with some as low as 3%. The overall risk of bias in the reviewed literature was considerably low in 72 studies. Key findings highlight a substantial heterogeneity in LLM performance across different medical tasks and contexts, preventing meta-analysis due to a lack of standardized methodologies. Future efforts should prioritize developing domain-specific LLMs using robust medical data and establishing rigorous validation standards to ensure their safe and effective clinical integration. Trial registration: PROSPERO (CRD42024561381).