Skip to main navigation Skip to search Skip to main content

Do clinical question answering systems really need specialised medical fine tuning?

Sushant Kumar Ray, Gautam Siddharth Kashyap, Sahil Tripathi, Nipun Joshi, Vijay Govindarajan*, Rafiq Ali, Jiechao Gao*, Usman Naseem*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

Abstract

Clinical Question-Answering (CQA) industry systems are increasingly rely on Large Language Models (LLMs), yet their deployment is often guided by the assumption that domain-specific fine-tuning is essential. Although specialised medical LLMs such as BioBERT, BioGPT, and PubMedBERT remain popular, they face practical limitations including narrow coverage, high retraining costs, and limited adaptability. Efforts based on Supervised Fine-Tuning (SFT) have attempted to address these assumptions but continue to reinforce what we term the SPECIALISATION FALLACY—the belief that specialised medical LLMs are inherently superior for CQA. To address this assumption, we introduce MEDASSESS-X, a deployment-industry-oriented CQA framework that applies alignment at inference time rather than through SFT. MEDASSESS-X uses lightweight steering vectors to guide model activations toward medically consistent reasoning without updating model weights or requiring domain-specific retraining. This inference-time alignment layer stabilises CQA performance across both general-purpose and specialised medical LLMs, thereby resolving the SPECIALISATION FALLACY. Empirically, MEDASSESS-X delivers consistent gains across all LLM families, improving Accuracy by up to +6%, Factual Consistency by +7%, and reducing Safety Error Rate by as much as 50%.

Original languageEnglish
Title of host publicationProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics
Subtitle of host publicationVolume 5: Industry Track
EditorsYevgen Matusevych, Gülşen Eryigit, Nikolaos Aletras
Place of PublicationKerrville, TX
PublisherAssociation for Computational Linguistics (ACL)
Pages869-876
Number of pages8
ISBN (Electronic)9798891763845
DOIs
Publication statusPublished - 2026
Event19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026 - Rabat, Morocco
Duration: 24 Mar 2026 → 29 Mar 2026

Conference

Conference19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026
Country/TerritoryMorocco
CityRabat
Period24/03/26 → 29/03/26

Bibliographical note

©2026 Association for Computational Linguistics. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Fingerprint

Dive into the research topics of 'Do clinical question answering systems really need specialised medical fine tuning?'. Together they form a unique fingerprint.

Cite this