Abstract
Interpretable and robust medical diagnoses are essential traits for practicing clinicians. Most computer-augmented diagnostic systems suffer from three major problems: non-interpretability, limited modality analysis, and narrow focus. Existing frameworks can either deal with multimodality to some extent but suffer from non-interpretability or partially interpretable but provide a limited modality and multifaceted capabilities. Our work aims to integrate all these aspects in one complete framework to fully utilize the full spectrum of information offered by multiple modalities and facets. We propose our solution via our novel architecture VR-DiagNet, consisting of a planner and a classifier, optimized iteratively and cohesively. VR-DiagNet simulates the perceptual process of clinicians via the use of volumetric imaging information integrated with radiomic features modality; at the same time, it recreates human thought processes via a customized Monte Carlo Tree Search (MCTS) which constructs a volume-tailored experience tree to identify slices of interest (SoIs) in our multi-slice perception space. We conducted extensive experiments across two diagnostic tasks comprising six public medical volumetric benchmark datasets. Our findings showcase superior performance, as evidenced by heightened accuracy and area under the curve (AUC) metrics, reduced computational overhead, and expedited convergence while conclusively illustrating the immense value of integrating volumetric and radiomic modalities for our current problem setup.
Original language | English |
---|---|
Title of host publication | MM '24 |
Subtitle of host publication | proceedings of the 32nd ACM International Conference on Multimedia |
Place of Publication | New York |
Publisher | Association for Computing Machinery |
Pages | 10459-10467 |
Number of pages | 9 |
ISBN (Electronic) | 9798400706868 |
DOIs | |
Publication status | Published - 2024 |
Event | ACM International Conference on Multimedia (32nd : 2024) - Melbourne, Australia Duration: 28 Oct 2024 → 1 Nov 2024 Conference number: 32nd |
Conference
Conference | ACM International Conference on Multimedia (32nd : 2024) |
---|---|
Abbreviated title | MM '24 |
Country/Territory | Australia |
City | Melbourne |
Period | 28/10/24 → 1/11/24 |
Keywords
- Medical Volume Diagnosis
- Multimodality
- Radiomics
- Unsupervised Planning