Vision-based freezing of gait detection with anatomic patch based representation

Parkinson’s disease (PD) impacts millions of people in the world. As freezing of gait (FoG) is a common symptom of PD patients that leads to falls and nursing home placement, it is very important to identify FoG event effectively and efficiently. Direct observation based assessment of FoG by doctors or trained experts is the de facto ‘gold standard’ for clinical diagnosis, which is time consuming. While several computer aided FoG event detection methods have been proposed, they were not particularly designed for video data collected during clinical diagnosis. In this paper, we treat video based FoG detection as a fine grained human action recognition problem and reduce the interference of visual content which is irrelevant to FoG, such as gait motion of supporting staff involved in clinical assessment. In order to effectively characterize FoG patterns, we propose to identify anatomic patches as the candidate regions which could be relevant to FoG events, and formulate FoG detection as a weakly-supervised learning task. The formulation will help identify the patches contributing to FoG events. To take both the global context of a clinical video and the local anatomic patches into account, several fusion strategies are investigated. Experimental results on videos collected from 45 subjects during clinical trials demonstrated promising results of our proposed method in terms of AUC of 0.869. To the best of our knowledge, this is one of the first studies on automatic FoG detection from clinical assessment videos.