Video object segmentation with occlusion map

Extracting foreground objects from a video captured by a hand-held camera has been a new challenge in video segmentation, since most of the existing approaches generally work well provided that certain assumptions on the background scene or camera motion (e. g. still surveillance cameras) are imposed. While some approaches exploit several clues such as depth and motion to extract the foreground layer from handheld camera videos, we propose to leverage the advances in high quality interactive image segmentation. That is, we treat each video frame as an individual image and segment foreground objects with interactive image segmentation algorithms. In order to simulate user interactions, we derive reliable occlusion map for foreground objects and use the occlusion map as the "seeding" interactive input to an interactive image segmentation approach. In this paper, we employ an optical flow based occlusion detection approach for extracting the occlusion map and Geodesic star convexity based interactive image segmentation approach. In order to obtain accurate "seeding" user interactions, both forward and backward occlusion maps are computed and utilized. As a result, our approach is able to extract the whole objects having only partial movements, which overcomes the limitation of the state-of-the-art algorithm. Experimental results demonstrate both the effectiveness and efficiency of our proposed approach.