Keyframe extraction from laparoscopic videos via diverse and weighted dictionary selection

Mingyang Ma, Shaohui Mei*, Shuai Wan, Zhiyong Wang, Zongyuan Ge, Vincent Lam, Dagan Feng

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    3 Citations (Scopus)

    Abstract

    Laparoscopic videos have been increasingly acquired for various purposes including surgical training and quality assurance, due to the wide adoption of laparoscopy in minimally invasive surgeries. However, it is very time consuming to view a large amount of laparoscopic videos, which prevents the values of laparoscopic video archives from being well exploited. In this paper, a dictionary selection based video summarization method is proposed to effectively extract keyframes for fast access of laparoscopic videos. Firstly, unlike the low-level feature used in most existing summarization methods, deep features are extracted from a convolutional neural network to effectively represent video frames. Secondly, based on such a deep representation, laparoscopic video summarization is formulated as a diverse and weighted dictionary selection model, in which image quality is taken into account to select high quality keyframes, and a diversity regularization term is added to reduce redundancy among the selected keyframes. Finally, an iterative algorithm with a rapid convergence rate is designed for model optimization, and the convergence of the proposed method is also analyzed. Experimental results on a recently released laparoscopic dataset demonstrate the clear superiority of the proposed methods. The proposed method can facilitate the access of key information in surgeries, training of junior clinicians, explanations to patients, and archive of case files.

    Original languageEnglish
    Pages (from-to)1686-1698
    Number of pages13
    JournalIEEE Journal of Biomedical and Health Informatics
    Volume25
    Issue number5
    DOIs
    Publication statusPublished - May 2021

    Keywords

    • Dictionary selection
    • keyframe extraction
    • laparoscopic videos
    • video summarization

    Fingerprint

    Dive into the research topics of 'Keyframe extraction from laparoscopic videos via diverse and weighted dictionary selection'. Together they form a unique fingerprint.

    Cite this