Joint action segmentation and classification by an extended hidden Markov model

Ehsan Zare Borzeshi, Oscar Perez Concha, Richard Yi Da Xu, Massimo Piccardi

Research output: Contribution to journalArticlepeer-review

20 Citations (Scopus)


Hidden Markov models (HMMs) provide joint segmentation and classification of sequential data by efficient inference algorithms and have therefore been employed in fields as diverse as speech recognition, document processing, and genomics. However, conventional HMMs do not suit action segmentation in video due to the nature of the measurements which are often irregular in space and time, high dimensional and affected by outliers. For this reason, in this paper we present a joint action segmentation and classification approach based on an extended model: the hidden Markov model for multiple, irregular observations (HMM-MIO). Experiments performed over a concatenated version of the popular KTH action dataset and the challenging CMU multi-modal activity dataset (CMU-MMAC) report accuracies comparable to or higher than those of a bag-of-features approach, showing the usefulness of improved sequential models for joint action segmentation and classification tasks.

Original languageEnglish
Article number6616578
Pages (from-to)1207-1210
Number of pages4
JournalIEEE Signal Processing Letters
Issue number12
Publication statusPublished - 2013
Externally publishedYes


  • Action classification
  • Action segmentation
  • Hidden Markov Model
  • Joint segmentation and classification
  • Probabilistic PCA
  • Student's


Dive into the research topics of 'Joint action segmentation and classification by an extended hidden Markov model'. Together they form a unique fingerprint.

Cite this