Interpretable parallel recurrent neural networks with convolutional attentions for multi-modality activity modeling

Kaixuan Chen, Lina Yao, Xianzhi Wang, Dalin Zhang, Tao Gu, Zhiwen Yu, Zheng Yang

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

15 Citations (Scopus)

Abstract

Multimodal features play a key role in wearable sensor based human activity recognition (HAR). Selecting the most salient features adaptively is a promising way to maximize the effectiveness of multimodal sensor data. In this regard, we propose a 'collect fully and select wisely' principle as well as an interpretable parallel recurrent model with convolutional attentions to improve the recognition performance. We first collect modality features and the relations between each pair of features to generate activity frames, and then introduce an attention mechanism to select the most prominent regions from activity frames precisely. The selected frames not only maximize the utilization of valid features but also reduce the number of features to be computed effectively. We further analyze the accuracy and interpretability of the proposed model based on extensive experiments. The results show that our model achieves competitive performance on two benchmarked datasets and works well in real life scenarios.

Original languageEnglish
Title of host publication2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages8
ISBN (Electronic)9781509060146
DOIs
Publication statusPublished - 2018
Externally publishedYes
Event2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil
Duration: 8 Jul 201813 Jul 2018

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2018-July

Conference

Conference2018 International Joint Conference on Neural Networks, IJCNN 2018
CountryBrazil
CityRio de Janeiro
Period8/07/1813/07/18

Keywords

  • HAR
  • attention
  • deep learning
  • wearable sensors

Fingerprint

Dive into the research topics of 'Interpretable parallel recurrent neural networks with convolutional attentions for multi-modality activity modeling'. Together they form a unique fingerprint.

Cite this