Sentence pair modeling based on semantic feature map for human interaction with IoT devices

Rui Yu, Wenpeng Lu*, Huimin Lu, Shoujin Wang, Fangfang Li, Xu Zhang, Jiguo Yu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


The rapid development of Internet of Things (IoT) brings an urgent requirement on intelligent human–device interactions using natural language, which are critical for facilitating people to use IoT devices. The efficient interactive approaches depend on various natural language understanding technologies. Among them, sentence pair modeling (SPM) is essential, where neural networks have achieved great success in SPM area due to their powerful abilities in feature extraction and representation. However, as sentences are one-dimensional (1D) texts, the available neural networks are usually limited to 1D sequential models, which prevents the performance improvement of SPM task. To address this gap, in this paper, we propose a novel neural architecture for sentence pair modeling, which utilizes 1D sentences to construct multi-dimensional feature maps similar to images containing multiple color channels. Based on the feature maps, more kinds of neural models become applicable on SPM task, including 2D CNN. In the proposed model, first, the sentence on a specific granularity is encoded with BiLSTM to generate the representation on this granularity, which is viewed as a special channel of the sentence. The representations from different granularity are merged together to construct semantic feature map of the input sentence. Then, 2D CNN is employed to encode the feature map to capture the deeper semantic features contained in the sentence. Next, another 2D CNN is utilized to capture the interactive matching features between sentences, followed by 2D max-pooling and attention mechanism to generate the final matching representation. Finally, the matching degree of sentences are judged with a sigmoid function according to the matching representation. Extensive experiments are conducted on two real-world data sets. In comparison with benchmarks, the proposed model achieved remarkable results, and performed better or comparably with BERT-based models. Our work is beneficial to building a more powerful humanized interaction system with IoT devices.
Original languageEnglish
Pages (from-to)3081-3099
Number of pages19
JournalInternational Journal of Machine Learning and Cybernetics
Issue number11
Early online date14 Jun 2021
Publication statusPublished - Nov 2021


  • 2D CNN
  • Human–device interaction
  • IoT
  • Semantic feature map
  • Sentence pair modeling


Dive into the research topics of 'Sentence pair modeling based on semantic feature map for human interaction with IoT devices'. Together they form a unique fingerprint.

Cite this