BERT-CoQAC: BERT-based conversational question answering in context

Munazza Zaib*, Dai Hoang Tran, Subhash Sagar, Adnan Mahmood, Wei E. Zhang, Quan Z. Sheng

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

3 Citations (Scopus)


As one promising way to inquire about any particular information through a dialog with the bot, question answering dialog systems have gained increasing research interests recently. Designing interactive QA systems has always been a challenging task in natural language processing and used as a benchmark to evaluate machine’s ability of natural language understanding. However, such systems often struggle when the question answering is carried out in multiple turns by the users to seek more information based on what they have already learned, thus, giving rise to another complicated form called Conversational Question Answering (CQA). CQA systems are often criticized for not understanding or utilizing the previous context of the conversation when answering the questions. To address the research gap, in this paper, we explore how to integrate the conversational history into the neural machine comprehension system. On one hand, we introduce a framework based on publicly available pre-trained language model called BERT for incorporating history turns into the system. On the other hand, we propose a history selection mechanism that selects the turns that are relevant and contributes the most to answer the current question. Experimentation results revealed that our framework is comparable in performance with the state-of-the-art models on the QuAC ( ) leader board. We also conduct a number of experiments to show the side effects of using entire context information which brings unnecessary information and noise signals resulting in a decline in the model’s performance.

Original languageEnglish
Title of host publicationParallel Architectures, Algorithms and Programming
Subtitle of host publication11th International Symposium, PAAP 2020 Shenzhen, China, December 28–30, 2020 Proceedings
EditorsLi Ning, Vincent Chau, Francis Lau
Place of PublicationSingapore
PublisherSpringer, Springer Nature
Number of pages11
ISBN (Electronic)9789811600104
ISBN (Print)9789811600098
Publication statusPublished - 2021
Event11th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2020 - Shenzhen, China
Duration: 28 Dec 202030 Dec 2020

Publication series

NameCommunications in Computer and Information Science
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937


Conference11th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2020


  • Machine comprehension
  • Information retrieval
  • Deep learning
  • Deep learning applications


Dive into the research topics of 'BERT-CoQAC: BERT-based conversational question answering in context'. Together they form a unique fingerprint.

Cite this