Query-focused extractive summarisation for finding ideal answers to biomedical and COVID-19 questions

Diego Mollá*, Urvashi Khanna, Dima Galat, Vincent Nguyen, Maciej Rybinski

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

Abstract

This paper presents Macquarie University's participation to the BioASQ Synergy Task, and BioASQ9b Phase B. In each of these tasks, our participation focused on the use of query-focused extractive summarisation to obtain the ideal answers to medical questions. The Synergy Task is an end-to-end question answering task on COVID-19 where systems are required to return relevant documents, snippets, and answers to a given question. Given the absence of training data, we used a query-focused summarisation system that was trained with the BioASQ8b training data set and we experimented with methods to retrieve the documents and snippets. Considering the poor quality of the documents and snippets retrieved by our system, we observed reasonably good quality in the answers returned. For phase B of the BioASQ9b task, the relevant documents and snippets were already included in the test data. Our system split the snippets into candidate sentences and used BERT variants under a sentence classification setup. The system used the question and candidate sentence as input and was trained to predict the likelihood of the candidate sentence being part of the ideal answer. The runs obtained either the best or second best ROUGE-F1 results of all participants to all batches of BioASQ9b. This shows that using BERT in a classification setup is a very strong baseline for the identification of ideal answers.

Original languageEnglish
Title of host publicationCLEF 2021 Working Notes
Subtitle of host publicationProceedings of the Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum
EditorsGuglielmo Faggioli, Nicola Ferro, Alexis Joly, Maria Maistro, Florina Piroi
Place of PublicationAachen, Germany
PublisherCEUR
Pages274-285
Number of pages12
Publication statusPublished - 2021
Event2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021 - Virtual, Bucharest, Romania
Duration: 21 Sep 202124 Sep 2021

Publication series

NameCEUR Workshop Proceedings
PublisherCEUR
Volume2936
ISSN (Electronic)1613-0073

Conference

Conference2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021
CountryRomania
CityVirtual, Bucharest
Period21/09/2124/09/21

Bibliographical note

Publisher Copyright:
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

Keywords

  • BioASQ
  • Synergy
  • Biomedical
  • Query-focused summarisation
  • COVID-19
  • BERT

Cite this