This paper presents Macquarie University's participation to the BioASQ Synergy Task, and BioASQ9b Phase B. In each of these tasks, our participation focused on the use of query-focused extractive summarisation to obtain the ideal answers to medical questions. The Synergy Task is an end-to-end question answering task on COVID-19 where systems are required to return relevant documents, snippets, and answers to a given question. Given the absence of training data, we used a query-focused summarisation system that was trained with the BioASQ8b training data set and we experimented with methods to retrieve the documents and snippets. Considering the poor quality of the documents and snippets retrieved by our system, we observed reasonably good quality in the answers returned. For phase B of the BioASQ9b task, the relevant documents and snippets were already included in the test data. Our system split the snippets into candidate sentences and used BERT variants under a sentence classification setup. The system used the question and candidate sentence as input and was trained to predict the likelihood of the candidate sentence being part of the ideal answer. The runs obtained either the best or second best ROUGE-F1 results of all participants to all batches of BioASQ9b. This shows that using BERT in a classification setup is a very strong baseline for the identification of ideal answers.
|Title of host publication||CLEF 2021 Working Notes|
|Subtitle of host publication||Proceedings of the Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum|
|Editors||Guglielmo Faggioli, Nicola Ferro, Alexis Joly, Maria Maistro, Florina Piroi|
|Place of Publication||Aachen, Germany|
|Number of pages||12|
|Publication status||Published - 2021|
|Event||2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021 - Virtual, Bucharest, Romania|
Duration: 21 Sep 2021 → 24 Sep 2021
|Name||CEUR Workshop Proceedings|
|Conference||2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021|
|Period||21/09/21 → 24/09/21|
Bibliographical notePublisher Copyright:
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
- Query-focused summarisation