Automatic grading of evidence: the 2011 ALTA shared task

Diego Mollá, Abeed Sarker

Research output: Contribution to journalConference paper

6 Citations (Scopus)
10 Downloads (Pure)


The ALTA shared tasks are programming competitions where all participants attempt to solve the same problem, and the winner is the system with the best results. The 2011 ALTA shared task is the second in the series and it focuses on trying to automatically grade the level of clinical evidence in medical research papers. In this paper we describe the task, present the results of several baselines, and the results of our method. We apply a sequence of high precision machine learning classifiers with varying feature sets for each. In addition to using n-grams, we incorporate domain knowledge by representing specific medical concepts using their semantic categories. We also apply a specialised rule-based approach for automatically identifying the publication types of articles, which is then used as a feature set. Our approach obtains an accuracy of 62.84% which is a significant improvement over the baselines.
Original languageEnglish
Pages (from-to)4-8
Number of pages5
JournalProceedings of the Australasian Language Technology Association Workshop 2011
Publication statusPublished - 2011
EventAustralasian Language Technology Workshop (9th : 2011) - Canberra
Duration: 1 Dec 20112 Dec 2011

Bibliographical note

Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.


Dive into the research topics of 'Automatic grading of evidence: the 2011 ALTA shared task'. Together they form a unique fingerprint.

Cite this