Automatic grading of evidence: the 2011 ALTA shared task

Diego Mollá; Abeed Sarker

Automatic grading of evidence

the 2011 ALTA shared task

Diego Mollá, Abeed Sarker

Department of Computing

Research output: Contribution to journal › Conference paper

4 Downloads (Pure)

Abstract

The ALTA shared tasks are programming competitions where all participants attempt to solve the same problem, and the winner is the system with the best results. The 2011 ALTA shared task is the second in the series and it focuses on trying to automatically grade the level of clinical evidence in medical research papers. In this paper we describe the task, present the results of several baselines, and the results of our method. We apply a sequence of high precision machine learning classifiers with varying feature sets for each. In addition to using n-grams, we incorporate domain knowledge by representing specific medical concepts using their semantic categories. We also apply a specialised rule-based approach for automatically identifying the publication types of articles, which is then used as a feature set. Our approach obtains an accuracy of 62.84% which is a significant improvement over the baselines.

Original language	English
Pages (from-to)	4-8
Number of pages	5
Journal	Proceedings of the Australasian Language Technology Association Workshop 2011
Publication status	Published - 2011
Event	Australasian Language Technology Workshop (9th : 2011) - Canberra Duration: 1 Dec 2011 → 2 Dec 2011

Bibliographical note

Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Access to Document

Mq-18129-Publisher version (open access)Final published version, 374 KBLicence: Unspecified

Fingerprint Dive into the research topics of 'Automatic grading of evidence: the 2011 ALTA shared task'. Together they form a unique fingerprint.

Cite this

Mollá, D., & Sarker, A. (2011). Automatic grading of evidence: the 2011 ALTA shared task. Proceedings of the Australasian Language Technology Association Workshop 2011, 4-8.

@article{25168e96a64740d39605e29c862e4714,

title = "Automatic grading of evidence: the 2011 ALTA shared task",

abstract = "The ALTA shared tasks are programming competitions where all participants attempt to solve the same problem, and the winner is the system with the best results. The 2011 ALTA shared task is the second in the series and it focuses on trying to automatically grade the level of clinical evidence in medical research papers. In this paper we describe the task, present the results of several baselines, and the results of our method. We apply a sequence of high precision machine learning classifiers with varying feature sets for each. In addition to using n-grams, we incorporate domain knowledge by representing specific medical concepts using their semantic categories. We also apply a specialised rule-based approach for automatically identifying the publication types of articles, which is then used as a feature set. Our approach obtains an accuracy of 62.84{\%} which is a significant improvement over the baselines.",

author = "Diego Moll{\'a} and Abeed Sarker",

note = "Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.",

year = "2011",

language = "English",

pages = "4--8",

journal = "Proceedings of the Australasian Language Technology Association Workshop 2011",

issn = "1834-7037",

publisher = "Australian Language Technology Association",

}

TY - JOUR

T1 - Automatic grading of evidence

T2 - the 2011 ALTA shared task

AU - Mollá, Diego

AU - Sarker, Abeed

N1 - Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

PY - 2011

Y1 - 2011

N2 - The ALTA shared tasks are programming competitions where all participants attempt to solve the same problem, and the winner is the system with the best results. The 2011 ALTA shared task is the second in the series and it focuses on trying to automatically grade the level of clinical evidence in medical research papers. In this paper we describe the task, present the results of several baselines, and the results of our method. We apply a sequence of high precision machine learning classifiers with varying feature sets for each. In addition to using n-grams, we incorporate domain knowledge by representing specific medical concepts using their semantic categories. We also apply a specialised rule-based approach for automatically identifying the publication types of articles, which is then used as a feature set. Our approach obtains an accuracy of 62.84% which is a significant improvement over the baselines.

AB - The ALTA shared tasks are programming competitions where all participants attempt to solve the same problem, and the winner is the system with the best results. The 2011 ALTA shared task is the second in the series and it focuses on trying to automatically grade the level of clinical evidence in medical research papers. In this paper we describe the task, present the results of several baselines, and the results of our method. We apply a sequence of high precision machine learning classifiers with varying feature sets for each. In addition to using n-grams, we incorporate domain knowledge by representing specific medical concepts using their semantic categories. We also apply a specialised rule-based approach for automatically identifying the publication types of articles, which is then used as a feature set. Our approach obtains an accuracy of 62.84% which is a significant improvement over the baselines.

M3 - Conference paper

SP - 4

EP - 8

JO - Proceedings of the Australasian Language Technology Association Workshop 2011

JF - Proceedings of the Australasian Language Technology Association Workshop 2011

SN - 1834-7037

ER -