Investigating the effects of selective sampling on the annotation task

Ben Hachey, Beatrice Alex, Markus Becker

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

Abstract

We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.

LanguageEnglish
Title of host publicationCoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning
Pages144-151
Number of pages8
Publication statusPublished - 2005
Event9th Conference on Computational Natural Language Learning, CoNLL - 2005 - Ann Arbor, United States
Duration: 29 Jun 200530 Jun 2005

Other

Other9th Conference on Computational Natural Language Learning, CoNLL - 2005
CountryUnited States
CityAnn Arbor
Period29/06/0530/06/05

Fingerprint

Sampling
Astronomy
learning
Problem-Based Learning
experiment
Experiments
performance
time

Cite this

Hachey, B., Alex, B., & Becker, M. (2005). Investigating the effects of selective sampling on the annotation task. In CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning (pp. 144-151)
Hachey, Ben ; Alex, Beatrice ; Becker, Markus. / Investigating the effects of selective sampling on the annotation task. CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning. 2005. pp. 144-151
@inproceedings{ed2768f4a9d84e6d9b1760b0a0c820c8,
title = "Investigating the effects of selective sampling on the annotation task",
abstract = "We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.",
author = "Ben Hachey and Beatrice Alex and Markus Becker",
year = "2005",
language = "English",
pages = "144--151",
booktitle = "CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning",

}

Hachey, B, Alex, B & Becker, M 2005, Investigating the effects of selective sampling on the annotation task. in CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning. pp. 144-151, 9th Conference on Computational Natural Language Learning, CoNLL - 2005, Ann Arbor, United States, 29/06/05.

Investigating the effects of selective sampling on the annotation task. / Hachey, Ben; Alex, Beatrice; Becker, Markus.

CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning. 2005. p. 144-151.

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

TY - GEN

T1 - Investigating the effects of selective sampling on the annotation task

AU - Hachey, Ben

AU - Alex, Beatrice

AU - Becker, Markus

PY - 2005

Y1 - 2005

N2 - We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.

AB - We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.

UR - http://www.scopus.com/inward/record.url?scp=37249016011&partnerID=8YFLogxK

M3 - Conference proceeding contribution

SP - 144

EP - 151

BT - CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning

ER -

Hachey B, Alex B, Becker M. Investigating the effects of selective sampling on the annotation task. In CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning. 2005. p. 144-151