Investigating the effects of selective sampling on the annotation task

Ben Hachey; Beatrice Alex; Markus Becker

Investigating the effects of selective sampling on the annotation task

Ben Hachey^*, Beatrice Alex, Markus Becker

^*Corresponding author for this work

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

40 Citations (Scopus)

Abstract

We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.

Original language	English
Title of host publication	CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning
Pages	144-151
Number of pages	8
Publication status	Published - 2005
Event	9th Conference on Computational Natural Language Learning, CoNLL - 2005 - Ann Arbor, United States Duration: 29 Jun 2005 → 30 Jun 2005

Other

Other	9th Conference on Computational Natural Language Learning, CoNLL - 2005
Country/Territory	United States
City	Ann Arbor
Period	29/06/05 → 30/06/05

Cite this

@inproceedings{ed2768f4a9d84e6d9b1760b0a0c820c8,

title = "Investigating the effects of selective sampling on the annotation task",

abstract = "We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.",

author = "Ben Hachey and Beatrice Alex and Markus Becker",

year = "2005",

language = "English",

pages = "144--151",

booktitle = "CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning",

note = "9th Conference on Computational Natural Language Learning, CoNLL - 2005 ; Conference date: 29-06-2005 Through 30-06-2005",

}

TY - GEN

T1 - Investigating the effects of selective sampling on the annotation task

AU - Hachey, Ben

AU - Alex, Beatrice

AU - Becker, Markus

PY - 2005

Y1 - 2005

N2 - We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.

AB - We report on an active learning experiment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of labelled data required to train a supervised learner by selectively sampling more informative data points for human annotation. We inspect double annotation data from the same domain and quantify potential problems concerning annotators' performance. For data selectively sampled according to different selection metrics, we find lower inter-annotator agreement and higher per token annotation times. However, overall results confirm the utility of active learning.

UR - http://www.scopus.com/inward/record.url?scp=37249016011&partnerID=8YFLogxK

M3 - Conference proceeding contribution

AN - SCOPUS:37249016011

SP - 144

EP - 151

BT - CoNLL 2005 - Proceedings of the Ninth Conference on Computational Natural Language Learning

T2 - 9th Conference on Computational Natural Language Learning, CoNLL - 2005

Y2 - 29 June 2005 through 30 June 2005

ER -

Investigating the effects of selective sampling on the annotation task

Abstract

Other

Other files and links

Fingerprint

Cite this