The GREC challenge: Overview and evaluation results

Anja Belz; Eric Kow; Jette Viethen; Albert Gatt

The GREC challenge: Overview and evaluation results

Anja Belz^*, Eric Kow, Jette Viethen, Albert Gatt

^*Corresponding author for this work

Faculty of Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

11 Citations (Scopus)

Abstract

The GREC Task at REG'08 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted a total of 6 systems, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in a reading/comprehension experiment involving human subjects. This report describes the GREC Task and the evaluation methods, gives brief descriptions of the participating systems, and presents the evaluation results.

Original language	English
Title of host publication	INLG 2008 - 5th International Natural Language Generation Conference, Proceedings of the Conference
Place of Publication	Washington DC
Publisher	Association for Computing Machinery (ACM)
Pages	183-191
Number of pages	9
Publication status	Published - 2008
Event	5th International Natural Language Generation Conference, INLG 2008 - Salt Fork, OH, United States Duration: 12 Jun 2008 → 14 Jun 2008

Other

Other	5th International Natural Language Generation Conference, INLG 2008
Country/Territory	United States
City	Salt Fork, OH
Period	12/06/08 → 14/06/08

Cite this

@inproceedings{e90ffa5f49f645d391ae215f64fae8be,

title = "The GREC challenge: Overview and evaluation results",

abstract = "The GREC Task at REG'08 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted a total of 6 systems, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in a reading/comprehension experiment involving human subjects. This report describes the GREC Task and the evaluation methods, gives brief descriptions of the participating systems, and presents the evaluation results.",

author = "Anja Belz and Eric Kow and Jette Viethen and Albert Gatt",

year = "2008",

language = "English",

pages = "183--191",

booktitle = "INLG 2008 - 5th International Natural Language Generation Conference, Proceedings of the Conference",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "5th International Natural Language Generation Conference, INLG 2008 ; Conference date: 12-06-2008 Through 14-06-2008",

}

Belz, A, Kow, E, Viethen, J & Gatt, A 2008, The GREC challenge: Overview and evaluation results. in INLG 2008 - 5th International Natural Language Generation Conference, Proceedings of the Conference. Association for Computing Machinery (ACM), Washington DC, pp. 183-191, 5th International Natural Language Generation Conference, INLG 2008, Salt Fork, OH, United States, 12/06/08.

The GREC challenge : Overview and evaluation results. / Belz, Anja; Kow, Eric; Viethen, Jette et al.

INLG 2008 - 5th International Natural Language Generation Conference, Proceedings of the Conference. Washington DC : Association for Computing Machinery (ACM), 2008. p. 183-191.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - The GREC challenge

T2 - 5th International Natural Language Generation Conference, INLG 2008

AU - Belz, Anja

AU - Kow, Eric

AU - Viethen, Jette

AU - Gatt, Albert

PY - 2008

Y1 - 2008

N2 - The GREC Task at REG'08 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted a total of 6 systems, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in a reading/comprehension experiment involving human subjects. This report describes the GREC Task and the evaluation methods, gives brief descriptions of the participating systems, and presents the evaluation results.

AB - The GREC Task at REG'08 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted a total of 6 systems, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in a reading/comprehension experiment involving human subjects. This report describes the GREC Task and the evaluation methods, gives brief descriptions of the participating systems, and presents the evaluation results.

UR - http://www.scopus.com/inward/record.url?scp=77956337574&partnerID=8YFLogxK

M3 - Conference proceeding contribution

AN - SCOPUS:77956337574

SP - 183

EP - 191

BT - INLG 2008 - 5th International Natural Language Generation Conference, Proceedings of the Conference

PB - Association for Computing Machinery (ACM)

CY - Washington DC

Y2 - 12 June 2008 through 14 June 2008

ER -

The GREC challenge: Overview and evaluation results

Abstract

Other

Other files and links

Fingerprint

Cite this