Towards information retrieval evaluation with reduced and only positive judgements

Diego Mollá; David Martinez; Iman Amini

Towards information retrieval evaluation with reduced and only positive judgements

Diego Mollá, David Martinez, Iman Amini

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

Abstract

This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.

Original language	English
Title of host publication	Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013
Editors	Laurianne Sitbon
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	109-112
Number of pages	4
ISBN (Print)	9781450325240
Publication status	Published - 2013
Event	18th Australasian Document Computing Symposium, ADCS 2013 - Brisbane, QLD, Australia Duration: 5 Dec 2013 → 6 Dec 2013

Other

Other	18th Australasian Document Computing Symposium, ADCS 2013
Country/Territory	Australia
City	Brisbane, QLD
Period	5/12/13 → 6/12/13

Cite this

@inproceedings{710e0810bc234c139fa8b5277315546c,

title = "Towards information retrieval evaluation with reduced and only positive judgements",

abstract = "This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.",

author = "Diego Moll{\'a} and David Martinez and Iman Amini",

year = "2013",

language = "English",

isbn = "9781450325240",

pages = "109--112",

editor = "Laurianne Sitbon",

booktitle = "Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "18th Australasian Document Computing Symposium, ADCS 2013 ; Conference date: 05-12-2013 Through 06-12-2013",

}

Mollá, D, Martinez, D & Amini, I 2013, Towards information retrieval evaluation with reduced and only positive judgements. in L Sitbon (ed.), Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013. Association for Computing Machinery (ACM), New York, pp. 109-112, 18th Australasian Document Computing Symposium, ADCS 2013, Brisbane, QLD, Australia, 5/12/13.

Towards information retrieval evaluation with reduced and only positive judgements. / Mollá, Diego; Martinez, David; Amini, Iman.
Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013. ed. / Laurianne Sitbon. New York: Association for Computing Machinery (ACM), 2013. p. 109-112.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Towards information retrieval evaluation with reduced and only positive judgements

AU - Mollá, Diego

AU - Martinez, David

AU - Amini, Iman

PY - 2013

Y1 - 2013

N2 - This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.

AB - This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.

UR - http://www.scopus.com/inward/record.url?scp=84892724689&partnerID=8YFLogxK

M3 - Conference proceeding contribution

AN - SCOPUS:84892724689

SN - 9781450325240

SP - 109

EP - 112

BT - Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013

A2 - Sitbon, Laurianne

PB - Association for Computing Machinery (ACM)

CY - New York

T2 - 18th Australasian Document Computing Symposium, ADCS 2013

Y2 - 5 December 2013 through 6 December 2013

ER -

Towards information retrieval evaluation with reduced and only positive judgements

Abstract

Other

Other files and links

Fingerprint

Cite this