Towards information retrieval evaluation with reduced and only positive judgements

Diego Mollá, David Martinez, Iman Amini

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

Abstract

This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.

LanguageEnglish
Title of host publicationProceedings of the 18th Australasian Document Computing Symposium, ADCS 2013
EditorsLaurianne Sitbon
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Pages109-112
Number of pages4
ISBN (Print)9781450325240
Publication statusPublished - 2013
Event18th Australasian Document Computing Symposium, ADCS 2013 - Brisbane, QLD, Australia
Duration: 5 Dec 20136 Dec 2013

Other

Other18th Australasian Document Computing Symposium, ADCS 2013
CountryAustralia
CityBrisbane, QLD
Period5/12/136/12/13

Fingerprint

Information retrieval

Cite this

Mollá, D., Martinez, D., & Amini, I. (2013). Towards information retrieval evaluation with reduced and only positive judgements. In L. Sitbon (Ed.), Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013 (pp. 109-112). New York: Association for Computing Machinery (ACM).
Mollá, Diego ; Martinez, David ; Amini, Iman. / Towards information retrieval evaluation with reduced and only positive judgements. Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013. editor / Laurianne Sitbon. New York : Association for Computing Machinery (ACM), 2013. pp. 109-112
@inproceedings{710e0810bc234c139fa8b5277315546c,
title = "Towards information retrieval evaluation with reduced and only positive judgements",
abstract = "This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.",
author = "Diego Moll{\'a} and David Martinez and Iman Amini",
year = "2013",
language = "English",
isbn = "9781450325240",
pages = "109--112",
editor = "Laurianne Sitbon",
booktitle = "Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013",
publisher = "Association for Computing Machinery (ACM)",
address = "United States",

}

Mollá, D, Martinez, D & Amini, I 2013, Towards information retrieval evaluation with reduced and only positive judgements. in L Sitbon (ed.), Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013. Association for Computing Machinery (ACM), New York, pp. 109-112, 18th Australasian Document Computing Symposium, ADCS 2013, Brisbane, QLD, Australia, 5/12/13.

Towards information retrieval evaluation with reduced and only positive judgements. / Mollá, Diego; Martinez, David; Amini, Iman.

Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013. ed. / Laurianne Sitbon. New York : Association for Computing Machinery (ACM), 2013. p. 109-112.

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

TY - GEN

T1 - Towards information retrieval evaluation with reduced and only positive judgements

AU - Mollá, Diego

AU - Martinez, David

AU - Amini, Iman

PY - 2013

Y1 - 2013

N2 - This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.

AB - This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.

UR - http://www.scopus.com/inward/record.url?scp=84892724689&partnerID=8YFLogxK

M3 - Conference proceeding contribution

SN - 9781450325240

SP - 109

EP - 112

BT - Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013

A2 - Sitbon, Laurianne

PB - Association for Computing Machinery (ACM)

CY - New York

ER -

Mollá D, Martinez D, Amini I. Towards information retrieval evaluation with reduced and only positive judgements. In Sitbon L, editor, Proceedings of the 18th Australasian Document Computing Symposium, ADCS 2013. New York: Association for Computing Machinery (ACM). 2013. p. 109-112