Reputation management in crowdsourcing systems

Mohammad Allahbakhsh; Aleksandar Ignjatovic; Boualem Benatallah; Seyed Mehdi Reza Beheshti; Elisa Bertino; Norman Foo

doi:10.4108/icst.collaboratecom.2012.250499

Reputation management in crowdsourcing systems

Mohammad Allahbakhsh^*, Aleksandar Ignjatovic, Boualem Benatallah, Seyed Mehdi Reza Beheshti, Elisa Bertino, Norman Foo

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

42 Citations (Scopus)

Abstract

Worker selection is a significant and challenging issue in crowdsourcing systems. Such selection is usually based on an assessment of the reputation of the individual workers participating in such systems. However, assessing the credibility and adequacy of such calculated reputation is a real challenge. In this paper, we propose a reputation management model which leverages the values of the tasks completed, the credibility of the evaluators of the results of the tasks and time of evaluation of the results of these tasks in order to calculate more dependable quality metrics for workers and evaluators. The model has been implemented and experimentally validated.

Original language	English
Title of host publication	CollaborateCom 2012 - Proceedings of the 8th International Conference on Collaborative Computing
Subtitle of host publication	Networking, Applications and Worksharing
Pages	664-671
Number of pages	8
DOIs	https://doi.org/10.4108/icst.collaboratecom.2012.250499
Publication status	Published - 1 Dec 2012
Externally published	Yes
Event	8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, CollaborateCom 2012 - Pittsburgh, PA, United States Duration: 14 Oct 2012 → 17 Oct 2012

Conference

Conference	8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, CollaborateCom 2012
Country/Territory	United States
City	Pittsburgh, PA
Period	14/10/12 → 17/10/12

Keywords

Crowdsourcing
Degree of Fairness
Reputation

Access to Document

10.4108/icst.collaboratecom.2012.250499

Cite this

Allahbakhsh, M., Ignjatovic, A., Benatallah, B., Beheshti, S. M. R., Bertino, E., & Foo, N. (2012). Reputation management in crowdsourcing systems. In CollaborateCom 2012 - Proceedings of the 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing (pp. 664-671). [6450966] https://doi.org/10.4108/icst.collaboratecom.2012.250499

@inproceedings{403b00128abd461fb389c7af31babada,

title = "Reputation management in crowdsourcing systems",

abstract = "Worker selection is a significant and challenging issue in crowdsourcing systems. Such selection is usually based on an assessment of the reputation of the individual workers participating in such systems. However, assessing the credibility and adequacy of such calculated reputation is a real challenge. In this paper, we propose a reputation management model which leverages the values of the tasks completed, the credibility of the evaluators of the results of the tasks and time of evaluation of the results of these tasks in order to calculate more dependable quality metrics for workers and evaluators. The model has been implemented and experimentally validated.",

keywords = "Crowdsourcing, Degree of Fairness, Reputation",

author = "Mohammad Allahbakhsh and Aleksandar Ignjatovic and Boualem Benatallah and Beheshti, {Seyed Mehdi Reza} and Elisa Bertino and Norman Foo",

year = "2012",

month = dec,

day = "1",

doi = "10.4108/icst.collaboratecom.2012.250499",

language = "English",

isbn = "9781936968367",

pages = "664--671",

booktitle = "CollaborateCom 2012 - Proceedings of the 8th International Conference on Collaborative Computing",

note = "8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, CollaborateCom 2012 ; Conference date: 14-10-2012 Through 17-10-2012",

}

Allahbakhsh, M, Ignjatovic, A, Benatallah, B, Beheshti, SMR, Bertino, E & Foo, N 2012, Reputation management in crowdsourcing systems. in CollaborateCom 2012 - Proceedings of the 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing., 6450966, pp. 664-671, 8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, CollaborateCom 2012, Pittsburgh, PA, United States, 14/10/12. https://doi.org/10.4108/icst.collaboratecom.2012.250499

Reputation management in crowdsourcing systems. / Allahbakhsh, Mohammad; Ignjatovic, Aleksandar; Benatallah, Boualem et al.

CollaborateCom 2012 - Proceedings of the 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing. 2012. p. 664-671 6450966.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Reputation management in crowdsourcing systems

AU - Allahbakhsh, Mohammad

AU - Ignjatovic, Aleksandar

AU - Benatallah, Boualem

AU - Beheshti, Seyed Mehdi Reza

AU - Bertino, Elisa

AU - Foo, Norman

PY - 2012/12/1

Y1 - 2012/12/1

N2 - Worker selection is a significant and challenging issue in crowdsourcing systems. Such selection is usually based on an assessment of the reputation of the individual workers participating in such systems. However, assessing the credibility and adequacy of such calculated reputation is a real challenge. In this paper, we propose a reputation management model which leverages the values of the tasks completed, the credibility of the evaluators of the results of the tasks and time of evaluation of the results of these tasks in order to calculate more dependable quality metrics for workers and evaluators. The model has been implemented and experimentally validated.

AB - Worker selection is a significant and challenging issue in crowdsourcing systems. Such selection is usually based on an assessment of the reputation of the individual workers participating in such systems. However, assessing the credibility and adequacy of such calculated reputation is a real challenge. In this paper, we propose a reputation management model which leverages the values of the tasks completed, the credibility of the evaluators of the results of the tasks and time of evaluation of the results of these tasks in order to calculate more dependable quality metrics for workers and evaluators. The model has been implemented and experimentally validated.

KW - Crowdsourcing

KW - Degree of Fairness

KW - Reputation

UR - http://www.scopus.com/inward/record.url?scp=84874414989&partnerID=8YFLogxK

U2 - 10.4108/icst.collaboratecom.2012.250499

DO - 10.4108/icst.collaboratecom.2012.250499

M3 - Conference proceeding contribution

AN - SCOPUS:84874414989

SN - 9781936968367

SP - 664

EP - 671

BT - CollaborateCom 2012 - Proceedings of the 8th International Conference on Collaborative Computing

T2 - 8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, CollaborateCom 2012

Y2 - 14 October 2012 through 17 October 2012

ER -

Reputation management in crowdsourcing systems

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this