CrowdRec

Trust-aware worker recommendation in crowdsourcing environments

Bin Ye, Yan Wang

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contribution

9 Citations (Scopus)

Abstract

On a crowdsourcing platform consisting of requestersand workers, it is a challenge to recommend suitableworkers for a human intelligence task (HIT) published by a requester. A suitable worker is the one who has a high probability of submitting a correct answer for the published HIT. However, there are four problems that make the existing methods be less effective in recommending suitable workers. First, on most of crowdsourcing platforms, the great majority of workers have good reputations and thus are regarded as homogenous workers who have equal opportunities to be recommended. Secondly, dishonest workers may gain recommendations by counterfeiting good reputations and overstating personal skills. Moreover, the classical data sparsity and cold start problems co-exist in crowdsourcing environments. To effectively differentiate homogenous workers, we firstlycalculate a worker's performance in different types of HITspublished by different requesters. Aiming to improve the accuracy of predicting a worker's performance, we propose a metric which separately considers two requesters' similarities in transacting with the common workers they trust and in transacting with the common workers they distrust. Afterwards, targeting dishonest behaviours, we propose a transaction-based trust model. Targeting the data sparsity problem, we propose a new trust sub-network extraction algorithm (TSE) to discover more requesters who can provide trustworthy opinions for generating recommendations. Furthermore, we propose two strategiesfor solving the cold start problem. Finally, by incorporatingthe similarity metric, the new trust model, the new trustsub-network extraction algorithm and the new strategies, wepropose a novel trust-aware worker recommendation methodCrowdRec. The experimental results illustrate that CrowdRecsignificantly outperforms CF and three state-of-the-art trustbased recommendation methods both in terms of accuracy and coverage.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Web Services, ICWS 2016
EditorsStephan Reiff-Marganiec
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1-8
Number of pages8
ISBN (Electronic)9781509026753
DOIs
Publication statusPublished - 31 Aug 2016
Event23rd IEEE International Conference on Web Services, ICWS 2016 - San Francisco, United States
Duration: 27 Jun 20162 Jul 2016

Other

Other23rd IEEE International Conference on Web Services, ICWS 2016
CountryUnited States
CitySan Francisco
Period27/06/162/07/16

Keywords

  • Collaborative filtering
  • Crowdsourcing
  • Trust network
  • Worker recommendation

Cite this

Ye, B., & Wang, Y. (2016). CrowdRec: Trust-aware worker recommendation in crowdsourcing environments. In S. Reiff-Marganiec (Ed.), Proceedings - 2016 IEEE International Conference on Web Services, ICWS 2016 (pp. 1-8). [7557978] Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICWS.2016.10