TWLR: a novel truth inference approach based on worker representations for crowdsourcing in the low redundancy situation

Qianli Xing*, Weiliang Zhao, Jian Yang, Jia Wu, Qi Wang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

1 Citation (Scopus)

Abstract

A redundancy-based strategy is widely employed by assigning each task to multiple workers and then inferring the correct answer (called truth) for each task in crowdsourcing. Most existing truth inference methods are designed for the situation with a fairly big number of answers for each task (referred to as high redundancy). However, the high redundancy unavoidably leads to a high cost. In this work, we propose a novel truth inference approach called TWLR based on worker representations for the situation with a small number of answers for each task (referred to as low redundancy). We develop a deep model to learn the representations of workers considering both answers and worker-task relations. For each task, we identify the worker with the highest quality, and select his/her answer as the predicted answer. To the best of our knowledge, this is the first work to perform truth inference by utilizing deep learning techniques to deal with the low redundancy situation in crowdsourcing. We have conducted a set of experiments against 7 real-world datasets to show the accuracy improvement of our truth inference approach by comparing with 11 baseline methods.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Conference on Web Services, ICWS 2021
EditorsCarl Chang, Ernesto Damiani, Jing Fan, Parisa Ghodous, Michael Maximilien, Zhongjie Wang, Robert Ward, Jia Zhang
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages79-86
Number of pages8
ISBN (Electronic)9781665416818
DOIs
Publication statusPublished - 2021
Event14th IEEE International Conference on Web Services, ICWS 2021 - Virtual, Online, United States
Duration: 5 Sept 202111 Sept 2021

Publication series

NameProceedings - 2021 IEEE International Conference on Web Services, ICWS 2021

Conference

Conference14th IEEE International Conference on Web Services, ICWS 2021
Country/TerritoryUnited States
CityVirtual, Online
Period5/09/2111/09/21

Keywords

  • Crowdsourcing
  • Low-Redundancy
  • Network Representations
  • Truth Inference
  • Worker Similarity

Fingerprint

Dive into the research topics of 'TWLR: a novel truth inference approach based on worker representations for crowdsourcing in the low redundancy situation'. Together they form a unique fingerprint.

Cite this