TWLR: a novel truth inference approach based on worker representations for crowdsourcing in the low redundancy situation

A redundancy-based strategy is widely employed by assigning each task to multiple workers and then inferring the correct answer (called truth) for each task in crowdsourcing. Most existing truth inference methods are designed for the situation with a fairly big number of answers for each task (referred to as high redundancy). However, the high redundancy unavoidably leads to a high cost. In this work, we propose a novel truth inference approach called TWLR based on worker representations for the situation with a small number of answers for each task (referred to as low redundancy). We develop a deep model to learn the representations of workers considering both answers and worker-task relations. For each task, we identify the worker with the highest quality, and select his/her answer as the predicted answer. To the best of our knowledge, this is the first work to perform truth inference by utilizing deep learning techniques to deal with the low redundancy situation in crowdsourcing. We have conducted a set of experiments against 7 real-world datasets to show the accuracy improvement of our truth inference approach by comparing with 11 baseline methods.