SCiMet: Stable, sCalable and reliable Metric-based framework for quality assessment in collaborative content generation systems

In collaborative content generation (CCG), such as publishing scientific articles, a group of contributors collaboratively generates artifacts available through a venue. The main concern in such systems is the quality. A remarkable range of research considers quality metrics partially when dealing with the quality of artifacts, contributors, and venues. However, such approaches have several drawbacks. One of the most notable ones is that they are not comprehensive in terms of the metrics to evaluate all entities, including artifacts, contributors, and venues. Also, they are vulnerable to potential attacks.

In this paper, we propose a novel iterative definition in which the quality of artifacts, collaborators, and venues are defined interconnectedly. In our framework, the quality of an artifact is defined based on the quality of its contributors, venue, references, and citations. The quality of a contributor is defined based on the quality of his artifacts, collaborators, and the venues. Quality of a venue is defined based on both quality of artifacts and contributors. We propose a data model, formulations, and an algorithm for the proposed approach. We also compare the robustness of our approach against malicious manipulations with two well-known related approaches. The comparison results show the superiority of our method over other related approaches.