How Many Truth Levels? Six? One Hundred? Even more? Validating Truthfulness of Statements via Crowdsourcing

Abstract

We report on collecting truthfulness values (i) by means of crowd-sourcing and (ii) using fine-grained scales. In our experiment we collect truthfulness values using a bounded and discrete scale with 100 levels as well as a magnitude estimation scale, which is unbounded, continuous and has infinite amount of levels. We compare the two scales and discuss the agreement with a groundtruth provided by experts on a six-level scale.

Publication
RDSM'18 CIKM Workshop on Rumours and Deception in Social Media