Towards Measuring Sensitivity of Psychometrics in Crowdsourcing Tasks: Engaging with Fact-checked Content Online

Abstract

This study attempts to explore the relationships between psychometric test outcomes and how participants rate the truthfulness of online news articles. We seek to be able to help in the identification of misinformation. We attempt to explore the relationship between political orientation and several other psychometrics. We then ask the user to rate social media article as well as fact-checked article before finding the relationship between the variables and the difference in rating of each articles. We faced an issue in aggregating the result of each participant as well as computing the distance in ordered scale (magnitude given values in limited range).

Publication
Proceedings of the CSCW 2023 Workshop on Undertanding and Mitigating Cognitive Biases in Human-AI Collaboration