Towards a Basic Principle for Ranking Effectiveness Prediction without Human Assessments: A Preliminary Study
CIKM Workshop on 1st International Workshop on GeneraLization in InformAtion REtrieval (GLARE'18), 2018
We present in this paper a preliminary study about the Observational Information Linearity (OIL) assumption as a basic principle that explains the accuracy of pseudo relevance assessments in the IR literature. The proposed model predicts the effectiveness drop curve along positions in a single ranking, the relative performance of two rankings, and converges into the traditional pseudo assessment method when considering multiple rankings and statistical independence between them.
@InProceedings{amigo2018towards,
author = {Amig{\'o}, Enrique and Mizzaro, Stefano and Spina, Damiano},
title = {Towards a Basic Principle for Ranking Effectiveness Prediction without Human Assessments: A Preliminary Study},
booktitle = {CIKM Workshop on 1st International Workshop on GeneraLization in InformAtion REtrieval (GLARE'18)},
year = {2018}
}