Towards a Basic Principle for Ranking Effectiveness Prediction without Human Assessments: A Preliminary Study

  • Enrique Amigó
  • Stefano Mizzaro
  • Damiano Spina
CIKM Workshop on 1st International Workshop on GeneraLization in InformAtion REtrieval (GLARE'18), 2018

We present in this paper a preliminary study about the Observational Information Linearity (OIL) assumption as a basic principle that explains the accuracy of pseudo relevance assessments in the IR literature. The proposed model predicts the effectiveness drop curve along positions in a single ranking, the relative performance of two rankings, and converges into the traditional pseudo assessment method when considering multiple rankings and statistical independence between them.

@InProceedings{amigo2018towards,
author = {Amig{\'o}, Enrique and Mizzaro, Stefano and Spina, Damiano},
title = {Towards a Basic Principle for Ranking Effectiveness Prediction without Human Assessments: A Preliminary Study},
booktitle = {CIKM Workshop on 1st International Workshop on GeneraLization in InformAtion REtrieval (GLARE'18)},
year = {2018}
}
Damiano Spina