Preaching to the ChoIR: Lessons IR Should Share with AI

Abstract

The field of Information Retrieval (IR) changed profoundly at the end of the 1990s with the rise of Web Search, and there are parallels with developments in Artificial Intelligence (AI) happening today with the advent of ChatGPT, Large Language Models, and Generative AI. We acknowledge that there are clear differences between IR and AI. For example, IR is a much smaller field, and new problems arise, like data contamination that may affect benchmark-based evaluation of AI systems. But looking through the lens of an IR researcher, there are many striking similarities between the two fields of IR (25 years ago) and AI (today), and many topics appearing in discussions in AI resemble those of 25 years ago in IR: benchmark reliability and robust evaluation, reproducibility of results for non-public models, privacy and copyright issues, efficiency and scalability, etc. In this paper, we discuss similarities and differences between IR and AI and then derive some lessons learned in the field of IR as a list of recommendations – urging the IR community to reflect on, discuss, and convey these lessons to the AI field. We believe that a joint community effort by all IR researchers is both necessary and dutiful to obtain a fruitful discussion with the AI community.

Publication
Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR)