A Living Lab Study of Query Amendment in Job Search

  • Bahar Salehi
  • Damiano Spina
  • Alistair Moffat
  • Sargol Sadeghi
  • Falk Scholer
  • Timothy Baldwin
  • Lawrence Cavedon
  • Mark Sanderson
  • Wilson Wong
  • Justin Zobel
Proceedings of SIGIR'18 (To Appear), 2018

Errors in formulation of queries made by users can lead to poor search results pages. We performed a living lab study using online A/B testing to measure the degree of improvement achieved with a query amendment technique when applied to a commercial job search engine. Of particular interest in this case study is a clear 'success' signal, namely, the number of job applications lodged by a user as a result of querying the service. A set of 276 queries was identified for amendment in four different categories through the use of word embeddings, with large gains in conversion rates being attained in all four of those categories. Our analysis of query reformulations also provides a better understanding of user satisfaction in the case of problematic queries (ones with fewer results than fill a single page) by observing that users tend to reformulate rewritten queries less.

@InProceedings{salehi2018living,
  author={Salehi, Bahar  and Spina, Damiano and Moffat, Alistair and Sadeghi, Sargol and Scholer, Falk 
      and Baldwin, Timothy and Cavedon, Lawrence and Sanderson, Mark and Wong, Wilson and Zobel, Justin},
  title={A Living Lab Study of Query Amendment in Job Search},
  booktitle={Proceedings of SIGIR'18},
  year={2018}}
Damiano Spina
Assign a menu in the Right Menu options.