Dynamic Shard Cutoff Prediction for Selective Search

Mohammad, H, Xu, K, Callan, J and Culpepper, J 2018, 'Dynamic Shard Cutoff Prediction for Selective Search', in The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, United States, 08-12 July 2018, pp. 85-94.


Document type: Conference Paper
Collection: Conference Papers

Title Dynamic Shard Cutoff Prediction for Selective Search
Author(s) Mohammad, H
Xu, K
Callan, J
Culpepper, J
Year 2018
Conference name SIGIR
Conference location Ann Arbor, United States
Conference dates 08-12 July 2018
Proceedings title The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval
Publisher ACM
Place of publication New York, United States
Start page 85
End page 94
Total pages 10
Abstract Selective search architectures use resource selection algorithms such as Rank-S or Taily to rank index shards and determine how many to search for a given query. Most prior research evaluated solutions by their ability to improve efficiency without significantly reducing early-precision metrics such as P@5 and NDCG@10. This paper recasts selective search as an early stage of a multi-stage retrieval architecture, which makes recall-oriented metrics more appropriate. A new algorithm is presented that predicts the number of shards that must be searched for a given query in order to meet recall-oriented goals. Decoupling shard ranking from deciding how many shards to search clarifies efficiency vs. effectiveness trade-offs, and enables them to be optimized independently. Experiments on two corpora demonstrate the value of this approach.
Subjects Data Structures
Information Retrieval and Web Search
Keyword(s) Information Retrieval
DOI - identifier 10.1145/3209978.3210005
Copyright notice © 2018 Association for Computing Machinery
ISBN 9781450356572
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 1 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 0 times in Scopus Article
Altmetric details:
Access Statistics: 15 Abstract Views  -  Detailed Statistics
Created: Thu, 21 Feb 2019, 12:10:00 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us