A comparison of document-at-a-time and score-at-a-time query evaluation

Crane, M, Culpepper, J, Lin, J, Mackenzie, J and Trotman, A 2017, 'A comparison of document-at-a-time and score-at-a-time query evaluation', in M. de Rijke and M. Shokouhi (ed.) Proceedings of the 10th International Conference on Web Search and Data Mining (WSDM 2017), Cambridge, United Kingdom, 6-10 February 2017, pp. 201-210.


Document type: Conference Paper
Collection: Conference Papers

Title A comparison of document-at-a-time and score-at-a-time query evaluation
Author(s) Crane, M
Culpepper, J
Lin, J
Mackenzie, J
Trotman, A
Year 2017
Conference name WSDM 2017: 10th ACM International Conference on Web Search and Data Mining
Conference location Cambridge, United Kingdom
Conference dates 6-10 February 2017
Proceedings title Proceedings of the 10th International Conference on Web Search and Data Mining (WSDM 2017)
Editor(s) M. de Rijke and M. Shokouhi
Publisher Association for Computing Machinery
Place of publication New York, United States
Start page 201
End page 210
Total pages 10
Abstract We present an empirical comparison between document-at-a-time (DAAT) and score-at-a-time (SAATt) document ranking strategies within a common framework. Although both strategies have been extensively explored, the literature lacks a fair, direct comparison: such a study has been difficult due to vastly different query evaluation mechanics and index organizations. Our work controls for score quantization, document processing, compression, implementation language, implementation effort, and a number of details, arriving at an empirical evaluation that fairly characterizes the performance of three specific techniques: WAND (DAAT), BMW (DAAT), and JASS (SAAT). Experiments reveal a number of interesting findings. The performance gap between WAND and BMW is not as clear as the literature suggests, and both methods are susceptible to tail queries that may take orders of magnitude longer than the median query to execute. Surprisingly, approximate query evaluation in WAND and BMW does not significantly reduce the risk of these tail queries. Overall, JASS is slightly slower than either WAND or BMW, but exhibits much lower variance in query latencies and is much less susceptible to tail query effects. Furthermore, JASS query latency is not particularly sensitive to the retrieval depth, making it an appealing solution for performance-sensitive applications where bounds on query latencies are desirable
Subjects Information Retrieval and Web Search
Data Structures
Keyword(s) Efficiency
Experimentation
Measurement
DOI - identifier 10.1145/3018661.3018726
Copyright notice © 2017 The Authors
ISBN 9781450346757
Versions
Version Filter Type
Citation counts: Scopus Citation Count Cited 3 times in Scopus Article | Citations
Altmetric details:
Access Statistics: 208 Abstract Views  -  Detailed Statistics
Created: Wed, 07 Jun 2017, 08:19:00 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us