HiXEval: Highlighting XML retrieval evaluation

Pehcevski, J and Thom, J 2006, 'HiXEval: Highlighting XML retrieval evaluation', in N. Fuhr, M. Lalmas, S. Malik & G. Kazai (ed.) Revised selected papers for the 4th international workshop of the initiative for the evaluation of XML retrieval, INEX 2005, Schloss Dagstuhl, Germany, 28 June 2006.


Document type: Conference Paper
Collection: Conference Papers

Title HiXEval: Highlighting XML retrieval evaluation
Author(s) Pehcevski, J
Thom, J
Year 2006
Conference name Advances in XML information retrieval and evaluation
Conference location Schloss Dagstuhl, Germany
Conference dates 28 June 2006
Proceedings title Revised selected papers for the 4th international workshop of the initiative for the evaluation of XML retrieval, INEX 2005
Editor(s) N. Fuhr, M. Lalmas, S. Malik & G. Kazai
Publisher Springer
Place of publication Heidelberg, Germany
Abstract This paper describes our proposal for an evaluation metric for XML retrieval that is solely based on the highlighted text. We support our decision of ignoring the exhaustivity dimension by undertaking a critical investigation of the two INEX 2005 relevance dimensions. We present a fine grained empirical analysis of the level of assessor agreement of the five topics double-judged at INEX 2005, and show that the agreement is higher for specificity than for exhaustivity. We use the proposed metric to evaluate the INEX 2005 runs for each retrieval strategy of the CO and CAS retrieval tasks. A correlation analysis of the rank orderings obtained by the new metric and two XCG metrics shows that the orderings are strongly correlated, which demonstrates the usefulness of the proposed metric for evaluation of XML retrieval performance.
Subjects Business Information Management (incl. Records, Knowledge and Information Management, and Intelligence)
DOI - identifier 10.1007/11766278
Copyright notice © Springer-Verlag Berlin Heidelberg 2006
Versions
Version Filter Type
Altmetric details:
Access Statistics: 176 Abstract Views  -  Detailed Statistics
Created: Wed, 08 Apr 2009, 09:42:32 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us