Cost and benefit analysis of mediated enterprise search

Wu, M, Thom, J, Turpin, A and Wilkinson, R 2009, 'Cost and benefit analysis of mediated enterprise search', in Richard Furuta (ed.) Proceedings of the Proceedings of Joint Conference on Digital Libraries, Austin, Texas,, 15 - 19 June 2009, pp. 267-276.


Document type: Conference Paper
Collection: Conference Papers

Title Cost and benefit analysis of mediated enterprise search
Author(s) Wu, M
Thom, J
Turpin, A
Wilkinson, R
Year 2009
Conference name JCDL 09
Conference location Austin, Texas,
Conference dates 15 - 19 June 2009
Proceedings title Proceedings of the Proceedings of Joint Conference on Digital Libraries
Editor(s) Richard Furuta
Publisher ACM
Place of publication New York
Start page 267
End page 276
Total pages 10
Abstract The utility of an enterprise search system is determined by three key players: the information retrieval (IR) system (the search engine), the enterprise users, and the service provider who delivers the tailored IR service to its designated enterprise users. Currently, evaluations of enterprise search have been focused largely on the IR system effectiveness and efficiency, only a relatively small amount of effort on the user's involvement, and hardly any effort on the service provider's role. This paper will investigate the role of the service provider. We propose a method that evaluates the cost and benefit for a service provider of using a mediated search engine - in particular, where domain experts intervene on the ranking of the search results from a search engine. We test our cost and benefit evaluation method in a case study and conduct user experiments to demonstrate it. Our study shows that: 1) by making use of domain experts' relevance assessments in search result ranking, the precision and the discount cumulated gain of ranked lists have been improved significantly (144% and 40% respectively); 2) the service provider gains substantial return on investment and higher search success rate by investing in domain experts' relevance assessments; and 3) the cost and benefit evaluation also indicates the type of queries to be selected from a query log for evaluating an enterprise search engine. 2-s2.0-70450227238
Subjects Information Retrieval and Web Search
Keyword(s) Information Retrieval
Enterprise Search
Relevance Feedback
Evaluation
Mediated Search
Cost and Benefit Analysis
ISBN 9781605583228/09/06
Versions
Version Filter Type
Citation counts: Scopus Citation Count Cited 6 times in Scopus Article | Citations
Access Statistics: 111 Abstract Views  -  Detailed Statistics
Created: Mon, 20 Jun 2011, 07:36:00 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us