Assessing the cognitive complexity of information needs

Moffat, A, Bailey, P, Scholer, F and Thomas, P 2014, 'Assessing the cognitive complexity of information needs', in J. Shane Culpepper, Laurence Park, and Guido Zuccon (ed.) Proceedings of the 19th Australasian Document Computing Symposium (ADCS 2014), Melbourne, Australia, 27 - 28 November 2014, pp. 97-100.


Document type: Conference Paper
Collection: Conference Papers

Title Assessing the cognitive complexity of information needs
Author(s) Moffat, A
Bailey, P
Scholer, F
Thomas, P
Year 2014
Conference name ADCS 2014
Conference location Melbourne, Australia
Conference dates 27 - 28 November 2014
Proceedings title Proceedings of the 19th Australasian Document Computing Symposium (ADCS 2014)
Editor(s) J. Shane Culpepper, Laurence Park, and Guido Zuccon
Publisher Association for Computing Machinery
Place of publication New York, United States
Start page 97
End page 100
Total pages 4
Abstract Information retrieval systems can be evaluated in laboratory settings through the use of user studies, and through the use of test collections and effectiveness metrics. In a larger investigation we are exploring the extent to which individual user differences and behaviours can affect the scores generated by a retrieval system. Our objective in the first phase of that project is to define information need statements corresponding to a range of TREC search tasks, and to categorise those statements in terms of task complexity. The goal is to reach a position from which we can determine whether user actions while searching are influenced by the way the information need is expressed, and by the fundamental nature of the information need. We describe the process used to create information need statements, and then report inter- and intra-assessor agreements across four annotators. We conclude that assessing the relative cognitive complexity of tasks is a complex activity, even for experienced annotators.
Subjects Information Retrieval and Web Search
DOI - identifier 10.1145/2682862.2682874
Copyright notice © 2014 by the Association for Computing Machinery, Inc.
ISBN 9781450330008
Versions
Version Filter Type
Altmetric details:
Access Statistics: 146 Abstract Views  -  Detailed Statistics
Created: Wed, 17 Jun 2015, 08:06:00 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us