Estimating cognitive load from speech gathered in a complex real-life training exercise

Vukovic, M, Sethu, V, Parker, J, Cavedon, L, Lech, M and Thangarajah, J 2019, 'Estimating cognitive load from speech gathered in a complex real-life training exercise', International Journal of Human-Computer Studies, vol. 124, pp. 116-133.

Document type: Journal Article
Collection: Journal Articles

Title Estimating cognitive load from speech gathered in a complex real-life training exercise
Author(s) Vukovic, M
Sethu, V
Parker, J
Cavedon, L
Lech, M
Thangarajah, J
Year 2019
Journal name International Journal of Human-Computer Studies
Volume number 124
Start page 116
End page 133
Total pages 18
Publisher Elsevier
Abstract Speech-enabled applications are becoming prevalent, providing opportunities for real-time detection of speaker characteristics. Estimation of cognitive load from speech is one type of speaker characteristic that can provide insight into the human state in complex, highly dynamic human-machine teaming scenarios and be used to adapt interaction with the user to their current cognitive state. Cognitive load estimation from speech experiments are typically performed on speech gathered in laboratory settings. By contrast, this research is performed on a real-life dataset that was not created for the purpose of cognitive load assessment. Speech was extracted from recordings of a military simulation exercise in which air battle managers communicated with pilots flying simulated aircraft. This paper assesses whether cognitive load can be estimated from speech self-labelled by exercise participants and collected in a realistic setting, and examines how well cognitive load estimation methods translate from the laboratory setting to the real-world. Analysis suggests that participants' self-assessment of workload at periodic intervals can be used to label speech to create 2-class cognitive load classifiers. The analysis also shows that including some target speaker speech in speaker independent training data results in higher classification accuracy than when classifiers are built solely from speaker dependent data.
Subject Pattern Recognition and Data Mining
Computer-Human Interaction
Keyword(s) Cognitive load measurement
Speech interfaces
DOI - identifier 10.1016/j.ijhcs.2018.12.003
Copyright notice Crown Copyright © 2018 Published by Elsevier Ltd. All rights reserved.
ISSN 1071-5819
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 2 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 0 times in Scopus Article
Altmetric details:
Access Statistics: 75 Abstract Views  -  Detailed Statistics
Created: Thu, 31 Jan 2019, 11:26:00 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us