A QoS-Aware Controller for Apache Storm

Hoseiny Farahabady, R, Dehghani Samani, H, Wang, Y, Zomaya, A and Tari, Z 2016, 'A QoS-Aware Controller for Apache Storm', in Alessandro Pellegrini, Aris Gkoulalas-Divanis, Pierangelo Di Sanzo and Dimiter R. Avresky (ed.) Proceedings of the15th IEEE International Symposium on Network Computing and Applications (NCA 2016), Cambridge, MA, United States, 31 October - 2 November 2016, pp. 334-342.


Document type: Conference Paper
Collection: Conference Papers

Title A QoS-Aware Controller for Apache Storm
Author(s) Hoseiny Farahabady, R
Dehghani Samani, H
Wang, Y
Zomaya, A
Tari, Z
Year 2016
Conference name NCA 2016
Conference location Cambridge, MA, United States
Conference dates 31 October - 2 November 2016
Proceedings title Proceedings of the15th IEEE International Symposium on Network Computing and Applications (NCA 2016)
Editor(s) Alessandro Pellegrini, Aris Gkoulalas-Divanis, Pierangelo Di Sanzo and Dimiter R. Avresky
Publisher IEEE
Place of publication United States
Start page 334
End page 342
Total pages 9
Abstract Apache Storm has recently emerged as an attractive fault-tolerant open-source distributed data processing platform that has been chosen by many industry leaders to develop realtime applications for processing a huge amount of data in a scalable manner. A key aspect to achieve the best performance in this system lies on the design of an efficient scheduler for component execution, called topology, on the available computing resources. In response to workload fluctuations, we propose an advanced scheduler for Apache Storm that provides improved performance with highly dynamic behavior. While enforcing the required Quality-of-Service (QoS) of individual data streams, the controller allocates computing resources based on decisions that consider the future states of non-controllable disturbance parameters, e.g. arriving rate of tuples or resource utilization in each worker node. The performance evaluation is carried out by comparing the proposed solution with two well-known alternatives, namely the Storm's default scheduler and the best effort approach (i.e. the heuristic that is based on the first-fit decreasing approximation algorithm). Experimental results clearly show that the proposed controller increases the overall resource utilization by 31% on average compared to the two others solutions, without significant negative impact on the QoS enforcement level.
Subjects Distributed and Grid Systems
Keyword(s) Streaming Data Processing
Apache Storm
Model Predictive Control
Resource Allocation/Scheduling
Copyright notice © 2016 IEEE
ISBN 9781509032167
Versions
Version Filter Type
Access Statistics: 118 Abstract Views  -  Detailed Statistics
Created: Wed, 11 Jan 2017, 12:29:00 EST by Catalyst Administrator
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us