INSTRUMENTATION OF COMPUTATIONAL INTENSIVE TASK TO ELIMINATE OUTLIERS DURING PARALLEL PROCESSING
Keywords:
Instrumentation, Outliers Clustering, Scheduler, Parallel ProcessingAbstract
In clusters with heterogeneous systems the progress of data intensive task is estimated using mapreduce framework. But it is not suitable for computational intensive task due to biased estimation of task progress, traditional frameworks cannot timely cut off outliers and therefore largely prolong execution time. Here proposed new framework No Outlier with the instrumentation and outlier clustering techniques for identification of outliers. Since dynamic instrumentation is more precise than static instrumentation, the exact outlier can be identified at runtime and speculative task execution is taken place with average CPU usage.
References
M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica, “Improving mapreduce performance in heterogeneous environments,” in Proceedings of the 8th USENIX conference on Operating systems design and implementation, ser. OSDI’08. Berkeley, CA, USA: USENIX Association, 2008, pp. 29–42.
G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris, “Reining in the outliers in mapreduce clusters using mantri,” in Proceedings of the 9th USENIX conference on Operating systems design and implementation, ser. OSDI’10. Berkeley, CA, USA: USENIX Association, 2010, pp. 1–16.
I. Ahmad and Y. kwong Kwok, “On exploiting task duplication in parallel program scheduling,” IEEE Transactions on Parallel and Distributed Systems, vol. 9, pp. 872–892, 1998.
E. B. Nightingale, P. M. Chen, and J. Flinn, “Speculative execution in a distributed file system,” ACM Trans. Comput. Syst., vol. 24, no. 4, pp. 361–392, Nov. 2006.
L. Carrington, A. Snavely, and N. Wolter, “A performance prediction framework for scientific applications,” Future Gener. Comput. Syst., vol. 22, no. 3, pp. 336–346, Feb. 2006.
J. Dean and S. Ghemawat, “Mapreduce: Simplied data processing on large clusters,” in Operating Systems Design and Implementation, 2004, pp. 137–150.
S. N. Srirama, P. Jakovits, and E. Vainikko, “Adapting scientific computing problems to clouds using mapreduce,” Future Gener. Comput. Syst., vol. 28, no. 1, pp. 184–192, Jan. 2012.
NO2: Speeding Up Parallel Processing of Massive Compute-Intensive Tasks, Yongwei Wu, Weichao Guo, Jinglei Ren, Xun Zhao, and Weimin Zheng, Member, IEEE.
Nilay Narlawar and Ila Naresh Patil, “A Speedy Approach: User-Based Collaborative Filtering with Mapreduce”, International journal of Computer Engineering & Technology (IJCET), Volume 5, Issue 5, 2014, pp. 32 - 39, ISSN Print: 0976 – 6367, ISSN Online: 0976 – 6375.
Priya Deshpande and Sunayna Giroti, “Priority Based Dynamic Adaptive Checkpointing Strategy in Distributed Environment”, International Journal of Computer Engineering & Technology (IJCET), Volume 4, Issue 6, 2013, pp. 378 - 385, ISSN Print: 0976 – 6367, ISSN Online: 0976 – 6375.
Paulo J. G. Lisboa, Huda Naji Nawaf and Wesam S. Bhaya, “Recommendation System Based on Association Rules Applied to Consistent Behavior Over Time”, International Journal of Computer Engineering & Technology (IJCET), Volume 4, Issue 4, 2013, pp. 412 - 421, ISSN Print: 0976 – 6367, ISSN Online: 0976 – 6375.
Downloads
Published
Issue
Section
License
Copyright (c) -1 Chethana V (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
