Home
Scholarly Works
COSHH: A classification and optimization based...
Journal article

COSHH: A classification and optimization based scheduler for heterogeneous Hadoop systems

Abstract

A Hadoop system provides execution and multiplexing of many tasks in a common datacenter. There is a rising demand for sharing Hadoop clusters amongst various users, which leads to increasing system heterogeneity. However, heterogeneity is a neglected issue in most Hadoop schedulers. In this work we design and implement a new Hadoop scheduling system, named COSHH, which considers heterogeneity at both the application and cluster levels. The main objective of COSHH is to improve the mean completion time of jobs. However, as it is concerned with other key Hadoop performance metrics, our proposed scheduler also achieves competitive performance under minimum share satisfaction, fairness and locality metrics with respect to other well-known Hadoop schedulers.

Authors

Rasooli A; Down DG

Journal

Future Generation Computer Systems, Vol. 36, , pp. 1–15

Publisher

Elsevier

Publication Date

July 1, 2014

DOI

10.1016/j.future.2014.01.002

ISSN

0167-739X

Contact the Experts team