|Title||Towards Optimal Resource Provisioning for Running MapReduce Programs in Public Clouds|
|Publication Type||Conference Paper|
|Year of Publication||2011|
|Authors||Fengguang Tian, Keke Chen|
|Conference Name||IEEE Conference on Cloud Computing|
|Keywords||Cloud Computing, MapReduce, Performance Modeling, Resource Provisioning|
Running MapReduce programs in the public cloud introduces the important problem: how to optimize resource provisioning to minimize the ﬁnancial charge for a speciﬁc job? In this paper, we study the whole process of MapReduce processing and build up a cost function that explicitly models the relationship between the amount of input data, the available system resources (Map and Reduce slots), and the complexity of the Reduce function for the target MapReduce job. The model parameters can be learned from test runs with a small number of nodes. Based on this cost model, we can solve a number of decision problems, such as the optimal amount of resources that can minimize the ﬁnancial cost with a time deadline or minimize the time under certain ﬁnancial budget. Experimental results show that this cost model performs well on tested MapReduce programs.
|Full Text|| |
Fengguang Tian, Keke Chen, 'Towards Optimal Resource Provisioning for Running MapReduce Programs in Public Clouds', IEEE Conference on Cloud Computing, Washington DC, 2011.