Local Resource Consumption Shaping: A Case for MapReduce

P. Lu; Y. C. Lee; T. Ryan; V. Gramoli; A. Y. Zomaya

doi:10.1016/B978-0-12-805394-2.00008-8

Local Resource Consumption Shaping: A Case for MapReduce

P. Lu^*, Y. C. Lee, T. Ryan, V. Gramoli, A. Y. Zomaya

^*Corresponding author for this work

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

Abstract

Today, resource capacity is no longer an issue for running large-scale distributed systems, such as MapReduce. As a result, it is often the case that resources are provisioned, for such systems, at the level of peak loads. This overprovisioning has become a serious efficiency issue in cloud data centers with poor resource utilization. The improvement of resource utilization can be achieved by concurrently running tasks sharing a common set of resources. However, many distributed systems spawn a large number of tasks that exhibit similar resource consumption patterns causing much performance interference/degradation that is primarily due to fair resource sharing. In this study, we consider the problem of "local resource consumption shaping" - an alternative to fair resource sharing - at the local node/core level.

Original language	English
Title of host publication	Big Data: Principles and Paradigms
Editors	Rajkumar Buyya, Rodrigo N. Calheiros, Amir Vahid Dastjredi
Place of Publication	Cambridge, MA, USA
Publisher	Morgan Kaufmann
Pages	189-214
Number of pages	26
ISBN (Electronic)	9780128093467
ISBN (Print)	9780128053942
DOIs	https://doi.org/10.1016/B978-0-12-805394-2.00008-8
Publication status	Published - 3 Jun 2016

Access to Document

10.1016/B978-0-12-805394-2.00008-8

Cite this

@inbook{3d215a4a6ba448ccbcf09de2fb239b66,

title = "Local Resource Consumption Shaping: A Case for MapReduce",

abstract = "Today, resource capacity is no longer an issue for running large-scale distributed systems, such as MapReduce. As a result, it is often the case that resources are provisioned, for such systems, at the level of peak loads. This overprovisioning has become a serious efficiency issue in cloud data centers with poor resource utilization. The improvement of resource utilization can be achieved by concurrently running tasks sharing a common set of resources. However, many distributed systems spawn a large number of tasks that exhibit similar resource consumption patterns causing much performance interference/degradation that is primarily due to fair resource sharing. In this study, we consider the problem of {"}local resource consumption shaping{"} - an alternative to fair resource sharing - at the local node/core level.",

author = "P. Lu and Lee, {Y. C.} and T. Ryan and V. Gramoli and Zomaya, {A. Y.}",

year = "2016",

month = jun,

day = "3",

doi = "10.1016/B978-0-12-805394-2.00008-8",

language = "English",

isbn = "9780128053942",

pages = "189--214",

editor = "Rajkumar Buyya and Calheiros, {Rodrigo N.} and Dastjredi, {Amir Vahid}",

booktitle = "Big Data: Principles and Paradigms",

publisher = "Morgan Kaufmann",

}

TY - CHAP

T1 - Local Resource Consumption Shaping

T2 - A Case for MapReduce

AU - Lu, P.

AU - Lee, Y. C.

AU - Ryan, T.

AU - Gramoli, V.

AU - Zomaya, A. Y.

PY - 2016/6/3

Y1 - 2016/6/3

N2 - Today, resource capacity is no longer an issue for running large-scale distributed systems, such as MapReduce. As a result, it is often the case that resources are provisioned, for such systems, at the level of peak loads. This overprovisioning has become a serious efficiency issue in cloud data centers with poor resource utilization. The improvement of resource utilization can be achieved by concurrently running tasks sharing a common set of resources. However, many distributed systems spawn a large number of tasks that exhibit similar resource consumption patterns causing much performance interference/degradation that is primarily due to fair resource sharing. In this study, we consider the problem of "local resource consumption shaping" - an alternative to fair resource sharing - at the local node/core level.

AB - Today, resource capacity is no longer an issue for running large-scale distributed systems, such as MapReduce. As a result, it is often the case that resources are provisioned, for such systems, at the level of peak loads. This overprovisioning has become a serious efficiency issue in cloud data centers with poor resource utilization. The improvement of resource utilization can be achieved by concurrently running tasks sharing a common set of resources. However, many distributed systems spawn a large number of tasks that exhibit similar resource consumption patterns causing much performance interference/degradation that is primarily due to fair resource sharing. In this study, we consider the problem of "local resource consumption shaping" - an alternative to fair resource sharing - at the local node/core level.

UR - http://www.scopus.com/inward/record.url?scp=84988844024&partnerID=8YFLogxK

U2 - 10.1016/B978-0-12-805394-2.00008-8

DO - 10.1016/B978-0-12-805394-2.00008-8

M3 - Chapter

AN - SCOPUS:84988844024

SN - 9780128053942

SP - 189

EP - 214

BT - Big Data: Principles and Paradigms

A2 - Buyya, Rajkumar

A2 - Calheiros, Rodrigo N.

A2 - Dastjredi, Amir Vahid

PB - Morgan Kaufmann

CY - Cambridge, MA, USA

ER -

Local Resource Consumption Shaping: A Case for MapReduce

Abstract

Access to Document

Other files and links

Fingerprint

Cite this