2PTS: A two-phase task scheduling algorithm for MapReduce

Byungnam Lim; Yeeun Shim; Yon Dohn Chung

doi:10.1587/transinf.2016EDL8075

2PTS: A two-phase task scheduling algorithm for MapReduce

Byungnam Lim, Yeeun Shim, Yon Dohn Chung

Department of Computer Science and Engineering

Research output: Contribution to journal › Article › peer-review

Abstract

For an efficient processing of large data in a distributed system, Hadoop MapReduce performs task scheduling such that tasks are distributed with consideration of the data locality. The data locality, however, is limitedly exploited, since it is pursued one node at a time basis without considering the global optimality. In this paper, we propose a novel task scheduling algorithm that globally considers the data locality. Through experiments, we show our algorithm improves the performance of MapReduce in various situations.

Original language	English
Pages (from-to)	2377-2380
Number of pages	4
Journal	IEICE Transactions on Information and Systems
Volume	E99D
Issue number	9
DOIs	https://doi.org/10.1587/transinf.2016EDL8075
Publication status	Published - 2016 Sept

Keywords

Data locality
MapReduce
Task scheduling algorithm

ASJC Scopus subject areas

Software
Hardware and Architecture
Computer Vision and Pattern Recognition
Electrical and Electronic Engineering
Artificial Intelligence

Access to Document

10.1587/transinf.2016EDL8075

Cite this

@article{cb6109035fd24acaa7ebb8dc54813e4d,

title = "2PTS: A two-phase task scheduling algorithm for MapReduce",

abstract = "For an efficient processing of large data in a distributed system, Hadoop MapReduce performs task scheduling such that tasks are distributed with consideration of the data locality. The data locality, however, is limitedly exploited, since it is pursued one node at a time basis without considering the global optimality. In this paper, we propose a novel task scheduling algorithm that globally considers the data locality. Through experiments, we show our algorithm improves the performance of MapReduce in various situations.",

keywords = "Data locality, MapReduce, Task scheduling algorithm",

author = "Byungnam Lim and Yeeun Shim and Chung, {Yon Dohn}",

note = "Funding Information: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. NRF-2014R1A2A1A11053657). Publisher Copyright: Copyright {\textcopyright} 2016 The Institute of Electronics, Information and Communication Engineers.",

year = "2016",

month = sep,

doi = "10.1587/transinf.2016EDL8075",

language = "English",

volume = "E99D",

pages = "2377--2380",

journal = "IEICE Transactions on Information and Systems",

issn = "0916-8532",

publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",

number = "9",

}

TY - JOUR

T1 - 2PTS

T2 - A two-phase task scheduling algorithm for MapReduce

AU - Lim, Byungnam

AU - Shim, Yeeun

AU - Chung, Yon Dohn

N1 - Funding Information: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. NRF-2014R1A2A1A11053657). Publisher Copyright: Copyright © 2016 The Institute of Electronics, Information and Communication Engineers.

PY - 2016/9

Y1 - 2016/9

N2 - For an efficient processing of large data in a distributed system, Hadoop MapReduce performs task scheduling such that tasks are distributed with consideration of the data locality. The data locality, however, is limitedly exploited, since it is pursued one node at a time basis without considering the global optimality. In this paper, we propose a novel task scheduling algorithm that globally considers the data locality. Through experiments, we show our algorithm improves the performance of MapReduce in various situations.

AB - For an efficient processing of large data in a distributed system, Hadoop MapReduce performs task scheduling such that tasks are distributed with consideration of the data locality. The data locality, however, is limitedly exploited, since it is pursued one node at a time basis without considering the global optimality. In this paper, we propose a novel task scheduling algorithm that globally considers the data locality. Through experiments, we show our algorithm improves the performance of MapReduce in various situations.

KW - Data locality

KW - MapReduce

KW - Task scheduling algorithm

UR - http://www.scopus.com/inward/record.url?scp=84984916905&partnerID=8YFLogxK

U2 - 10.1587/transinf.2016EDL8075

DO - 10.1587/transinf.2016EDL8075

M3 - Article

AN - SCOPUS:84984916905

SN - 0916-8532

VL - E99D

SP - 2377

EP - 2380

JO - IEICE Transactions on Information and Systems

JF - IEICE Transactions on Information and Systems

IS - 9

ER -

2PTS: A two-phase task scheduling algorithm for MapReduce

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this