Improving HDFS performance using local caching system

Sang Deok Yoon; In Yang Jung; Ki Hyun Kim; Chang Sung Jeong

doi:10.1109/FGCT.2013.6767200

Improving HDFS performance using local caching system

Sang Deok Yoon, In Yang Jung, Ki Hyun Kim, Chang Sung Jeong

School of Electrical Engineering

Research output: Contribution to conference › Paper › peer-review

3 Citations (Scopus)

Abstract

I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

Original language	English
Pages	153-156
Number of pages	4
DOIs	https://doi.org/10.1109/FGCT.2013.6767200
Publication status	Published - 2013
Event	2nd International Conference on Future Generation Communication Technologies, FGCT 2013 - London, United Kingdom Duration: 2013 Nov 12 → 2013 Nov 14

Other

Other	2nd International Conference on Future Generation Communication Technologies, FGCT 2013
Country/Territory	United Kingdom
City	London
Period	13/11/12 → 13/11/14

Keywords

HDFS
cache
distributed file system
hadoop
locality

ASJC Scopus subject areas

Computer Networks and Communications

Access to Document

10.1109/FGCT.2013.6767200

Cite this

@conference{5a60ef67721643928769c18f731c31f6,

title = "Improving HDFS performance using local caching system",

abstract = "I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.",

keywords = "HDFS, cache, distributed file system, hadoop, locality",

author = "Yoon, {Sang Deok} and Jung, {In Yang} and Kim, {Ki Hyun} and Jeong, {Chang Sung}",

year = "2013",

doi = "10.1109/FGCT.2013.6767200",

language = "English",

pages = "153--156",

note = "2nd International Conference on Future Generation Communication Technologies, FGCT 2013 ; Conference date: 12-11-2013 Through 14-11-2013",

}

TY - CONF

T1 - Improving HDFS performance using local caching system

AU - Yoon, Sang Deok

AU - Jung, In Yang

AU - Kim, Ki Hyun

AU - Jeong, Chang Sung

PY - 2013

Y1 - 2013

N2 - I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

AB - I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

KW - HDFS

KW - cache

KW - distributed file system

KW - hadoop

KW - locality

UR - http://www.scopus.com/inward/record.url?scp=84898969542&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898969542&partnerID=8YFLogxK

U2 - 10.1109/FGCT.2013.6767200

DO - 10.1109/FGCT.2013.6767200

M3 - Paper

AN - SCOPUS:84898969542

SP - 153

EP - 156

T2 - 2nd International Conference on Future Generation Communication Technologies, FGCT 2013

Y2 - 12 November 2013 through 14 November 2013

ER -

Improving HDFS performance using local caching system

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Cite this