Xonar: Profiling-based Job Orderer for Distributed Deep Learning

Changyong Shin; Gyeongsik Yang; Yeonho Yoo; Jeunghwan Lee; Chuck Yoo

doi:10.1109/CLOUD55607.2022.00030

Xonar: Profiling-based Job Orderer for Distributed Deep Learning

Changyong Shin, Gyeongsik Yang, Yeonho Yoo, Jeunghwan Lee, Chuck Yoo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

Deep learning models have a wide spectrum of GPU execution time and memory size. When running distributed training jobs, however, their GPU execution time and memory size have not been taken into account, which leads to the high variance of job completion time (JCT). Moreover, the jobs often run into the GPU out-of-memory (OoM) problem so that the unlucky job has to restart all over. To address the problems, we propose Xonar to profile the deep learning jobs and order them in the queue. The experiments show that Xonar with TensorFlow v1.6 reduces the tail JCT by 44% with the OoM problem eliminated.

Original language	English
Title of host publication	Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022
Editors	Claudio Agostino Ardagna, Nimanthi Atukorala, Rajkumar Buyya, Carl K. Chang, Rong N. Chang, Ernesto Damiani, Gargi Banerjee Dasgupta, Fabrizio Gagliardi, Christoph Hagleitner, Dejan Milojicic, Tuan M Hoang Trong, Robert Ward, Fatos Xhafa, Jia Zhang
Publisher	IEEE Computer Society
Pages	112-114
Number of pages	3
ISBN (Electronic)	9781665481373
DOIs	https://doi.org/10.1109/CLOUD55607.2022.00030
Publication status	Published - 2022
Event	15th IEEE International Conference on Cloud Computing, CLOUD 2022 - Barcelona, Spain Duration: 2021 Jul 10 → 2021 Jul 16

Publication series

Name	IEEE International Conference on Cloud Computing, CLOUD
Volume	2022-July
ISSN (Print)	2159-6182
ISSN (Electronic)	2159-6190

Conference

Conference	15th IEEE International Conference on Cloud Computing, CLOUD 2022
Country/Territory	Spain
City	Barcelona
Period	21/7/10 → 21/7/16

Bibliographical note

Funding Information:
This work was partly supported by Institute of Information & communications Technology Planning & Evaluation funded by the Korea government (Ministry of Science and ICT) (2015-0-00280, (SW Starlab) Next generation cloud infra-software toward the guarantee of performance and security SLA) and by Basic Science Research Program through National Research Foundation of Korea funded by the Ministry of Education (NRF-2021R1A 6A1A13044830). Co-corresponding authors: Chuck Yoo and Gyeongsik Yang.

Publisher Copyright:
© 2022 IEEE.

Keywords

distributed deep learning
GPU cloud
GPU utilization
job completion time
parallel training

ASJC Scopus subject areas

Artificial Intelligence
Information Systems
Software

Access to Document

10.1109/CLOUD55607.2022.00030

Cite this

Shin, C., Yang, G., Yoo, Y., Lee, J., & Yoo, C. (2022). Xonar: Profiling-based Job Orderer for Distributed Deep Learning. In C. A. Ardagna, N. Atukorala, R. Buyya, C. K. Chang, R. N. Chang, E. Damiani, G. B. Dasgupta, F. Gagliardi, C. Hagleitner, D. Milojicic, T. M. H. Trong, R. Ward, F. Xhafa, & J. Zhang (Eds.), Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022 (pp. 112-114). (IEEE International Conference on Cloud Computing, CLOUD; Vol. 2022-July). IEEE Computer Society. https://doi.org/10.1109/CLOUD55607.2022.00030

Xonar: Profiling-based Job Orderer for Distributed Deep Learning. / Shin, Changyong; Yang, Gyeongsik; Yoo, Yeonho et al.
Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022. ed. / Claudio Agostino Ardagna; Nimanthi Atukorala; Rajkumar Buyya; Carl K. Chang; Rong N. Chang; Ernesto Damiani; Gargi Banerjee Dasgupta; Fabrizio Gagliardi; Christoph Hagleitner; Dejan Milojicic; Tuan M Hoang Trong; Robert Ward; Fatos Xhafa; Jia Zhang. IEEE Computer Society, 2022. p. 112-114 (IEEE International Conference on Cloud Computing, CLOUD; Vol. 2022-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Shin, C, Yang, G, Yoo, Y, Lee, J & Yoo, C 2022, Xonar: Profiling-based Job Orderer for Distributed Deep Learning. in CA Ardagna, N Atukorala, R Buyya, CK Chang, RN Chang, E Damiani, GB Dasgupta, F Gagliardi, C Hagleitner, D Milojicic, TMH Trong, R Ward, F Xhafa & J Zhang (eds), Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022. IEEE International Conference on Cloud Computing, CLOUD, vol. 2022-July, IEEE Computer Society, pp. 112-114, 15th IEEE International Conference on Cloud Computing, CLOUD 2022, Barcelona, Spain, 21/7/10. https://doi.org/10.1109/CLOUD55607.2022.00030

Shin C, Yang G, Yoo Y, Lee J, Yoo C. Xonar: Profiling-based Job Orderer for Distributed Deep Learning. In Ardagna CA, Atukorala N, Buyya R, Chang CK, Chang RN, Damiani E, Dasgupta GB, Gagliardi F, Hagleitner C, Milojicic D, Trong TMH, Ward R, Xhafa F, Zhang J, editors, Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022. IEEE Computer Society. 2022. p. 112-114. (IEEE International Conference on Cloud Computing, CLOUD). doi: 10.1109/CLOUD55607.2022.00030

Shin, Changyong ; Yang, Gyeongsik ; Yoo, Yeonho et al. / Xonar : Profiling-based Job Orderer for Distributed Deep Learning. Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022. editor / Claudio Agostino Ardagna ; Nimanthi Atukorala ; Rajkumar Buyya ; Carl K. Chang ; Rong N. Chang ; Ernesto Damiani ; Gargi Banerjee Dasgupta ; Fabrizio Gagliardi ; Christoph Hagleitner ; Dejan Milojicic ; Tuan M Hoang Trong ; Robert Ward ; Fatos Xhafa ; Jia Zhang. IEEE Computer Society, 2022. pp. 112-114 (IEEE International Conference on Cloud Computing, CLOUD).

@inproceedings{297346f87b6948e7a24ed0746f9952fb,

title = "Xonar: Profiling-based Job Orderer for Distributed Deep Learning",

abstract = "Deep learning models have a wide spectrum of GPU execution time and memory size. When running distributed training jobs, however, their GPU execution time and memory size have not been taken into account, which leads to the high variance of job completion time (JCT). Moreover, the jobs often run into the GPU out-of-memory (OoM) problem so that the unlucky job has to restart all over. To address the problems, we propose Xonar to profile the deep learning jobs and order them in the queue. The experiments show that Xonar with TensorFlow v1.6 reduces the tail JCT by 44% with the OoM problem eliminated.",

keywords = "distributed deep learning, GPU cloud, GPU utilization, job completion time, parallel training",

author = "Changyong Shin and Gyeongsik Yang and Yeonho Yoo and Jeunghwan Lee and Chuck Yoo",

note = "Funding Information: This work was partly supported by Institute of Information & communications Technology Planning & Evaluation funded by the Korea government (Ministry of Science and ICT) (2015-0-00280, (SW Starlab) Next generation cloud infra-software toward the guarantee of performance and security SLA) and by Basic Science Research Program through National Research Foundation of Korea funded by the Ministry of Education (NRF-2021R1A 6A1A13044830). Co-corresponding authors: Chuck Yoo and Gyeongsik Yang. Publisher Copyright: {\textcopyright} 2022 IEEE.; 15th IEEE International Conference on Cloud Computing, CLOUD 2022 ; Conference date: 10-07-2021 Through 16-07-2021",

year = "2022",

doi = "10.1109/CLOUD55607.2022.00030",

language = "English",

series = "IEEE International Conference on Cloud Computing, CLOUD",

publisher = "IEEE Computer Society",

pages = "112--114",

editor = "Ardagna, {Claudio Agostino} and Nimanthi Atukorala and Rajkumar Buyya and Chang, {Carl K.} and Chang, {Rong N.} and Ernesto Damiani and Dasgupta, {Gargi Banerjee} and Fabrizio Gagliardi and Christoph Hagleitner and Dejan Milojicic and Trong, {Tuan M Hoang} and Robert Ward and Fatos Xhafa and Jia Zhang",

booktitle = "Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022",

}

TY - GEN

T1 - Xonar

T2 - 15th IEEE International Conference on Cloud Computing, CLOUD 2022

AU - Shin, Changyong

AU - Yang, Gyeongsik

AU - Yoo, Yeonho

AU - Lee, Jeunghwan

AU - Yoo, Chuck

N1 - Funding Information: This work was partly supported by Institute of Information & communications Technology Planning & Evaluation funded by the Korea government (Ministry of Science and ICT) (2015-0-00280, (SW Starlab) Next generation cloud infra-software toward the guarantee of performance and security SLA) and by Basic Science Research Program through National Research Foundation of Korea funded by the Ministry of Education (NRF-2021R1A 6A1A13044830). Co-corresponding authors: Chuck Yoo and Gyeongsik Yang. Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - Deep learning models have a wide spectrum of GPU execution time and memory size. When running distributed training jobs, however, their GPU execution time and memory size have not been taken into account, which leads to the high variance of job completion time (JCT). Moreover, the jobs often run into the GPU out-of-memory (OoM) problem so that the unlucky job has to restart all over. To address the problems, we propose Xonar to profile the deep learning jobs and order them in the queue. The experiments show that Xonar with TensorFlow v1.6 reduces the tail JCT by 44% with the OoM problem eliminated.

AB - Deep learning models have a wide spectrum of GPU execution time and memory size. When running distributed training jobs, however, their GPU execution time and memory size have not been taken into account, which leads to the high variance of job completion time (JCT). Moreover, the jobs often run into the GPU out-of-memory (OoM) problem so that the unlucky job has to restart all over. To address the problems, we propose Xonar to profile the deep learning jobs and order them in the queue. The experiments show that Xonar with TensorFlow v1.6 reduces the tail JCT by 44% with the OoM problem eliminated.

KW - distributed deep learning

KW - GPU cloud

KW - GPU utilization

KW - job completion time

KW - parallel training

UR - http://www.scopus.com/inward/record.url?scp=85137590759&partnerID=8YFLogxK

U2 - 10.1109/CLOUD55607.2022.00030

DO - 10.1109/CLOUD55607.2022.00030

M3 - Conference contribution

AN - SCOPUS:85137590759

T3 - IEEE International Conference on Cloud Computing, CLOUD

SP - 112

EP - 114

BT - Proceedings - 2022 IEEE 15th International Conference on Cloud Computing, CLOUD 2022

A2 - Ardagna, Claudio Agostino

A2 - Atukorala, Nimanthi

A2 - Buyya, Rajkumar

A2 - Chang, Carl K.

A2 - Chang, Rong N.

A2 - Damiani, Ernesto

A2 - Dasgupta, Gargi Banerjee

A2 - Gagliardi, Fabrizio

A2 - Hagleitner, Christoph

A2 - Milojicic, Dejan

A2 - Trong, Tuan M Hoang

A2 - Ward, Robert

A2 - Xhafa, Fatos

A2 - Zhang, Jia

PB - IEEE Computer Society

Y2 - 10 July 2021 through 16 July 2021

ER -

Xonar: Profiling-based Job Orderer for Distributed Deep Learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this