Parallel huge matrix multiplication on a cluster with GPGPU accelerators

Seungyo Ryu; Dong Seung Kim

doi:10.1109/IPDPSW.2018.00139

Parallel huge matrix multiplication on a cluster with GPGPU accelerators

Seungyo Ryu, Dong Seung Kim

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

We design a parallel huge matrix multiplication algorithm on a cluster of GPU nodes. Since input matrices are too big to accommodate in the memory, the algorithm repeats the loading, computing, storing partial matrix data from/to disk and GPU buffer. The key to achieve the best speedup is not only to use GPU with full performance, but to reduce the overhead in data movement between disk and GPU buffer. We devise an efficient way to lower the latency of supplying the matching pair of the partial matrices to the GPU buffer, and to optimize the data partition, distribution, and disk access using the pipelined way. Experimental results show our algorithm outperforms a generic algorithm, resulting in the computing time reduction by 45%. Also, the scalability of the algorithm enhances with more GPU nodes.

Original language	English
Title of host publication	Proceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	877-882
Number of pages	6
ISBN (Print)	9781538655559
DOIs	https://doi.org/10.1109/IPDPSW.2018.00139
Publication status	Published - 2018 Aug 3
Event	32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018 - Vancouver, Canada Duration: 2018 May 21 → 2018 May 25

Other

Other	32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018
Country/Territory	Canada
City	Vancouver
Period	18/5/21 → 18/5/25

Keywords

GPU computing
Matrix multiplication
MPI
Parallel computing

ASJC Scopus subject areas

Artificial Intelligence
Computer Networks and Communications
Hardware and Architecture
Information Systems and Management

Access to Document

10.1109/IPDPSW.2018.00139

Cite this

Parallel huge matrix multiplication on a cluster with GPGPU accelerators. / Ryu, Seungyo; Kim, Dong Seung.
Proceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 877-882 8425506.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ryu, S & Kim, DS 2018, Parallel huge matrix multiplication on a cluster with GPGPU accelerators. in Proceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018., 8425506, Institute of Electrical and Electronics Engineers Inc., pp. 877-882, 32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018, Vancouver, Canada, 18/5/21. https://doi.org/10.1109/IPDPSW.2018.00139

@inproceedings{5d84ddb587bd450091dd0ded86a92845,

title = "Parallel huge matrix multiplication on a cluster with GPGPU accelerators",

abstract = "We design a parallel huge matrix multiplication algorithm on a cluster of GPU nodes. Since input matrices are too big to accommodate in the memory, the algorithm repeats the loading, computing, storing partial matrix data from/to disk and GPU buffer. The key to achieve the best speedup is not only to use GPU with full performance, but to reduce the overhead in data movement between disk and GPU buffer. We devise an efficient way to lower the latency of supplying the matching pair of the partial matrices to the GPU buffer, and to optimize the data partition, distribution, and disk access using the pipelined way. Experimental results show our algorithm outperforms a generic algorithm, resulting in the computing time reduction by 45%. Also, the scalability of the algorithm enhances with more GPU nodes.",

keywords = "GPU computing, Matrix multiplication, MPI, Parallel computing",

author = "Seungyo Ryu and Kim, {Dong Seung}",

year = "2018",

month = aug,

day = "3",

doi = "10.1109/IPDPSW.2018.00139",

language = "English",

isbn = "9781538655559",

pages = "877--882",

booktitle = "Proceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

note = "32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018 ; Conference date: 21-05-2018 Through 25-05-2018",

}

TY - GEN

T1 - Parallel huge matrix multiplication on a cluster with GPGPU accelerators

AU - Ryu, Seungyo

AU - Kim, Dong Seung

PY - 2018/8/3

Y1 - 2018/8/3

N2 - We design a parallel huge matrix multiplication algorithm on a cluster of GPU nodes. Since input matrices are too big to accommodate in the memory, the algorithm repeats the loading, computing, storing partial matrix data from/to disk and GPU buffer. The key to achieve the best speedup is not only to use GPU with full performance, but to reduce the overhead in data movement between disk and GPU buffer. We devise an efficient way to lower the latency of supplying the matching pair of the partial matrices to the GPU buffer, and to optimize the data partition, distribution, and disk access using the pipelined way. Experimental results show our algorithm outperforms a generic algorithm, resulting in the computing time reduction by 45%. Also, the scalability of the algorithm enhances with more GPU nodes.

AB - We design a parallel huge matrix multiplication algorithm on a cluster of GPU nodes. Since input matrices are too big to accommodate in the memory, the algorithm repeats the loading, computing, storing partial matrix data from/to disk and GPU buffer. The key to achieve the best speedup is not only to use GPU with full performance, but to reduce the overhead in data movement between disk and GPU buffer. We devise an efficient way to lower the latency of supplying the matching pair of the partial matrices to the GPU buffer, and to optimize the data partition, distribution, and disk access using the pipelined way. Experimental results show our algorithm outperforms a generic algorithm, resulting in the computing time reduction by 45%. Also, the scalability of the algorithm enhances with more GPU nodes.

KW - GPU computing

KW - Matrix multiplication

KW - MPI

KW - Parallel computing

UR - http://www.scopus.com/inward/record.url?scp=85052227842&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052227842&partnerID=8YFLogxK

U2 - 10.1109/IPDPSW.2018.00139

DO - 10.1109/IPDPSW.2018.00139

M3 - Conference contribution

AN - SCOPUS:85052227842

SN - 9781538655559

SP - 877

EP - 882

BT - Proceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018

Y2 - 21 May 2018 through 25 May 2018

ER -

Parallel huge matrix multiplication on a cluster with GPGPU accelerators

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this