Enhancing matrix multiplication with a monolithic 3-D-based scratchpad memory

Cong Thuan Do; Jeong Hwan Choi; Young Seo Lee; Cheol Hong Kim; Sung Woo Chung

doi:10.1109/LES.2020.3001954

Enhancing matrix multiplication with a monolithic 3-D-based scratchpad memory

Cong Thuan Do, Jeong Hwan Choi, Young Seo Lee, Cheol Hong Kim, Sung Woo Chung

Department of Computer Science and Engineering

Research output: Contribution to journal › Article › peer-review

6 Citations (Scopus)

Abstract

Convolutional neural networks (CNNs) are one of the most popular machine learning algorithms. The convolutional layers, which account for the most execution time of CNNs, are implemented with matrix multiplication because the convolution operation performs dot products between filters and local regions of the input. On the other hand, GPUs with thousands of cores were proven to significantly accelerate matrix multiplication, compared to CPUs with a limited number of cores, especially for large matrices. However, the current memory architecture allows only one row access at a time so that multiple accesses are necessary to read the column data of the second matrix, thus slowing down matrix multiplication. In this study, we adopt the monolithic 3-D integration for the GPU scratchpad memory, called monolithic 3-D integration (M3D) scratchpad memory (SPM), to enhance matrix multiplication. The M3D SPM allows one access to read the column data of the second matrix, similar to the case of the first matrix. The simulation results show that our M3D SPM improves the system performance by 46.3% for the 32 × 32 matrix multiplication, over the conventional 2-D SPM where the column data of the second matrix are read sequentially.

Original language	English
Article number	9115599
Pages (from-to)	57-60
Number of pages	4
Journal	IEEE Embedded Systems Letters
Volume	13
Issue number	2
DOIs	https://doi.org/10.1109/LES.2020.3001954
Publication status	Published - 2021 Jun

Bibliographical note

Funding Information:
Manuscript received April 10, 2020; accepted May 10, 2020. Date of publication June 12, 2020; date of current version May 27, 2021. This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) under Grant 2020R1A2C2003500, in part Samsung Electronics, Korea University, and in part by the Hanoi University of Science and Technology. This manuscript was recommended for publication by J. Hu. (Corresponding authors: Sung Woo Chung; Cheol Hong Kim.) Cong Thuan Do was with the Department of Computer Science and Engineering, Korea University, Seoul 02841, South Korea. He is now with the School of Information and Communications Technology, Hanoi University of Science and Technology, Hanoi 100000, Vietnam (e-mail: [email protected]).

Publisher Copyright:
© 2009-2012 IEEE.

Keywords

High performance
matrix multiplication
monolithic 3-D
neural network
scratchpad memory (SPM)

ASJC Scopus subject areas

Control and Systems Engineering
General Computer Science

Access to Document

10.1109/LES.2020.3001954

Cite this

@article{32f90cf2b258459ab72a14569a8cd479,

title = "Enhancing matrix multiplication with a monolithic 3-D-based scratchpad memory",

abstract = "Convolutional neural networks (CNNs) are one of the most popular machine learning algorithms. The convolutional layers, which account for the most execution time of CNNs, are implemented with matrix multiplication because the convolution operation performs dot products between filters and local regions of the input. On the other hand, GPUs with thousands of cores were proven to significantly accelerate matrix multiplication, compared to CPUs with a limited number of cores, especially for large matrices. However, the current memory architecture allows only one row access at a time so that multiple accesses are necessary to read the column data of the second matrix, thus slowing down matrix multiplication. In this study, we adopt the monolithic 3-D integration for the GPU scratchpad memory, called monolithic 3-D integration (M3D) scratchpad memory (SPM), to enhance matrix multiplication. The M3D SPM allows one access to read the column data of the second matrix, similar to the case of the first matrix. The simulation results show that our M3D SPM improves the system performance by 46.3% for the 32 × 32 matrix multiplication, over the conventional 2-D SPM where the column data of the second matrix are read sequentially.",

keywords = "High performance, matrix multiplication, monolithic 3-D, neural network, scratchpad memory (SPM)",

author = "Do, {Cong Thuan} and Choi, {Jeong Hwan} and Lee, {Young Seo} and Kim, {Cheol Hong} and Chung, {Sung Woo}",

note = "Funding Information: Manuscript received April 10, 2020; accepted May 10, 2020. Date of publication June 12, 2020; date of current version May 27, 2021. This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) under Grant 2020R1A2C2003500, in part Samsung Electronics, Korea University, and in part by the Hanoi University of Science and Technology. This manuscript was recommended for publication by J. Hu. (Corresponding authors: Sung Woo Chung; Cheol Hong Kim.) Cong Thuan Do was with the Department of Computer Science and Engineering, Korea University, Seoul 02841, South Korea. He is now with the School of Information and Communications Technology, Hanoi University of Science and Technology, Hanoi 100000, Vietnam (e-mail: [email protected]). Publisher Copyright: {\textcopyright} 2009-2012 IEEE.",

year = "2021",

month = jun,

doi = "10.1109/LES.2020.3001954",

language = "English",

volume = "13",

pages = "57--60",

journal = "IEEE Embedded Systems Letters",

issn = "1943-0663",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "2",

}

TY - JOUR

T1 - Enhancing matrix multiplication with a monolithic 3-D-based scratchpad memory

AU - Do, Cong Thuan

AU - Choi, Jeong Hwan

AU - Lee, Young Seo

AU - Kim, Cheol Hong

AU - Chung, Sung Woo

N1 - Funding Information: Manuscript received April 10, 2020; accepted May 10, 2020. Date of publication June 12, 2020; date of current version May 27, 2021. This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) under Grant 2020R1A2C2003500, in part Samsung Electronics, Korea University, and in part by the Hanoi University of Science and Technology. This manuscript was recommended for publication by J. Hu. (Corresponding authors: Sung Woo Chung; Cheol Hong Kim.) Cong Thuan Do was with the Department of Computer Science and Engineering, Korea University, Seoul 02841, South Korea. He is now with the School of Information and Communications Technology, Hanoi University of Science and Technology, Hanoi 100000, Vietnam (e-mail: [email protected]). Publisher Copyright: © 2009-2012 IEEE.

PY - 2021/6

Y1 - 2021/6

N2 - Convolutional neural networks (CNNs) are one of the most popular machine learning algorithms. The convolutional layers, which account for the most execution time of CNNs, are implemented with matrix multiplication because the convolution operation performs dot products between filters and local regions of the input. On the other hand, GPUs with thousands of cores were proven to significantly accelerate matrix multiplication, compared to CPUs with a limited number of cores, especially for large matrices. However, the current memory architecture allows only one row access at a time so that multiple accesses are necessary to read the column data of the second matrix, thus slowing down matrix multiplication. In this study, we adopt the monolithic 3-D integration for the GPU scratchpad memory, called monolithic 3-D integration (M3D) scratchpad memory (SPM), to enhance matrix multiplication. The M3D SPM allows one access to read the column data of the second matrix, similar to the case of the first matrix. The simulation results show that our M3D SPM improves the system performance by 46.3% for the 32 × 32 matrix multiplication, over the conventional 2-D SPM where the column data of the second matrix are read sequentially.

AB - Convolutional neural networks (CNNs) are one of the most popular machine learning algorithms. The convolutional layers, which account for the most execution time of CNNs, are implemented with matrix multiplication because the convolution operation performs dot products between filters and local regions of the input. On the other hand, GPUs with thousands of cores were proven to significantly accelerate matrix multiplication, compared to CPUs with a limited number of cores, especially for large matrices. However, the current memory architecture allows only one row access at a time so that multiple accesses are necessary to read the column data of the second matrix, thus slowing down matrix multiplication. In this study, we adopt the monolithic 3-D integration for the GPU scratchpad memory, called monolithic 3-D integration (M3D) scratchpad memory (SPM), to enhance matrix multiplication. The M3D SPM allows one access to read the column data of the second matrix, similar to the case of the first matrix. The simulation results show that our M3D SPM improves the system performance by 46.3% for the 32 × 32 matrix multiplication, over the conventional 2-D SPM where the column data of the second matrix are read sequentially.

KW - High performance

KW - matrix multiplication

KW - monolithic 3-D

KW - neural network

KW - scratchpad memory (SPM)

UR - http://www.scopus.com/inward/record.url?scp=85086724294&partnerID=8YFLogxK

U2 - 10.1109/LES.2020.3001954

DO - 10.1109/LES.2020.3001954

M3 - Article

AN - SCOPUS:85086724294

SN - 1943-0663

VL - 13

SP - 57

EP - 60

JO - IEEE Embedded Systems Letters

JF - IEEE Embedded Systems Letters

IS - 2

M1 - 9115599

ER -

Enhancing matrix multiplication with a monolithic 3-D-based scratchpad memory

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this