Aggressive GPU cache bypassing with monolithic 3D-based NoC

Cong Thuan Do; Cheol Hong Kim; Sung Woo Chung

doi:10.1007/s11227-022-04878-6

Aggressive GPU cache bypassing with monolithic 3D-based NoC

Cong Thuan Do^*
, Cheol Hong Kim
, Sung Woo Chung

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

8 Citations (Scopus)

Abstract

Cache bypassing is widely employed to alleviate cache contention and pollution in GPUs. However, cache bypassing often puts more pressure on the network-on-chip (NoC) since the bypassed requests need to traverse the NoC to reach the lower-level memories, thus worsening the NoC congestion. In this paper, we propose an aggressive GPU cache bypassing technique (called SC-Table) to alleviate cache contention and pollution. The SC-Table relies on 2-bit saturating counters (SCs) to store the bypass history of warps. Memory requests issued by a warp are allowed to bypass the L1D when the corresponding SC’s value reaches the bypass threshold. In addition, we adopt the monolithic 3D-based NoC (M3D NoC) to provide better NoC throughput and latency. The combination of the SC-Table and the M3D NoC improves GPU performance by 34.6%, on average, over the baseline where there is no cache bypassing and the traditional 2D NoC is adopted.

Original language	English
Pages (from-to)	5421-5442
Number of pages	22
Journal	Journal of Supercomputing
Volume	79
Issue number	5
DOIs	https://doi.org/10.1007/s11227-022-04878-6
Publication status	Published - 2023 Mar

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1A2C2003500, No. 2020R1A6A3A13064398), Samsung Electronics, College of Information, Korea University, and School of Information and Communications Technology, Hanoi University of Science and Technology. We would also like to thank Dr. Young Seo Lee and Mr. Ji Heon Lee for their help with thermal simulation and anonymous reviewers for their helpful feedback.

Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

3D network-on-chip
Cache bypassing
GPGPU

ASJC Scopus subject areas

Theoretical Computer Science
Software
Information Systems
Hardware and Architecture

Access to Document

10.1007/s11227-022-04878-6

Cite this

@article{b5c571cb0a9746da9741da9345cfd65a,

title = "Aggressive GPU cache bypassing with monolithic 3D-based NoC",

abstract = "Cache bypassing is widely employed to alleviate cache contention and pollution in GPUs. However, cache bypassing often puts more pressure on the network-on-chip (NoC) since the bypassed requests need to traverse the NoC to reach the lower-level memories, thus worsening the NoC congestion. In this paper, we propose an aggressive GPU cache bypassing technique (called SC-Table) to alleviate cache contention and pollution. The SC-Table relies on 2-bit saturating counters (SCs) to store the bypass history of warps. Memory requests issued by a warp are allowed to bypass the L1D when the corresponding SC{\textquoteright}s value reaches the bypass threshold. In addition, we adopt the monolithic 3D-based NoC (M3D NoC) to provide better NoC throughput and latency. The combination of the SC-Table and the M3D NoC improves GPU performance by 34.6\%, on average, over the baseline where there is no cache bypassing and the traditional 2D NoC is adopted.",

keywords = "3D network-on-chip, Cache bypassing, GPGPU",

author = "Do, \{Cong Thuan\} and Kim, \{Cheol Hong\} and Chung, \{Sung Woo\}",

note = "Funding Information: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1A2C2003500, No. 2020R1A6A3A13064398), Samsung Electronics, College of Information, Korea University, and School of Information and Communications Technology, Hanoi University of Science and Technology. We would also like to thank Dr. Young Seo Lee and Mr. Ji Heon Lee for their help with thermal simulation and anonymous reviewers for their helpful feedback. Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.",

year = "2023",

month = mar,

doi = "10.1007/s11227-022-04878-6",

language = "English",

volume = "79",

pages = "5421--5442",

journal = "Journal of Supercomputing",

issn = "0920-8542",

publisher = "Springer Netherlands",

number = "5",

}

TY - JOUR

T1 - Aggressive GPU cache bypassing with monolithic 3D-based NoC

AU - Do, Cong Thuan

AU - Kim, Cheol Hong

AU - Chung, Sung Woo

N1 - Funding Information: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1A2C2003500, No. 2020R1A6A3A13064398), Samsung Electronics, College of Information, Korea University, and School of Information and Communications Technology, Hanoi University of Science and Technology. We would also like to thank Dr. Young Seo Lee and Mr. Ji Heon Lee for their help with thermal simulation and anonymous reviewers for their helpful feedback. Publisher Copyright: © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

PY - 2023/3

Y1 - 2023/3

N2 - Cache bypassing is widely employed to alleviate cache contention and pollution in GPUs. However, cache bypassing often puts more pressure on the network-on-chip (NoC) since the bypassed requests need to traverse the NoC to reach the lower-level memories, thus worsening the NoC congestion. In this paper, we propose an aggressive GPU cache bypassing technique (called SC-Table) to alleviate cache contention and pollution. The SC-Table relies on 2-bit saturating counters (SCs) to store the bypass history of warps. Memory requests issued by a warp are allowed to bypass the L1D when the corresponding SC’s value reaches the bypass threshold. In addition, we adopt the monolithic 3D-based NoC (M3D NoC) to provide better NoC throughput and latency. The combination of the SC-Table and the M3D NoC improves GPU performance by 34.6%, on average, over the baseline where there is no cache bypassing and the traditional 2D NoC is adopted.

AB - Cache bypassing is widely employed to alleviate cache contention and pollution in GPUs. However, cache bypassing often puts more pressure on the network-on-chip (NoC) since the bypassed requests need to traverse the NoC to reach the lower-level memories, thus worsening the NoC congestion. In this paper, we propose an aggressive GPU cache bypassing technique (called SC-Table) to alleviate cache contention and pollution. The SC-Table relies on 2-bit saturating counters (SCs) to store the bypass history of warps. Memory requests issued by a warp are allowed to bypass the L1D when the corresponding SC’s value reaches the bypass threshold. In addition, we adopt the monolithic 3D-based NoC (M3D NoC) to provide better NoC throughput and latency. The combination of the SC-Table and the M3D NoC improves GPU performance by 34.6%, on average, over the baseline where there is no cache bypassing and the traditional 2D NoC is adopted.

KW - 3D network-on-chip

KW - Cache bypassing

KW - GPGPU

UR - https://www.scopus.com/pages/publications/85140356816

U2 - 10.1007/s11227-022-04878-6

DO - 10.1007/s11227-022-04878-6

M3 - Article

AN - SCOPUS:85140356816

SN - 0920-8542

VL - 79

SP - 5421

EP - 5442

JO - Journal of Supercomputing

JF - Journal of Supercomputing

IS - 5

ER -

Aggressive GPU cache bypassing with monolithic 3D-based NoC

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this