Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models

Mun Seong Park; Seok Young Kim; Seon Wook Kim

doi:10.1109/ICEIC61013.2024.10457258

Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models

Mun Seong Park, Seok Young Kim, Seon Wook Kim

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The emerging transformers suffer from the memory bottleneck because of their low data locality and large data sizes, and we are actively studying Processing in Memory (PIM), which computes inside memory, to overcome the bottleneck. However, as their model parameters get bigger, a single PIM device would be insufficient because of its limited memory capacity and computing resources. In this paper, we develop a low-overhead data communication method between PIMs on a PCIe-based multi-PIM platform. We adopt an XDMA-based PIM-to-PIM (P2P) direct data communication mechanism to eliminate redundant data movement between CPU and PIMs. Our P2P achieves 1.69x, 1.70x, 1.61x, and 1.63x speedups compared to the DMA using the system memory as a buffer when transferring 16MB, 32MB, 64MB, and 128MB data, respectively.

Original language	English
Title of host publication	2024 International Conference on Electronics, Information, and Communication, ICEIC 2024
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9798350371888
DOIs	https://doi.org/10.1109/ICEIC61013.2024.10457258
Publication status	Published - 2024
Event	2024 International Conference on Electronics, Information, and Communication, ICEIC 2024 - Taipei, Taiwan, Province of China Duration: 2024 Jan 28 → 2024 Jan 31

Publication series

Name	2024 International Conference on Electronics, Information, and Communication, ICEIC 2024

Conference

Conference	2024 International Conference on Electronics, Information, and Communication, ICEIC 2024
Country/Territory	Taiwan, Province of China
City	Taipei
Period	24/1/28 → 24/1/31

Bibliographical note

Publisher Copyright:
© 2024 IEEE.

Keywords

Direct Memory Access
Multi-PIM Platform
Peer-to-Peer Communication
Processing-in-Memory
XDMA

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Hardware and Architecture
Information Systems
Energy Engineering and Power Technology
Electrical and Electronic Engineering

Access to Document

10.1109/ICEIC61013.2024.10457258

Cite this

Park, M. S., Kim, S. Y., & Kim, S. W. (2024). Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models. In 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024 (2024 International Conference on Electronics, Information, and Communication, ICEIC 2024). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICEIC61013.2024.10457258

Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models. / Park, Mun Seong; Kim, Seok Young; Kim, Seon Wook.
2024 International Conference on Electronics, Information, and Communication, ICEIC 2024. Institute of Electrical and Electronics Engineers Inc., 2024. (2024 International Conference on Electronics, Information, and Communication, ICEIC 2024).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Park, MS, Kim, SY & Kim, SW 2024, Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models. in 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024. 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024, Institute of Electrical and Electronics Engineers Inc., 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024, Taipei, Taiwan, Province of China, 24/1/28. https://doi.org/10.1109/ICEIC61013.2024.10457258

Park MS, Kim SY, Kim SW. Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models. In 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024. Institute of Electrical and Electronics Engineers Inc. 2024. (2024 International Conference on Electronics, Information, and Communication, ICEIC 2024). doi: 10.1109/ICEIC61013.2024.10457258

Park, Mun Seong ; Kim, Seok Young ; Kim, Seon Wook. / Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models. 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024. Institute of Electrical and Electronics Engineers Inc., 2024. (2024 International Conference on Electronics, Information, and Communication, ICEIC 2024).

@inproceedings{58a51af36a234392b04da6dcbbcfff84,

title = "Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models",

abstract = "The emerging transformers suffer from the memory bottleneck because of their low data locality and large data sizes, and we are actively studying Processing in Memory (PIM), which computes inside memory, to overcome the bottleneck. However, as their model parameters get bigger, a single PIM device would be insufficient because of its limited memory capacity and computing resources. In this paper, we develop a low-overhead data communication method between PIMs on a PCIe-based multi-PIM platform. We adopt an XDMA-based PIM-to-PIM (P2P) direct data communication mechanism to eliminate redundant data movement between CPU and PIMs. Our P2P achieves 1.69x, 1.70x, 1.61x, and 1.63x speedups compared to the DMA using the system memory as a buffer when transferring 16MB, 32MB, 64MB, and 128MB data, respectively.",

keywords = "Direct Memory Access, Multi-PIM Platform, Peer-to-Peer Communication, Processing-in-Memory, XDMA",

author = "Park, {Mun Seong} and Kim, {Seok Young} and Kim, {Seon Wook}",

note = "Publisher Copyright: {\textcopyright} 2024 IEEE.; 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024 ; Conference date: 28-01-2024 Through 31-01-2024",

year = "2024",

doi = "10.1109/ICEIC61013.2024.10457258",

language = "English",

series = "2024 International Conference on Electronics, Information, and Communication, ICEIC 2024",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2024 International Conference on Electronics, Information, and Communication, ICEIC 2024",

}

TY - GEN

T1 - Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models

AU - Park, Mun Seong

AU - Kim, Seok Young

AU - Kim, Seon Wook

PY - 2024

Y1 - 2024

N2 - The emerging transformers suffer from the memory bottleneck because of their low data locality and large data sizes, and we are actively studying Processing in Memory (PIM), which computes inside memory, to overcome the bottleneck. However, as their model parameters get bigger, a single PIM device would be insufficient because of its limited memory capacity and computing resources. In this paper, we develop a low-overhead data communication method between PIMs on a PCIe-based multi-PIM platform. We adopt an XDMA-based PIM-to-PIM (P2P) direct data communication mechanism to eliminate redundant data movement between CPU and PIMs. Our P2P achieves 1.69x, 1.70x, 1.61x, and 1.63x speedups compared to the DMA using the system memory as a buffer when transferring 16MB, 32MB, 64MB, and 128MB data, respectively.

AB - The emerging transformers suffer from the memory bottleneck because of their low data locality and large data sizes, and we are actively studying Processing in Memory (PIM), which computes inside memory, to overcome the bottleneck. However, as their model parameters get bigger, a single PIM device would be insufficient because of its limited memory capacity and computing resources. In this paper, we develop a low-overhead data communication method between PIMs on a PCIe-based multi-PIM platform. We adopt an XDMA-based PIM-to-PIM (P2P) direct data communication mechanism to eliminate redundant data movement between CPU and PIMs. Our P2P achieves 1.69x, 1.70x, 1.61x, and 1.63x speedups compared to the DMA using the system memory as a buffer when transferring 16MB, 32MB, 64MB, and 128MB data, respectively.

KW - Direct Memory Access

KW - Multi-PIM Platform

KW - Peer-to-Peer Communication

KW - Processing-in-Memory

KW - XDMA

UR - http://www.scopus.com/inward/record.url?scp=85189245166&partnerID=8YFLogxK

U2 - 10.1109/ICEIC61013.2024.10457258

DO - 10.1109/ICEIC61013.2024.10457258

M3 - Conference contribution

AN - SCOPUS:85189245166

T3 - 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024

BT - 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2024 International Conference on Electronics, Information, and Communication, ICEIC 2024

Y2 - 28 January 2024 through 31 January 2024

ER -

Low Overhead PIM-to-PIM Communication on PCIe-based Multi-PIM Platforms for Executing Large-Scale AI Models

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this