Abstract
The characteristics of recurrent and functional reduction in RNNs (Recurrent Neural Networks) may cause an inefficient execution in conventional hardware such as CPU (Central Processing Unit) and GPU (Graphics Processing Unit). Their large number of recurrent data prevents the CPU from exploiting their locality, while the reduction feature limits the GPU parallelism. As a result, several dedicated hardware for efficient RNN execution has been proposed recently. In this paper, we propose a processing element that performs primary operations (i.e., multiplication and accumulation operation and activation function operation) of LSTM (Long Short-Term Memory), the most dominant model of RNN We implemented the processing element on an FPGA (Field-Programmable Gate Array) and verified the accuracy by measuring the BLEU score (Bilingual Evaluation Understudy score) of the PyTorch-based LSTM application. It was decreased by 2.3% compared to the CPU result.
Original language | English |
---|---|
Title of host publication | 2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
ISBN (Electronic) | 9781665435536 |
DOIs | |
Publication status | Published - 2021 Jun 27 |
Event | 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 - Jeju, Korea, Republic of Duration: 2021 Jun 27 → 2021 Jun 30 |
Publication series
Name | 2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 |
---|---|
Volume | 2021-January |
Conference
Conference | 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 |
---|---|
Country/Territory | Korea, Republic of |
City | Jeju |
Period | 21/6/27 → 21/6/30 |
Bibliographical note
Funding Information:This work was supported in part by SK Hynix Inc.
Publisher Copyright:
© 2021 IEEE.
Keywords
- Bfloat16
- Floating-point MAC unit
- Long Short-Term Memory
- Recurrent Neural Networks
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Networks and Communications
- Hardware and Architecture
- Information Systems
- Electrical and Electronic Engineering