Implementation of Pipelined Adder Tree for Long Short-Term Memory Cells

Seok Young Kim, Chang Hyun Kim, Seon Wook Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The characteristics of recurrent and functional reduction in RNNs (Recurrent Neural Networks) may cause an inefficient execution in conventional hardware such as CPU (Central Processing Unit) and GPU (Graphics Processing Unit). Their large number of recurrent data prevents the CPU from exploiting their locality, while the reduction feature limits the GPU parallelism. As a result, several dedicated hardware for efficient RNN execution has been proposed recently. In this paper, we propose a processing element that performs primary operations (i.e., multiplication and accumulation operation and activation function operation) of LSTM (Long Short-Term Memory), the most dominant model of RNN We implemented the processing element on an FPGA (Field-Programmable Gate Array) and verified the accuracy by measuring the BLEU score (Bilingual Evaluation Understudy score) of the PyTorch-based LSTM application. It was decreased by 2.3% compared to the CPU result.

Original languageEnglish
Title of host publication2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665435536
DOIs
Publication statusPublished - 2021 Jun 27
Event36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 - Jeju, Korea, Republic of
Duration: 2021 Jun 272021 Jun 30

Publication series

Name2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
Volume2021-January

Conference

Conference36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
Country/TerritoryKorea, Republic of
CityJeju
Period21/6/2721/6/30

Bibliographical note

Funding Information:
This work was supported in part by SK Hynix Inc.

Publisher Copyright:
© 2021 IEEE.

Keywords

  • Bfloat16
  • Floating-point MAC unit
  • Long Short-Term Memory
  • Recurrent Neural Networks

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Implementation of Pipelined Adder Tree for Long Short-Term Memory Cells'. Together they form a unique fingerprint.

Cite this