Implementation of Pipelined Adder Tree for Long Short-Term Memory Cells

Seok Young Kim, Chang Hyun Kim, Seon Wook Kim

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    The characteristics of recurrent and functional reduction in RNNs (Recurrent Neural Networks) may cause an inefficient execution in conventional hardware such as CPU (Central Processing Unit) and GPU (Graphics Processing Unit). Their large number of recurrent data prevents the CPU from exploiting their locality, while the reduction feature limits the GPU parallelism. As a result, several dedicated hardware for efficient RNN execution has been proposed recently. In this paper, we propose a processing element that performs primary operations (i.e., multiplication and accumulation operation and activation function operation) of LSTM (Long Short-Term Memory), the most dominant model of RNN We implemented the processing element on an FPGA (Field-Programmable Gate Array) and verified the accuracy by measuring the BLEU score (Bilingual Evaluation Understudy score) of the PyTorch-based LSTM application. It was decreased by 2.3% compared to the CPU result.

    Original languageEnglish
    Title of host publication2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    ISBN (Electronic)9781665435536
    DOIs
    Publication statusPublished - 2021 Jun 27
    Event36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 - Jeju, Korea, Republic of
    Duration: 2021 Jun 272021 Jun 30

    Publication series

    Name2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
    Volume2021-January

    Conference

    Conference36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
    Country/TerritoryKorea, Republic of
    CityJeju
    Period21/6/2721/6/30

    Bibliographical note

    Funding Information:
    This work was supported in part by SK Hynix Inc.

    Publisher Copyright:
    © 2021 IEEE.

    Keywords

    • Bfloat16
    • Floating-point MAC unit
    • Long Short-Term Memory
    • Recurrent Neural Networks

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Networks and Communications
    • Hardware and Architecture
    • Information Systems
    • Electrical and Electronic Engineering

    Fingerprint

    Dive into the research topics of 'Implementation of Pipelined Adder Tree for Long Short-Term Memory Cells'. Together they form a unique fingerprint.

    Cite this