An Energy-Efficient SNN Processor Design based on Sparse Direct Feedback and Spike Prediction

Seunghwan Bang, Dongwoo Lew, Sunghyun Choi, Jongsun Park

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    4 Citations (Scopus)

    Abstract

    In this paper, we present a novel spike prediction technique based spiking neural network (SNN) architecture, which can provide low cost on-chip learning based on the sparse direct feedback alignment (DFA). First, in order to reduce the repetitive synaptic operations in feedforward operations, a spike prediction technique is proposed, where the output spikes of active and inactive neurons are predicted by tracing the membrane potential changes. The proposed spike prediction achieves 63.84% reduction of the synaptic operations, and it can be efficiently exploited in the training as well as inference process. In addition, the number of weight updates in backward operations has been reduced by applying sparse DFA. In the sparse DFA, the synaptic weight updates are computed using the sparse feedback connections and the output error that is sparse as well. As a result, the number of weight updates in the training process has been reduced to 65.17%. The SNN processor with the proposed spike prediction technique and the sparse DFA has been implemented using 65nm CMOS process. The implementation results show that the SNN processor achieves the training energy savings of 52.16% with 0.3% accuracy degradation in MNIST dataset. It also consumes 1.18 uJ/image and 1.34 uJ/image for inference and training, respectively, with 97.46% accuracy on MNIST dataset.

    Original languageEnglish
    Title of host publicationIJCNN 2021 - International Joint Conference on Neural Networks, Proceedings
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    ISBN (Electronic)9780738133669
    DOIs
    Publication statusPublished - 2021 Jul 18
    Event2021 International Joint Conference on Neural Networks, IJCNN 2021 - Virtual, Shenzhen, China
    Duration: 2021 Jul 182021 Jul 22

    Publication series

    NameProceedings of the International Joint Conference on Neural Networks
    Volume2021-July

    Conference

    Conference2021 International Joint Conference on Neural Networks, IJCNN 2021
    Country/TerritoryChina
    CityVirtual, Shenzhen
    Period21/7/1821/7/22

    Bibliographical note

    Funding Information:
    and 2 have a similar architecture, and only the number of input spikes processed in a cycle is different. Each hidden layer consists of the input spike buffer, prediction flag memory to store neuron states, and membrane update module (MUM) for computing membrane potentials and generating output spikes. The proposed spike prediction technique is supported by the prediction module located in MUM. Spike count memory, weight update module, and direct feedback connection generator (DFG) are also designed to support on-chip learning capability. The structure of the output layer is similar to hidden layers except DFG, as output error is directly used to update weights.

    Publisher Copyright:
    © 2021 IEEE.

    Keywords

    • Spiking neural network
    • energy-efficient neuromorphic system
    • on-chip learning
    • sparse direct feedback alignment
    • spike prediction

    ASJC Scopus subject areas

    • Software
    • Artificial Intelligence

    Fingerprint

    Dive into the research topics of 'An Energy-Efficient SNN Processor Design based on Sparse Direct Feedback and Spike Prediction'. Together they form a unique fingerprint.

    Cite this