Optimization Techniques for Conversion of Quantization Aware Trained Deep Neural Networks to Lightweight Spiking Neural Networks

Kyungchul Lee, Sunghyun Choi, Dongwoo Lew, Jongsun Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we present spiking neural network (SNN) conversion technique optimized for converting low bit-width artificial neural networks (ANN) trained with quantization aware training (QAT). Conventional conversion technique suffers significant accuracy drop on QAT ANNs due to different activation function used for QAT ANNs. To minimize such accuracy drop of the conventional conversion, the proposed technique uses Spike-Norm Skip, which selectively applies threshold balancing. In addition, subtraction based reset is used to further reduce accuracy degradation. The proposed conversion technique achieves an accuracy of 89.92% (0.68% drop) with a 5-bit weight on CIFAR-10 using VGG-16.

Original languageEnglish
Title of host publication2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665435536
DOIs
Publication statusPublished - 2021 Jun 27
Event36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 - Jeju, Korea, Republic of
Duration: 2021 Jun 272021 Jun 30

Publication series

Name2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021

Conference

Conference36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021
Country/TerritoryKorea, Republic of
CityJeju
Period21/6/2721/6/30

Bibliographical note

Funding Information:
This work was supported in part by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2021-2018-0-01433) supervised by the IITP(Institute for Information & communications Technology Promotion), and in part by the Industrial Strategic Technology Development Program(10077445, Development of SoC technology based on Spiking Neural Cell for smart mobile and IoT Devices) funded By the Ministry of Trade, Industry & Energy(MOTIE, Korea)

Publisher Copyright:
© 2021 IEEE.

Keywords

  • ANN-SNN conversion
  • quantization aware training
  • spiking neural networks

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Optimization Techniques for Conversion of Quantization Aware Trained Deep Neural Networks to Lightweight Spiking Neural Networks'. Together they form a unique fingerprint.

Cite this