Clipped Quantization Aware Training for Hardware Friendly Implementation of Image Classification Networks

Kyungchul Lee, Jongsun Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Although deep neural networks (DNNs) show excellent performance in the image processing field, a massive amount of computation and memory access makes it difficult to deploy DNNs on mobile devices. To reduce the computation and the memory access, quantization shows great results. However, it is challenging to quantize the network into low bit-width without significant accuracy degradation. In this paper, we propose Clipped Quantization Aware Training (CQAT) to decrease the accuracy drop during the low bit-width quantization aware training. In the proposed CQAT, the original values are first quantized into 8-bit and then the quantized values are clipped so that only 4-bit for activations and 5-bit for weights are used. With the proposed technique, ResNet-18 for the CIFAR-100 dataset using 5-bit weight and 4-bit activation shows an accuracy drop of only 0.96% compared to the network using full precision weight and activation.

Original languageEnglish
Title of host publicationProceedings - International SoC Design Conference 2022, ISOCC 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages370-371
Number of pages2
ISBN (Electronic)9781665459716
DOIs
Publication statusPublished - 2022
Event19th International System-on-Chip Design Conference, ISOCC 2022 - Gangneung-si, Korea, Republic of
Duration: 2022 Oct 192022 Oct 22

Publication series

NameProceedings - International SoC Design Conference 2022, ISOCC 2022

Conference

Conference19th International System-on-Chip Design Conference, ISOCC 2022
Country/TerritoryKorea, Republic of
CityGangneung-si
Period22/10/1922/10/22

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea grant funded by the Korea government (NRF-2020R1A2C3014820)

Publisher Copyright:
© 2022 IEEE.

Keywords

  • deep neural network
  • quantization
  • quantization aware training

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Hardware and Architecture
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Clipped Quantization Aware Training for Hardware Friendly Implementation of Image Classification Networks'. Together they form a unique fingerprint.

Cite this