Abstract
Although deep neural networks (DNNs) show excellent performance in the image processing field, a massive amount of computation and memory access makes it difficult to deploy DNNs on mobile devices. To reduce the computation and the memory access, quantization shows great results. However, it is challenging to quantize the network into low bit-width without significant accuracy degradation. In this paper, we propose Clipped Quantization Aware Training (CQAT) to decrease the accuracy drop during the low bit-width quantization aware training. In the proposed CQAT, the original values are first quantized into 8-bit and then the quantized values are clipped so that only 4-bit for activations and 5-bit for weights are used. With the proposed technique, ResNet-18 for the CIFAR-100 dataset using 5-bit weight and 4-bit activation shows an accuracy drop of only 0.96% compared to the network using full precision weight and activation.
Original language | English |
---|---|
Title of host publication | Proceedings - International SoC Design Conference 2022, ISOCC 2022 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 370-371 |
Number of pages | 2 |
ISBN (Electronic) | 9781665459716 |
DOIs | |
Publication status | Published - 2022 |
Event | 19th International System-on-Chip Design Conference, ISOCC 2022 - Gangneung-si, Korea, Republic of Duration: 2022 Oct 19 → 2022 Oct 22 |
Publication series
Name | Proceedings - International SoC Design Conference 2022, ISOCC 2022 |
---|
Conference
Conference | 19th International System-on-Chip Design Conference, ISOCC 2022 |
---|---|
Country/Territory | Korea, Republic of |
City | Gangneung-si |
Period | 22/10/19 → 22/10/22 |
Bibliographical note
Funding Information:This work was supported by the National Research Foundation of Korea grant funded by the Korea government (NRF-2020R1A2C3014820)
Publisher Copyright:
© 2022 IEEE.
Keywords
- deep neural network
- quantization
- quantization aware training
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Science Applications
- Hardware and Architecture
- Safety, Risk, Reliability and Quality