Abstract
For successful deployment of deep neural networks (DNNs) on resource-constrained devices, retraining-based quantization has been widely adopted to reduce the number of DRAM accesses. By properly setting training parameters, such as batch size and learning rate, bit widths of both weights and activations can be uniformly quantized down to 4 bit while maintaining full precision accuracy. In this article, we present a retraining-based mixed-precision quantization approach and its customized DNN accelerator to achieve high energy efficiency. In the proposed quantization, in the middle of retraining, an additional bit (extra quantization level) is assigned to the weights that have shown frequent switching between two contiguous quantization levels since it means that both quantization levels cannot help to reduce quantization loss. We also mitigate the gradient noise that occurs in the retraining process by taking a lower learning rate near the quantization threshold. For the proposed novel mixed-precision quantized network (MPQ-network), we have implemented a customized accelerator using a 65-nm CMOS process. In the accelerator, the proposed processing elements (PEs) can be dynamically reconfigured to process variable bit widths from 2 to 4 bit for both weights and activations. The numerical results show that the proposed quantization can achieve 1.37 {\times } better compression ratio for VGG-9 using CIFAR-10 data set compared with a uniform 4-bit (both weights and activations) model without loss of classification accuracy. The proposed accelerator also shows 1.29\times of energy savings for VGG-9 using the CIFAR-10 data set over the state-of-the-art accelerator.
Original language | English |
---|---|
Article number | 9154572 |
Pages (from-to) | 2925-2938 |
Number of pages | 14 |
Journal | IEEE Transactions on Neural Networks and Learning Systems |
Volume | 32 |
Issue number | 7 |
DOIs | |
Publication status | Published - 2021 Jul |
Bibliographical note
Funding Information:Manuscript received September 3, 2019; revised April 7, 2020; accepted July 7, 2020. Date of publication August 3, 2020; date of current version July 7, 2021. This work was supported in part by the National Research Foundation of Korea grant funded by the Korea Government under Grant NRF-2020R1A2C3014820, in part by the Ministry of Science and ICT (MSIT), South Korea, through the Information Technology Research Center (ITRC) Support Program under Grant IITP-2020-2018-0-01433, supervised by the Institute for Information & Communications Technology Promotion (IITP), and in part by the Industrial Strategic Technology Development Program (Development of SoC Technology Based on Spiking Neural Cell for Smart Mobile and IoT Devices) under Grant 10077445. (Corresponding author: Jongsun Park.) Nahsung Kim is with the Memory Solution Product Team, SK Hynix Inc., Icheon 17336, South Korea (e-mail: nahsung.kim@sk.com).
Publisher Copyright:
© 2012 IEEE.
Keywords
- Deep neural network (DNN) accelerator
- energy-efficient accelerator
- model compression
- quantization
ASJC Scopus subject areas
- Software
- Computer Science Applications
- Computer Networks and Communications
- Artificial Intelligence