Clipped Quantization Aware Training for Hardware Friendly Implementation of Image Classification Networks

Kyungchul Lee, Jongsun Park

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    Although deep neural networks (DNNs) show excellent performance in the image processing field, a massive amount of computation and memory access makes it difficult to deploy DNNs on mobile devices. To reduce the computation and the memory access, quantization shows great results. However, it is challenging to quantize the network into low bit-width without significant accuracy degradation. In this paper, we propose Clipped Quantization Aware Training (CQAT) to decrease the accuracy drop during the low bit-width quantization aware training. In the proposed CQAT, the original values are first quantized into 8-bit and then the quantized values are clipped so that only 4-bit for activations and 5-bit for weights are used. With the proposed technique, ResNet-18 for the CIFAR-100 dataset using 5-bit weight and 4-bit activation shows an accuracy drop of only 0.96% compared to the network using full precision weight and activation.

    Original languageEnglish
    Title of host publicationProceedings - International SoC Design Conference 2022, ISOCC 2022
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages370-371
    Number of pages2
    ISBN (Electronic)9781665459716
    DOIs
    Publication statusPublished - 2022
    Event19th International System-on-Chip Design Conference, ISOCC 2022 - Gangneung-si, Korea, Republic of
    Duration: 2022 Oct 192022 Oct 22

    Publication series

    NameProceedings - International SoC Design Conference 2022, ISOCC 2022

    Conference

    Conference19th International System-on-Chip Design Conference, ISOCC 2022
    Country/TerritoryKorea, Republic of
    CityGangneung-si
    Period22/10/1922/10/22

    Bibliographical note

    Funding Information:
    This work was supported by the National Research Foundation of Korea grant funded by the Korea government (NRF-2020R1A2C3014820)

    Publisher Copyright:
    © 2022 IEEE.

    Keywords

    • deep neural network
    • quantization
    • quantization aware training

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Science Applications
    • Hardware and Architecture
    • Safety, Risk, Reliability and Quality

    Fingerprint

    Dive into the research topics of 'Clipped Quantization Aware Training for Hardware Friendly Implementation of Image Classification Networks'. Together they form a unique fingerprint.

    Cite this