Class Difficulty based Mixed Precision Quantization for Low Complexity CNN Training

Joongho Jo, Jongsun Park

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    Low-bit quantization of CNN training is highly needed for reducing the computational complexity of convolutional neural network (CNN) training. In CNN training, some of the classes can finish training early (reaches high accuracy in early training epochs) while other classes need more time (epochs) to finish training. This measure of training difficulty can be efficiently exploited for the mixed precision quantization to reduce the computational complexity of CNN training. In this paper, we present a training difficulty based mixed precision training approach, where easy-to-train classes are trained using low-bit quantization and the hard-to-train classes are trained using high bit quantization. The simulation results show that the proposed mixed precision training can achieve 1.33X improved compression ratio with the same accuracy compared to 8-bit (activations and weights) and 16-bit (gradients of activation and weight) uniform quantization training for ResNet-20 using the CIFAR-10 dataset.

    Original languageEnglish
    Title of host publicationProceedings - International SoC Design Conference 2022, ISOCC 2022
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages372-373
    Number of pages2
    ISBN (Electronic)9781665459716
    DOIs
    Publication statusPublished - 2022
    Event19th International System-on-Chip Design Conference, ISOCC 2022 - Gangneung-si, Korea, Republic of
    Duration: 2022 Oct 192022 Oct 22

    Publication series

    NameProceedings - International SoC Design Conference 2022, ISOCC 2022

    Conference

    Conference19th International System-on-Chip Design Conference, ISOCC 2022
    Country/TerritoryKorea, Republic of
    CityGangneung-si
    Period22/10/1922/10/22

    Bibliographical note

    Publisher Copyright:
    © 2022 IEEE.

    Keywords

    • Convolutional Neural Network (CNN)
    • Low-bit quantization training
    • Mixed precision training

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Science Applications
    • Hardware and Architecture
    • Safety, Risk, Reliability and Quality

    Fingerprint

    Dive into the research topics of 'Class Difficulty based Mixed Precision Quantization for Low Complexity CNN Training'. Together they form a unique fingerprint.

    Cite this