Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network

Seungeon Hwang; Jongsun Park

doi:10.1109/ISOCC56007.2022.10031332

Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network

Seungeon Hwang, Jongsun Park

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Low bit-precision quantization is an efficient method of reducing the computation complexity of neural networks. However, low bit-precision quantization of complex tasks such as depth estimation is a challenging issue due to the multiple skip connections of U-Net like networks. To maintain accuracy while lowering bit width, activation statistics should be considered. In this paper, we propose a percentile clipping technique for low bit-precision quantization for depth estimation network. By selecting the clipping point as the percentile value considering the distribution of the skip connection parts, our proposed technique can efficiently reduce the relative squared quantization error by 34.78% when using uniform 4-bit weights and activations.

Original language	English
Title of host publication	Proceedings - International SoC Design Conference 2022, ISOCC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	73-74
Number of pages	2
ISBN (Electronic)	9781665459716
DOIs	https://doi.org/10.1109/ISOCC56007.2022.10031332
Publication status	Published - 2022
Event	19th International System-on-Chip Design Conference, ISOCC 2022 - Gangneung-si, Korea, Republic of Duration: 2022 Oct 19 → 2022 Oct 22

Publication series

Name	Proceedings - International SoC Design Conference 2022, ISOCC 2022

Conference

Conference	19th International System-on-Chip Design Conference, ISOCC 2022
Country/Territory	Korea, Republic of
City	Gangneung-si
Period	22/10/19 → 22/10/22

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea grant funded by the Korea government (NRF-2020R1A2C3014820).

Publisher Copyright:
© 2022 IEEE.

Keywords

Clipping
Depth Estimation
Quantization
U-Net

ASJC Scopus subject areas

Artificial Intelligence
Computer Science Applications
Hardware and Architecture
Safety, Risk, Reliability and Quality

Access to Document

10.1109/ISOCC56007.2022.10031332

Cite this

Hwang, S., & Park, J. (2022). Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network. In Proceedings - International SoC Design Conference 2022, ISOCC 2022 (pp. 73-74). (Proceedings - International SoC Design Conference 2022, ISOCC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISOCC56007.2022.10031332

Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network. / Hwang, Seungeon; Park, Jongsun.
Proceedings - International SoC Design Conference 2022, ISOCC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 73-74 (Proceedings - International SoC Design Conference 2022, ISOCC 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hwang, S & Park, J 2022, Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network. in Proceedings - International SoC Design Conference 2022, ISOCC 2022. Proceedings - International SoC Design Conference 2022, ISOCC 2022, Institute of Electrical and Electronics Engineers Inc., pp. 73-74, 19th International System-on-Chip Design Conference, ISOCC 2022, Gangneung-si, Korea, Republic of, 22/10/19. https://doi.org/10.1109/ISOCC56007.2022.10031332

@inproceedings{664ccb39c5fc4d86bf30f1f93cac7c9d,

title = "Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network",

abstract = "Low bit-precision quantization is an efficient method of reducing the computation complexity of neural networks. However, low bit-precision quantization of complex tasks such as depth estimation is a challenging issue due to the multiple skip connections of U-Net like networks. To maintain accuracy while lowering bit width, activation statistics should be considered. In this paper, we propose a percentile clipping technique for low bit-precision quantization for depth estimation network. By selecting the clipping point as the percentile value considering the distribution of the skip connection parts, our proposed technique can efficiently reduce the relative squared quantization error by 34.78% when using uniform 4-bit weights and activations.",

keywords = "Clipping, Depth Estimation, Quantization, U-Net",

author = "Seungeon Hwang and Jongsun Park",

note = "Funding Information: This work was supported by the National Research Foundation of Korea grant funded by the Korea government (NRF-2020R1A2C3014820). Publisher Copyright: {\textcopyright} 2022 IEEE.; 19th International System-on-Chip Design Conference, ISOCC 2022 ; Conference date: 19-10-2022 Through 22-10-2022",

year = "2022",

doi = "10.1109/ISOCC56007.2022.10031332",

language = "English",

series = "Proceedings - International SoC Design Conference 2022, ISOCC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "73--74",

booktitle = "Proceedings - International SoC Design Conference 2022, ISOCC 2022",

}

TY - GEN

T1 - Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network

AU - Hwang, Seungeon

AU - Park, Jongsun

PY - 2022

Y1 - 2022

N2 - Low bit-precision quantization is an efficient method of reducing the computation complexity of neural networks. However, low bit-precision quantization of complex tasks such as depth estimation is a challenging issue due to the multiple skip connections of U-Net like networks. To maintain accuracy while lowering bit width, activation statistics should be considered. In this paper, we propose a percentile clipping technique for low bit-precision quantization for depth estimation network. By selecting the clipping point as the percentile value considering the distribution of the skip connection parts, our proposed technique can efficiently reduce the relative squared quantization error by 34.78% when using uniform 4-bit weights and activations.

AB - Low bit-precision quantization is an efficient method of reducing the computation complexity of neural networks. However, low bit-precision quantization of complex tasks such as depth estimation is a challenging issue due to the multiple skip connections of U-Net like networks. To maintain accuracy while lowering bit width, activation statistics should be considered. In this paper, we propose a percentile clipping technique for low bit-precision quantization for depth estimation network. By selecting the clipping point as the percentile value considering the distribution of the skip connection parts, our proposed technique can efficiently reduce the relative squared quantization error by 34.78% when using uniform 4-bit weights and activations.

KW - Clipping

KW - Depth Estimation

KW - Quantization

KW - U-Net

UR - http://www.scopus.com/inward/record.url?scp=85148468767&partnerID=8YFLogxK

U2 - 10.1109/ISOCC56007.2022.10031332

DO - 10.1109/ISOCC56007.2022.10031332

M3 - Conference contribution

AN - SCOPUS:85148468767

T3 - Proceedings - International SoC Design Conference 2022, ISOCC 2022

SP - 73

EP - 74

BT - Proceedings - International SoC Design Conference 2022, ISOCC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 19th International System-on-Chip Design Conference, ISOCC 2022

Y2 - 19 October 2022 through 22 October 2022

ER -

Percentile Clipping based Low Bit-Precision Quantization for Depth Estimation Network

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this