Image Compression-Aware Deep Camera ISP Network

Kwang Hyun Uhm; Kyuyeon Choi; Seung Won Jung; Sung Jea Ko

doi:10.1109/ACCESS.2021.3116702

Image Compression-Aware Deep Camera ISP Network

Kwang Hyun Uhm, Kyuyeon Choi, Seung Won Jung, Sung Jea Ko

School of Electrical Engineering

Research output: Contribution to journal › Article › peer-review

1 Citation (Scopus)

Abstract

Several recent studies have attempted to fully replace the conventional camera image signal processing (ISP) pipeline with convolutional neural networks (CNNs). However, the previous CNN-based ISPs, simply referred to as ISP-Nets, have not explicitly considered that images have to be lossy-compressed in most cases, especially by the off-the-shelf JPEG. To address this issue, in this paper, we propose a novel compression-aware deep camera ISP learning framework. At first, we introduce a new use case of compression artifacts simulation network (CAS-Net), which operates in the opposite way of commonly used compression artifacts reduction networks. Then, the CAS-Net is connected with an ISP-Net such that the ISP network can be trained with consideration of image compression. Throughout experimental studies, we show that our compression-aware camera ISP network can produce images with a better tradeoff between bit-rate and image quality compared to its compression-agnostic version when the performance is evaluated after JPEG compression.

Original language	English
Pages (from-to)	137824-137832
Number of pages	9
Journal	IEEE Access
Volume	9
DOIs	https://doi.org/10.1109/ACCESS.2021.3116702
Publication status	Published - 2021

Bibliographical note

Funding Information:
This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT Future Planning under Grant NRF-2020R1F1A1069009, and in part by the Institute for Information and Communications Technology Promotion (IITP) grant funded by the Korea Government (MSIT) under Grant 2019-0-00268, Development of SW Technology for Recognition, Judgment and Path Control Algorithm Verification Simulation and Dataset Generation.

Publisher Copyright:
© 2013 IEEE.

Keywords

Camera ISP
compression artifacts
convolutional neural network
image compression

ASJC Scopus subject areas

Computer Science(all)
Materials Science(all)
Engineering(all)

Access to Document

10.1109/ACCESS.2021.3116702

Cite this

@article{149f2dc70b934184a3b55616b64b94dc,

title = "Image Compression-Aware Deep Camera ISP Network",

abstract = "Several recent studies have attempted to fully replace the conventional camera image signal processing (ISP) pipeline with convolutional neural networks (CNNs). However, the previous CNN-based ISPs, simply referred to as ISP-Nets, have not explicitly considered that images have to be lossy-compressed in most cases, especially by the off-the-shelf JPEG. To address this issue, in this paper, we propose a novel compression-aware deep camera ISP learning framework. At first, we introduce a new use case of compression artifacts simulation network (CAS-Net), which operates in the opposite way of commonly used compression artifacts reduction networks. Then, the CAS-Net is connected with an ISP-Net such that the ISP network can be trained with consideration of image compression. Throughout experimental studies, we show that our compression-aware camera ISP network can produce images with a better tradeoff between bit-rate and image quality compared to its compression-agnostic version when the performance is evaluated after JPEG compression. ",

keywords = "Camera ISP, compression artifacts, convolutional neural network, image compression",

author = "Uhm, {Kwang Hyun} and Kyuyeon Choi and Jung, {Seung Won} and Ko, {Sung Jea}",

note = "Funding Information: This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT Future Planning under Grant NRF-2020R1F1A1069009, and in part by the Institute for Information and Communications Technology Promotion (IITP) grant funded by the Korea Government (MSIT) under Grant 2019-0-00268, Development of SW Technology for Recognition, Judgment and Path Control Algorithm Verification Simulation and Dataset Generation. Publisher Copyright: {\textcopyright} 2013 IEEE.",

year = "2021",

doi = "10.1109/ACCESS.2021.3116702",

language = "English",

volume = "9",

pages = "137824--137832",

journal = "IEEE Access",

issn = "2169-3536",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Image Compression-Aware Deep Camera ISP Network

AU - Uhm, Kwang Hyun

AU - Choi, Kyuyeon

AU - Jung, Seung Won

AU - Ko, Sung Jea

N1 - Funding Information: This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT Future Planning under Grant NRF-2020R1F1A1069009, and in part by the Institute for Information and Communications Technology Promotion (IITP) grant funded by the Korea Government (MSIT) under Grant 2019-0-00268, Development of SW Technology for Recognition, Judgment and Path Control Algorithm Verification Simulation and Dataset Generation. Publisher Copyright: © 2013 IEEE.

PY - 2021

Y1 - 2021

N2 - Several recent studies have attempted to fully replace the conventional camera image signal processing (ISP) pipeline with convolutional neural networks (CNNs). However, the previous CNN-based ISPs, simply referred to as ISP-Nets, have not explicitly considered that images have to be lossy-compressed in most cases, especially by the off-the-shelf JPEG. To address this issue, in this paper, we propose a novel compression-aware deep camera ISP learning framework. At first, we introduce a new use case of compression artifacts simulation network (CAS-Net), which operates in the opposite way of commonly used compression artifacts reduction networks. Then, the CAS-Net is connected with an ISP-Net such that the ISP network can be trained with consideration of image compression. Throughout experimental studies, we show that our compression-aware camera ISP network can produce images with a better tradeoff between bit-rate and image quality compared to its compression-agnostic version when the performance is evaluated after JPEG compression.

AB - Several recent studies have attempted to fully replace the conventional camera image signal processing (ISP) pipeline with convolutional neural networks (CNNs). However, the previous CNN-based ISPs, simply referred to as ISP-Nets, have not explicitly considered that images have to be lossy-compressed in most cases, especially by the off-the-shelf JPEG. To address this issue, in this paper, we propose a novel compression-aware deep camera ISP learning framework. At first, we introduce a new use case of compression artifacts simulation network (CAS-Net), which operates in the opposite way of commonly used compression artifacts reduction networks. Then, the CAS-Net is connected with an ISP-Net such that the ISP network can be trained with consideration of image compression. Throughout experimental studies, we show that our compression-aware camera ISP network can produce images with a better tradeoff between bit-rate and image quality compared to its compression-agnostic version when the performance is evaluated after JPEG compression.

KW - Camera ISP

KW - compression artifacts

KW - convolutional neural network

KW - image compression

UR - http://www.scopus.com/inward/record.url?scp=85116855907&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2021.3116702

DO - 10.1109/ACCESS.2021.3116702

M3 - Article

AN - SCOPUS:85116855907

SN - 2169-3536

VL - 9

SP - 137824

EP - 137832

JO - IEEE Access

JF - IEEE Access

ER -

Image Compression-Aware Deep Camera ISP Network

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this