Compression Artifacts Reduction Using Fusion of Multiple Restoration Networks

Sung Jin Cho, Jae Ryun Chung, Seung Wook Kim, Seung Won Jung, Sung Jea Ko

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Lossy video compression achieves coding gains at the expense of the quality loss of the decoded images. Owing to the success of deep learning techniques, especially convolutional neural networks (CNNs), many compression artifacts reduction (CAR) techniques have been used to significantly improve the quality of decoded images by applying CNNs which are trained to predict the original artifact-free images from the decoded images. Most existing video compression standards control the compression ratio using a quantization parameter (QP), so the quality of the decoded images is strongly QP-dependent. Training individual CNNs for predetermined QPs is one of the common approaches to dealing with different levels of compression artifacts. However, compression artifacts are also dependent on the local characteristics of an image. Therefore, a CNN trained for specific QP cannot fully remove the compression artifacts of all images, even those encoded using the same QP. In this paper, we introduce a pixel-precise network selection network (PNSNet). From multiple reconstructed images obtained using multiple QP-specific CAR networks, PNSNet is trained to find the best CAR network for each pixel. The output of PNSNet is then used as an explicit spatial attention channel for an image fusion network that combines multiple reconstructed images. Experimental results demonstrated that the quality of decoded images can be significantly improved by the proposed multiple CAR network fusion method.

Original languageEnglish
Article number9420070
Pages (from-to)66176-66187
Number of pages12
JournalIEEE Access
Volume9
DOIs
Publication statusPublished - 2021

Bibliographical note

Funding Information:
This work was supported by the Korea Medical Device Development Fund Grant by the Korean Government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health and Welfare, and the Ministry of Food and Drug Safety) under Project 202012D01.

Publisher Copyright:
© 2013 IEEE.

Keywords

  • Compression artifacts reduction
  • convolutional neural network
  • deep learning
  • network fusion
  • video compression

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint

Dive into the research topics of 'Compression Artifacts Reduction Using Fusion of Multiple Restoration Networks'. Together they form a unique fingerprint.

Cite this