Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment

Sebastian Bosse, Dominique Maniry, Klaus Robert Müller, Thomas Wiegand, Wojciech Samek

Research output: Contribution to journalArticlepeer-review

801 Citations (Scopus)

Abstract

We present a deep neural network-based approach to image quality assessment (IQA). The network is trained end-to-end and comprises ten convolutional layers and five pooling layers for feature extraction, and two fully connected layers for regression, which makes it significantly deeper than related IQA models. Unique features of the proposed architecture are that: 1) with slight adaptations it can be used in a no-reference (NR) as well as in a full-reference (FR) IQA setting and 2) it allows for joint learning of local quality and local weights, i.e., relative importance of local quality to the global quality estimate, in an unified framework. Our approach is purely data-driven and does not rely on hand-crafted features or other types of prior domain knowledge about the human visual system or image statistics. We evaluate the proposed approach on the LIVE, CISQ, and TID2013 databases as well as the LIVE In the wild image quality challenge database and show superior performance to state-of-the-art NR and FR IQA methods. Finally, cross-database evaluation shows a high ability to generalize between different databases, indicating a high robustness of the learned features.

Original languageEnglish
Article number8063957
Pages (from-to)206-219
Number of pages14
JournalIEEE Transactions on Image Processing
Volume27
Issue number1
DOIs
Publication statusPublished - 2018 Jan

Bibliographical note

Funding Information:
This work was supported in part by the German Ministry for Education and Research as Berlin Big Data Center under Grant 01IS14013A, in part by the Institute for Information and Communications Technology Promotion through the Korea Government under Grant 2017-0-00451, and in part by DFG. The work of K.-R. Müller was supported by the National Research Foundation of Korea through the Ministry of Education, Science, and Technology in the BK21 Program. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Kalpana Seshadrinathan.

Publisher Copyright:
© 2017 IEEE.

Keywords

  • Full-reference image quality assessment
  • deep learning
  • feature extraction
  • neural networks
  • no-reference image quality assessment
  • quality pooling
  • regression

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment'. Together they form a unique fingerprint.

Cite this