Identifying individual facial expressions by deconstructing a neural network

Farhad Arbabzadah, Grégoire Montavon, Klaus Robert Müller, Wojciech Samek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Citations (Scopus)


This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.

Original languageEnglish
Title of host publicationPattern Recognition - 38th German Conference, GCPR 2016, Proceedings
EditorsBjoern Andres, Bodo Rosenhahn
PublisherSpringer Verlag
Number of pages11
ISBN (Print)9783319458854
Publication statusPublished - 2016
Event38th German Conference on Pattern Recognition, GCPR 2016 - Hannover, Germany
Duration: 2016 Sept 122016 Sept 15

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9796 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other38th German Conference on Pattern Recognition, GCPR 2016

Bibliographical note

Funding Information:
This work was supported by the German Ministry for Education and Research as Berlin Big Data Center BBDC (01IS14013A), the Deutsche Forschungsgesellschaft (MU 987/19-1) and the Brain Korea 21 Plus Program through the National Research Foundation of Korea funded by the Ministry of Education. Correspondence to KRM and WS.

Publisher Copyright:
© Springer International Publishing AG 2016.

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Identifying individual facial expressions by deconstructing a neural network'. Together they form a unique fingerprint.

Cite this