Feature-based interpretation of the deep neural network

Eun Hun Lee, Hyeoncheol Kim

Research output: Contribution to journalArticlepeer-review


The significant advantage of deep neural networks is that the upper layer can capture the high-level features of data based on the information acquired from the lower layer by stacking layers deeply. Since it is challenging to interpret what knowledge the neural network has learned, various studies for explaining neural networks have emerged to overcome this problem. However, these studies generate the local explanation of a single instance rather than providing a generalized global interpretation of the neural network model itself. To overcome such drawbacks of the previous approaches, we propose the global interpretation method for the deep neural network through features of the model. We first analyzed the relationship between the input and hidden layers to represent the high-level features of the model, then interpreted the decision-making process of neural networks through high-level features. In addition, we applied network pruning techniques to make concise explanations and analyzed the effect of layer complexity on interpretability. We present experiments on the proposed approach using three different datasets and show that our approach could generate global explanations on deep neural network models with high accuracy and fidelity.

Original languageEnglish
Article number2687
JournalElectronics (Switzerland)
Issue number21
Publication statusPublished - 2021 Nov 1


  • Explainable artificial intelligence (XAI)
  • Interpretability
  • Neural network

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Hardware and Architecture
  • Computer Networks and Communications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Feature-based interpretation of the deep neural network'. Together they form a unique fingerprint.

Cite this