Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks

Mohammad Azam Khan, Soonwook Kwon, Jaegul Choo, Seok Min Hong, Sung Hun Kang, Il Ho Park, Sung Kyun Kim, Seok Jin Hong

Research output: Contribution to journalArticlepeer-review

53 Citations (Scopus)


Convolutional neural networks (CNNs), a popular type of deep neural network, have been actively applied to image recognition, object detection, object localization, semantic segmentation, and object instance segmentation. Accordingly, the applicability of deep learning to the analysis of medical images has increased. This paper presents a novel application of state-of-the-art CNN models, such as DenseNet, to the automatic detection of the tympanic membrane (TM) and middle ear (ME) infection. We collected 2,484 oto-endoscopic images (OEIs) and classified them into one of three categories: normal, chronic otitis media (COM) with TM perforation, and otitis media with effusion (OME). Our results indicate that CNN models have significant potential for the automatic recognition of TM and ME infections, demonstrating a competitive accuracy of 95% in classifying TM and middle ear effusion (MEE) from OEIs. In addition to accuracy measurement, our approach achieves nearly perfect measures of 0.99 in terms of the average area under the receiver operating characteristics curve (AUROC). All these results indicate robust performance when recognizing TM and ME effusions in OEIs. Visualization through a class activation mapping (CAM) heatmap demonstrates that our proposed model performs prediction based on the correct region of OEIs. All these outcomes ensure the reliability of our method; hence, the study can aid otolaryngologists and primary care physicians in real-world scenarios.

Original languageEnglish
Pages (from-to)384-394
Number of pages11
JournalNeural Networks
Publication statusPublished - 2020 Jun 1

Bibliographical note

Copyright © 2020 Elsevier Ltd. All rights reserved.


  • Convolutional neural networks
  • Otitis media
  • Otoscope
  • Tympanic membrane

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence


Dive into the research topics of 'Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks'. Together they form a unique fingerprint.

Cite this