Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks

Mohammad Azam Khan, Soonwook Kwon, Jaegul Choo, Seok Min Hong, Sung Hun Kang, Il Ho Park, Sung Kyun Kim, Seok Jin Hong

    Research output: Contribution to journalArticlepeer-review

    65 Citations (Scopus)

    Abstract

    Convolutional neural networks (CNNs), a popular type of deep neural network, have been actively applied to image recognition, object detection, object localization, semantic segmentation, and object instance segmentation. Accordingly, the applicability of deep learning to the analysis of medical images has increased. This paper presents a novel application of state-of-the-art CNN models, such as DenseNet, to the automatic detection of the tympanic membrane (TM) and middle ear (ME) infection. We collected 2,484 oto-endoscopic images (OEIs) and classified them into one of three categories: normal, chronic otitis media (COM) with TM perforation, and otitis media with effusion (OME). Our results indicate that CNN models have significant potential for the automatic recognition of TM and ME infections, demonstrating a competitive accuracy of 95% in classifying TM and middle ear effusion (MEE) from OEIs. In addition to accuracy measurement, our approach achieves nearly perfect measures of 0.99 in terms of the average area under the receiver operating characteristics curve (AUROC). All these results indicate robust performance when recognizing TM and ME effusions in OEIs. Visualization through a class activation mapping (CAM) heatmap demonstrates that our proposed model performs prediction based on the correct region of OEIs. All these outcomes ensure the reliability of our method; hence, the study can aid otolaryngologists and primary care physicians in real-world scenarios.

    Original languageEnglish
    Pages (from-to)384-394
    Number of pages11
    JournalNeural Networks
    Volume126
    DOIs
    Publication statusPublished - 2020 Jun 1

    Bibliographical note

    Copyright © 2020 Elsevier Ltd. All rights reserved.

    Keywords

    • Convolutional neural networks
    • Otitis media
    • Otoscope
    • Tympanic membrane

    ASJC Scopus subject areas

    • Cognitive Neuroscience
    • Artificial Intelligence

    Fingerprint

    Dive into the research topics of 'Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks'. Together they form a unique fingerprint.

    Cite this