Examining the impact of adaptive convolution on natural language understanding

Jun Hyung Park, Byung Ju Choi, Sang Keun Lee

    Research output: Contribution to journalArticlepeer-review

    3 Citations (Scopus)

    Abstract

    Convolutional neural networks (CNNs) have shown promising results on many natural language understanding (NLU) tasks owing to their ability to capture informative features in local patches. However, they extract informative patterns in a static manner; they use the same set of filters regardless of different inputs. In this article, we propose an adaptive convolution to provide greater flexibility to traditional CNNs. Unlike the traditional convolution, the adaptive convolution utilizes adaptively generated convolution filters which are conditioned on inputs. We achieve this by attaching filter-generating networks, which are carefully designed to generate input-specific filters, to each convolution block in existing CNNs. We show the efficacy of our approach for existing CNNs through extensive performance evaluations. Our results indicate that adaptive convolutions improve all the baselines, without any exception, by as much as 2.6 percentage points (%p) on sentiment analysis, 1.6 %p on text classification, and 3.6 %p on textual entailment.

    Original languageEnglish
    Article number116044
    JournalExpert Systems With Applications
    Volume189
    DOIs
    Publication statusPublished - 2022 Mar 1

    Bibliographical note

    Publisher Copyright:
    © 2021 Elsevier Ltd

    Keywords

    • Adaptive convolution
    • Natural language understanding
    • Network adaptation

    ASJC Scopus subject areas

    • General Engineering
    • Computer Science Applications
    • Artificial Intelligence

    Fingerprint

    Dive into the research topics of 'Examining the impact of adaptive convolution on natural language understanding'. Together they form a unique fingerprint.

    Cite this