TY - JOUR
T1 - Examining the impact of adaptive convolution on natural language understanding
AU - Park, Jun Hyung
AU - Choi, Byung Ju
AU - Lee, Sang Keun
N1 - Funding Information:
We thank the anonymous reviewers for their helpful comments. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) ( 2021R1A2C30 10430 ) and the Basic Research Program through the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) ( 2020R1A4A1018309 ).
Publisher Copyright:
© 2021 Elsevier Ltd
PY - 2022/3/1
Y1 - 2022/3/1
N2 - Convolutional neural networks (CNNs) have shown promising results on many natural language understanding (NLU) tasks owing to their ability to capture informative features in local patches. However, they extract informative patterns in a static manner; they use the same set of filters regardless of different inputs. In this article, we propose an adaptive convolution to provide greater flexibility to traditional CNNs. Unlike the traditional convolution, the adaptive convolution utilizes adaptively generated convolution filters which are conditioned on inputs. We achieve this by attaching filter-generating networks, which are carefully designed to generate input-specific filters, to each convolution block in existing CNNs. We show the efficacy of our approach for existing CNNs through extensive performance evaluations. Our results indicate that adaptive convolutions improve all the baselines, without any exception, by as much as 2.6 percentage points (%p) on sentiment analysis, 1.6 %p on text classification, and 3.6 %p on textual entailment.
AB - Convolutional neural networks (CNNs) have shown promising results on many natural language understanding (NLU) tasks owing to their ability to capture informative features in local patches. However, they extract informative patterns in a static manner; they use the same set of filters regardless of different inputs. In this article, we propose an adaptive convolution to provide greater flexibility to traditional CNNs. Unlike the traditional convolution, the adaptive convolution utilizes adaptively generated convolution filters which are conditioned on inputs. We achieve this by attaching filter-generating networks, which are carefully designed to generate input-specific filters, to each convolution block in existing CNNs. We show the efficacy of our approach for existing CNNs through extensive performance evaluations. Our results indicate that adaptive convolutions improve all the baselines, without any exception, by as much as 2.6 percentage points (%p) on sentiment analysis, 1.6 %p on text classification, and 3.6 %p on textual entailment.
KW - Adaptive convolution
KW - Natural language understanding
KW - Network adaptation
UR - http://www.scopus.com/inward/record.url?scp=85118491236&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2021.116044
DO - 10.1016/j.eswa.2021.116044
M3 - Article
AN - SCOPUS:85118491236
SN - 0957-4174
VL - 189
JO - Expert Systems with Applications
JF - Expert Systems with Applications
M1 - 116044
ER -