TY - GEN
T1 - SANVis
T2 - 2019 IEEE Visualization Conference, VIS 2019
AU - Park, Cheonbok
AU - Choo, Jaegul
AU - Na, Inyoup
AU - Jo, Yongjang
AU - Shin, Sungbok
AU - Yoo, Jaehyo
AU - Kwon, Bum Chul
AU - Zhao, Jian
AU - Noh, Hyungjong
AU - Lee, Yeonsoo
N1 - Funding Information:
The authors wish to thank all reviewers who provided constructive feedback for our project. This work was partially supported by NCSOFT NLP Center. This work was also supported by the National Research Foundation of Korea (NRF) grant funded bythe Korean government (MSIP) (No. NRF-2018M3E3A1057305).
Publisher Copyright:
© 2019 IEEE.
PY - 2019/10
Y1 - 2019/10
N2 - Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org.
AB - Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org.
KW - Deep neural networks
KW - interpretability
KW - natural language processing
KW - self-attention networks
KW - visual analytics
UR - http://www.scopus.com/inward/record.url?scp=85077997739&partnerID=8YFLogxK
U2 - 10.1109/VISUAL.2019.8933677
DO - 10.1109/VISUAL.2019.8933677
M3 - Conference contribution
AN - SCOPUS:85077997739
T3 - 2019 IEEE Visualization Conference, VIS 2019
SP - 146
EP - 150
BT - 2019 IEEE Visualization Conference, VIS 2019
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 20 October 2019 through 25 October 2019
ER -