Abstract
Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org.
Original language | English |
---|---|
Title of host publication | 2019 IEEE Visualization Conference, VIS 2019 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 146-150 |
Number of pages | 5 |
ISBN (Electronic) | 9781728149417 |
DOIs | |
Publication status | Published - 2019 Oct |
Event | 2019 IEEE Visualization Conference, VIS 2019 - Vancouver, Canada Duration: 2019 Oct 20 → 2019 Oct 25 |
Publication series
Name | 2019 IEEE Visualization Conference, VIS 2019 |
---|
Conference
Conference | 2019 IEEE Visualization Conference, VIS 2019 |
---|---|
Country/Territory | Canada |
City | Vancouver |
Period | 19/10/20 → 19/10/25 |
Bibliographical note
Publisher Copyright:© 2019 IEEE.
Keywords
- Deep neural networks
- interpretability
- natural language processing
- self-attention networks
- visual analytics
ASJC Scopus subject areas
- Computer Graphics and Computer-Aided Design
- Media Technology
- Modelling and Simulation