Abstract
Self-supervised learning methods have shown excellent performance in improving the performance of existing networks by learning visual representations from large amounts of unlabeled data. In this paper, we propose a end-to-end multi-task self-supervision method for vision transformer. The network is given two task: inpainting, position prediction. Given a masked image, the network predicts the missing pixel information and also predicts the position of the given puzzle patches. Through classification experiment, we demonstrate that the proposed method improves performance of the network compared to the direct supervised learning method.
Original language | English |
---|---|
Title of host publication | 2022 22nd International Conference on Control, Automation and Systems, ICCAS 2022 |
Publisher | IEEE Computer Society |
Pages | 1963-1965 |
Number of pages | 3 |
ISBN (Electronic) | 9788993215243 |
DOIs | |
Publication status | Published - 2022 |
Event | 22nd International Conference on Control, Automation and Systems, ICCAS 2022 - Busan, Korea, Republic of Duration: 2022 Nov 27 → 2022 Dec 1 |
Publication series
Name | International Conference on Control, Automation and Systems |
---|---|
Volume | 2022-November |
ISSN (Print) | 1598-7833 |
Conference
Conference | 22nd International Conference on Control, Automation and Systems, ICCAS 2022 |
---|---|
Country/Territory | Korea, Republic of |
City | Busan |
Period | 22/11/27 → 22/12/1 |
Bibliographical note
Publisher Copyright:© 2022 ICROS.
Keywords
- Deep Learning
- Self-Supervision
- Vision Transformer
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Science Applications
- Control and Systems Engineering
- Electrical and Electronic Engineering