Robust auxiliary learning with weighting function for biased data

Dasol Hwang, Sojin Lee, Joonmyung Choi, Je Keun Rhee, Hyunwoo J. Kim

Research output: Contribution to journalArticlepeer-review

Abstract

Deep neural networks easily suffer from weak generalization caused by overfitting on biased data. One popular remedy to alleviate this issue is sample reweighting methods that adaptively adjust the importance of biased samples. Separate from the effort to reduce bias, recent works show that the generalization power can be improved by auxiliary tasks. Inspired by the two lines of works, we extend the sample reweighting methods to auxiliary tasks. In this paper, we propose a novel auxiliary learning framework that improves the primary task by adaptively adjusting the weights of samples from multiple tasks rather than samples from a single task using a weighting function. The weighting function is optimized by meta-learning along the gradient of the loss for meta-data, which is a small unbiased validation data. We also present a task-activation score that indicates the correlation between the learning tendency of the training samples and meta-data samples. This score is utilized as a regularizer for meta-learning objective. Our framework can obtain powerful representations for the primary task on biased data by automatically identifying effective combinations of tasks. Our experiments demonstrate that our proposed method consistently outperforms all baselines and state-of-the-art methods on both corrupted labels and class imbalance settings.

Original languageEnglish
Pages (from-to)307-319
Number of pages13
JournalInformation Sciences
Volume628
DOIs
Publication statusPublished - 2023 May

Bibliographical note

Funding Information:
This research was supported by Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) and Korea Smart Farm Research and Development Foundation (KosFarm) through the Smart Farm Innovation Technology Development Program by the Ministry of Agriculture, Food and Rural Affairs (MAFRA), and MSIT, Rural Development Administration (RDA), under Grant 421025–04; the Korea Health Technology R&D Project, through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health and Welfare, Republic of Korea, under Grant HR20C0021; ICT Creative Consilience program (IITP-2023-2020-0-01819) supervised by the IITP; and KakaoBrain corporation.

Publisher Copyright:
© 2023 The Authors

Keywords

  • Auxiliary learning
  • Biased data
  • Meta-learning
  • Representation learning

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Robust auxiliary learning with weighting function for biased data'. Together they form a unique fingerprint.

Cite this