Robust auxiliary learning with weighting function for biased data

Deep neural networks easily suffer from weak generalization caused by overfitting on biased data. One popular remedy to alleviate this issue is sample reweighting methods that adaptively adjust the importance of biased samples. Separate from the effort to reduce bias, recent works show that the generalization power can be improved by auxiliary tasks. Inspired by the two lines of works, we extend the sample reweighting methods to auxiliary tasks. In this paper, we propose a novel auxiliary learning framework that improves the primary task by adaptively adjusting the weights of samples from multiple tasks rather than samples from a single task using a weighting function. The weighting function is optimized by meta-learning along the gradient of the loss for meta-data, which is a small unbiased validation data. We also present a task-activation score that indicates the correlation between the learning tendency of the training samples and meta-data samples. This score is utilized as a regularizer for meta-learning objective. Our framework can obtain powerful representations for the primary task on biased data by automatically identifying effective combinations of tasks. Our experiments demonstrate that our proposed method consistently outperforms all baselines and state-of-the-art methods on both corrupted labels and class imbalance settings.