Learning Non-Parametric Surrogate Losses with Correlated Gradients

Seungdong Yoa, Jinyoung Park, Hyunwoo J. Kim

Research output: Contribution to journalArticlepeer-review


Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods.

Original languageEnglish
Pages (from-to)141199-141209
Number of pages11
JournalIEEE Access
Publication statusPublished - 2021

Bibliographical note

Funding Information:
This work was supported in part by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government Ministry of Science and ICT (MSIT) (Regional Strategic Industry Convergence Security Core Talent Training Business) under Grant 2019-0-01343; in part by the ICT Creative Consilience Program under Grant IITP-2021-2020-0-01819; in part by the Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) and Korea Smart Farm Research and Development Foundation (KosFarm) through Smart Farm Innovation Technology Development Program, funded by the Ministry of Agriculture, Food and Rural Affairs (MAFRA) and MSIT, Rural Development Administration (RDA) under Grant 421025-04; and in part by Samsung Electronics Company Ltd.

Publisher Copyright:
© 2013 IEEE.


  • Learning loss
  • computer vision
  • deep learning
  • machine learning

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Learning Non-Parametric Surrogate Losses with Correlated Gradients'. Together they form a unique fingerprint.

Cite this