Learning Non-Parametric Surrogate Losses with Correlated Gradients

Seungdong Yoa; Jinyoung Park; Hyunwoo J. Kim

doi:10.1109/ACCESS.2021.3120092

Learning Non-Parametric Surrogate Losses with Correlated Gradients

Seungdong Yoa, Jinyoung Park, Hyunwoo J. Kim

Research output: Contribution to journal › Article › peer-review

Abstract

Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods.

Original language	English
Pages (from-to)	141199-141209
Number of pages	11
Journal	IEEE Access
Volume	9
DOIs	https://doi.org/10.1109/ACCESS.2021.3120092
Publication status	Published - 2021

Bibliographical note

Funding Information:
This work was supported in part by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government Ministry of Science and ICT (MSIT) (Regional Strategic Industry Convergence Security Core Talent Training Business) under Grant 2019-0-01343; in part by the ICT Creative Consilience Program under Grant IITP-2021-2020-0-01819; in part by the Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) and Korea Smart Farm Research and Development Foundation (KosFarm) through Smart Farm Innovation Technology Development Program, funded by the Ministry of Agriculture, Food and Rural Affairs (MAFRA) and MSIT, Rural Development Administration (RDA) under Grant 421025-04; and in part by Samsung Electronics Company Ltd.

Publisher Copyright:
© 2013 IEEE.

Keywords

Learning loss
computer vision
deep learning
machine learning

ASJC Scopus subject areas

General Computer Science
General Materials Science
General Engineering

Access to Document

10.1109/ACCESS.2021.3120092

Cite this

@article{7f449a44882f4d3480c6b739e22988c4,

title = "Learning Non-Parametric Surrogate Losses with Correlated Gradients",

abstract = "Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods.",

keywords = "Learning loss, computer vision, deep learning, machine learning",

author = "Seungdong Yoa and Jinyoung Park and Kim, {Hyunwoo J.}",

note = "Funding Information: This work was supported in part by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government Ministry of Science and ICT (MSIT) (Regional Strategic Industry Convergence Security Core Talent Training Business) under Grant 2019-0-01343; in part by the ICT Creative Consilience Program under Grant IITP-2021-2020-0-01819; in part by the Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) and Korea Smart Farm Research and Development Foundation (KosFarm) through Smart Farm Innovation Technology Development Program, funded by the Ministry of Agriculture, Food and Rural Affairs (MAFRA) and MSIT, Rural Development Administration (RDA) under Grant 421025-04; and in part by Samsung Electronics Company Ltd. Publisher Copyright: {\textcopyright} 2013 IEEE.",

year = "2021",

doi = "10.1109/ACCESS.2021.3120092",

language = "English",

volume = "9",

pages = "141199--141209",

journal = "IEEE Access",

issn = "2169-3536",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Learning Non-Parametric Surrogate Losses with Correlated Gradients

AU - Yoa, Seungdong

AU - Park, Jinyoung

AU - Kim, Hyunwoo J.

N1 - Funding Information: This work was supported in part by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government Ministry of Science and ICT (MSIT) (Regional Strategic Industry Convergence Security Core Talent Training Business) under Grant 2019-0-01343; in part by the ICT Creative Consilience Program under Grant IITP-2021-2020-0-01819; in part by the Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) and Korea Smart Farm Research and Development Foundation (KosFarm) through Smart Farm Innovation Technology Development Program, funded by the Ministry of Agriculture, Food and Rural Affairs (MAFRA) and MSIT, Rural Development Administration (RDA) under Grant 421025-04; and in part by Samsung Electronics Company Ltd. Publisher Copyright: © 2013 IEEE.

PY - 2021

Y1 - 2021

N2 - Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods.

AB - Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods.

KW - Learning loss

KW - computer vision

KW - deep learning

KW - machine learning

UR - http://www.scopus.com/inward/record.url?scp=85117762031&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2021.3120092

DO - 10.1109/ACCESS.2021.3120092

M3 - Article

AN - SCOPUS:85117762031

SN - 2169-3536

VL - 9

SP - 141199

EP - 141209

JO - IEEE Access

JF - IEEE Access

ER -

Learning Non-Parametric Surrogate Losses with Correlated Gradients

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this