Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference

Yong Ho Jung; Jun Hyung Park; Joon Young Choi; Mingyu Lee; Junho Kim; Kang Min Kim; Sang Keun Lee

Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference

Yong Ho Jung, Jun Hyung Park, Joon Young Choi, Mingyu Lee, Junho Kim, Kang Min Kim, Sang Keun Lee

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Commonsense inference poses a unique challenge to reason and generate the physical, social, and causal conditions of a given event. Existing approaches to commonsense inference utilize commonsense transformers, which are large-scale language models that learn commonsense knowledge graphs. However, they suffer from a lack of coverage and expressive diversity of the graphs, resulting in a degradation of the representation quality. In this paper, we focus on addressing missing relations in commonsense knowledge graphs, and propose a novel contrastive learning framework called SOLAR. Our framework contrasts sets of semantically similar and dissimilar events, learning richer inferential knowledge compared to existing approaches. Empirical results demonstrate the efficacy of SOLAR in commonsense inference of diverse commonsense knowledge graphs. Specifically, SOLAR outperforms the state-of-the-art commonsense transformer on commonsense inference with ConceptNet by 1.84% on average among 8 automatic evaluation metrics. In-depth analysis of SOLAR sheds light on the effects of the missing relations utilized in learning commonsense knowledge graphs.

Original language	English
Title of host publication	ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022
Editors	Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Publisher	Association for Computational Linguistics (ACL)
Pages	1514-1523
Number of pages	10
ISBN (Electronic)	9781955917254
Publication status	Published - 2022
Event	60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 - Dublin, Ireland Duration: 2022 May 22 → 2022 May 27

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	60th Annual Meeting of the Association for Computational Linguistics, ACL 2022
Country/Territory	Ireland
City	Dublin
Period	22/5/22 → 22/5/27

Bibliographical note

Publisher Copyright:
© 2022 Association for Computational Linguistics.

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Cite this

Jung, Y. H., Park, J. H., Choi, J. Y., Lee, M., Kim, J., Kim, K. M., & Lee, S. K. (2022). Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference. In S. Muresan, P. Nakov, & A. Villavicencio (Eds.), ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022 (pp. 1514-1523). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL).

Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference. / Jung, Yong Ho; Park, Jun Hyung; Choi, Joon Young et al.
ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022. ed. / Smaranda Muresan; Preslav Nakov; Aline Villavicencio. Association for Computational Linguistics (ACL), 2022. p. 1514-1523 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jung, YH, Park, JH, Choi, JY, Lee, M, Kim, J, Kim, KM & Lee, SK 2022, Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference. in S Muresan, P Nakov & A Villavicencio (eds), ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 1514-1523, 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022, Dublin, Ireland, 22/5/22.

Jung YH, Park JH, Choi JY, Lee M, Kim J, Kim KM et al. Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference. In Muresan S, Nakov P, Villavicencio A, editors, ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022. Association for Computational Linguistics (ACL). 2022. p. 1514-1523. (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Jung, Yong Ho ; Park, Jun Hyung ; Choi, Joon Young et al. / Learning from Missing Relations : Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference. ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022. editor / Smaranda Muresan ; Preslav Nakov ; Aline Villavicencio. Association for Computational Linguistics (ACL), 2022. pp. 1514-1523 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{5ef5fc1154544fff84e3242bff4ff285,

title = "Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference",

abstract = "Commonsense inference poses a unique challenge to reason and generate the physical, social, and causal conditions of a given event. Existing approaches to commonsense inference utilize commonsense transformers, which are large-scale language models that learn commonsense knowledge graphs. However, they suffer from a lack of coverage and expressive diversity of the graphs, resulting in a degradation of the representation quality. In this paper, we focus on addressing missing relations in commonsense knowledge graphs, and propose a novel contrastive learning framework called SOLAR. Our framework contrasts sets of semantically similar and dissimilar events, learning richer inferential knowledge compared to existing approaches. Empirical results demonstrate the efficacy of SOLAR in commonsense inference of diverse commonsense knowledge graphs. Specifically, SOLAR outperforms the state-of-the-art commonsense transformer on commonsense inference with ConceptNet by 1.84% on average among 8 automatic evaluation metrics. In-depth analysis of SOLAR sheds light on the effects of the missing relations utilized in learning commonsense knowledge graphs.",

author = "Jung, {Yong Ho} and Park, {Jun Hyung} and Choi, {Joon Young} and Mingyu Lee and Junho Kim and Kim, {Kang Min} and Lee, {Sang Keun}",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 ; Conference date: 22-05-2022 Through 27-05-2022",

year = "2022",

language = "English",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "1514--1523",

editor = "Smaranda Muresan and Preslav Nakov and Aline Villavicencio",

booktitle = "ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022",

}

TY - GEN

T1 - Learning from Missing Relations

T2 - 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022

AU - Jung, Yong Ho

AU - Park, Jun Hyung

AU - Choi, Joon Young

AU - Lee, Mingyu

AU - Kim, Junho

AU - Kim, Kang Min

AU - Lee, Sang Keun

PY - 2022

Y1 - 2022

N2 - Commonsense inference poses a unique challenge to reason and generate the physical, social, and causal conditions of a given event. Existing approaches to commonsense inference utilize commonsense transformers, which are large-scale language models that learn commonsense knowledge graphs. However, they suffer from a lack of coverage and expressive diversity of the graphs, resulting in a degradation of the representation quality. In this paper, we focus on addressing missing relations in commonsense knowledge graphs, and propose a novel contrastive learning framework called SOLAR. Our framework contrasts sets of semantically similar and dissimilar events, learning richer inferential knowledge compared to existing approaches. Empirical results demonstrate the efficacy of SOLAR in commonsense inference of diverse commonsense knowledge graphs. Specifically, SOLAR outperforms the state-of-the-art commonsense transformer on commonsense inference with ConceptNet by 1.84% on average among 8 automatic evaluation metrics. In-depth analysis of SOLAR sheds light on the effects of the missing relations utilized in learning commonsense knowledge graphs.

AB - Commonsense inference poses a unique challenge to reason and generate the physical, social, and causal conditions of a given event. Existing approaches to commonsense inference utilize commonsense transformers, which are large-scale language models that learn commonsense knowledge graphs. However, they suffer from a lack of coverage and expressive diversity of the graphs, resulting in a degradation of the representation quality. In this paper, we focus on addressing missing relations in commonsense knowledge graphs, and propose a novel contrastive learning framework called SOLAR. Our framework contrasts sets of semantically similar and dissimilar events, learning richer inferential knowledge compared to existing approaches. Empirical results demonstrate the efficacy of SOLAR in commonsense inference of diverse commonsense knowledge graphs. Specifically, SOLAR outperforms the state-of-the-art commonsense transformer on commonsense inference with ConceptNet by 1.84% on average among 8 automatic evaluation metrics. In-depth analysis of SOLAR sheds light on the effects of the missing relations utilized in learning commonsense knowledge graphs.

UR - http://www.scopus.com/inward/record.url?scp=85140203309&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85140203309

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 1514

EP - 1523

BT - ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Findings of ACL 2022

A2 - Muresan, Smaranda

A2 - Nakov, Preslav

A2 - Villavicencio, Aline

PB - Association for Computational Linguistics (ACL)

Y2 - 22 May 2022 through 27 May 2022

ER -

Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this