COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models

Jun Hyung Park; Mingyu Lee; Junho Kim; Sang Keun Lee

doi:10.18653/v1/2024.findings-acl.346

COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models

Jun Hyung Park
, Mingyu Lee
, Junho Kim
, Sang Keun Lee

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

In this paper, we introduce COCONUT to effectively guide the contextualization of structured commonsense knowledge based on large language models. COCONUT employs a contextualized knowledge prompting scheme to gather high-quality contextualization examples from a large language model. These examples are subsequently distilled into small language models to enhance their contextualization capability. Extensive evaluations show that COCONUT considerably improves commonsense reasoning performance across diverse benchmarks, models, and settings, exhibiting its flexibility and universality in generating contextualized commonsense knowledge. Notably, COCONUT consistently outperforms the state-of-the-art technique by an average of 5.8%.

Original language	English
Title of host publication	The 62nd Annual Meeting of the Association for Computational Linguistics
Subtitle of host publication	Findings of the Association for Computational Linguistics, ACL 2024
Editors	Lun-Wei Ku, Andre Martins, Vivek Srikumar
Publisher	Association for Computational Linguistics (ACL)
Pages	5815-5830
Number of pages	16
ISBN (Electronic)	9798891760998
DOIs	https://doi.org/10.18653/v1/2024.findings-acl.346
Publication status	Published - 2024
Event	Findings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 - Hybrid, Bangkok, Thailand Duration: 2024 Aug 11 → 2024 Aug 16

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	Findings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024
Country/Territory	Thailand
City	Hybrid, Bangkok
Period	24/8/11 → 24/8/16

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/2024.findings-acl.346

Cite this

Park, J. H., Lee, M., Kim, J., & Lee, S. K. (2024). COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models. In L.-W. Ku, A. Martins, & V. Srikumar (Eds.), The 62nd Annual Meeting of the Association for Computational Linguistics: Findings of the Association for Computational Linguistics, ACL 2024 (pp. 5815-5830). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2024.findings-acl.346

COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models. / Park, Jun Hyung; Lee, Mingyu; Kim, Junho et al.
The 62nd Annual Meeting of the Association for Computational Linguistics: Findings of the Association for Computational Linguistics, ACL 2024. ed. / Lun-Wei Ku; Andre Martins; Vivek Srikumar. Association for Computational Linguistics (ACL), 2024. p. 5815-5830 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Park, JH, Lee, M, Kim, J & Lee, SK 2024, COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models. in L-W Ku, A Martins & V Srikumar (eds), The 62nd Annual Meeting of the Association for Computational Linguistics: Findings of the Association for Computational Linguistics, ACL 2024. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 5815-5830, Findings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024, Hybrid, Bangkok, Thailand, 24/8/11. https://doi.org/10.18653/v1/2024.findings-acl.346

Park JH, Lee M, Kim J, Lee SK. COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models. In Ku LW, Martins A, Srikumar V, editors, The 62nd Annual Meeting of the Association for Computational Linguistics: Findings of the Association for Computational Linguistics, ACL 2024. Association for Computational Linguistics (ACL). 2024. p. 5815-5830. (Proceedings of the Annual Meeting of the Association for Computational Linguistics). doi: 10.18653/v1/2024.findings-acl.346

Park, Jun Hyung ; Lee, Mingyu ; Kim, Junho et al. / COCONUT : Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models. The 62nd Annual Meeting of the Association for Computational Linguistics: Findings of the Association for Computational Linguistics, ACL 2024. editor / Lun-Wei Ku ; Andre Martins ; Vivek Srikumar. Association for Computational Linguistics (ACL), 2024. pp. 5815-5830 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{084f093fbb21465ba829fa42df5417f4,

title = "COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models",

abstract = "In this paper, we introduce COCONUT to effectively guide the contextualization of structured commonsense knowledge based on large language models. COCONUT employs a contextualized knowledge prompting scheme to gather high-quality contextualization examples from a large language model. These examples are subsequently distilled into small language models to enhance their contextualization capability. Extensive evaluations show that COCONUT considerably improves commonsense reasoning performance across diverse benchmarks, models, and settings, exhibiting its flexibility and universality in generating contextualized commonsense knowledge. Notably, COCONUT consistently outperforms the state-of-the-art technique by an average of 5.8\%.",

author = "Park, \{Jun Hyung\} and Mingyu Lee and Junho Kim and Lee, \{Sang Keun\}",

note = "Publisher Copyright: {\textcopyright} 2024 Association for Computational Linguistics.; Findings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 ; Conference date: 11-08-2024 Through 16-08-2024",

year = "2024",

doi = "10.18653/v1/2024.findings-acl.346",

language = "English",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "5815--5830",

editor = "Lun-Wei Ku and Andre Martins and Vivek Srikumar",

booktitle = "The 62nd Annual Meeting of the Association for Computational Linguistics",

}

TY - GEN

T1 - COCONUT

T2 - Findings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024

AU - Park, Jun Hyung

AU - Lee, Mingyu

AU - Kim, Junho

AU - Lee, Sang Keun

PY - 2024

Y1 - 2024

N2 - In this paper, we introduce COCONUT to effectively guide the contextualization of structured commonsense knowledge based on large language models. COCONUT employs a contextualized knowledge prompting scheme to gather high-quality contextualization examples from a large language model. These examples are subsequently distilled into small language models to enhance their contextualization capability. Extensive evaluations show that COCONUT considerably improves commonsense reasoning performance across diverse benchmarks, models, and settings, exhibiting its flexibility and universality in generating contextualized commonsense knowledge. Notably, COCONUT consistently outperforms the state-of-the-art technique by an average of 5.8%.

AB - In this paper, we introduce COCONUT to effectively guide the contextualization of structured commonsense knowledge based on large language models. COCONUT employs a contextualized knowledge prompting scheme to gather high-quality contextualization examples from a large language model. These examples are subsequently distilled into small language models to enhance their contextualization capability. Extensive evaluations show that COCONUT considerably improves commonsense reasoning performance across diverse benchmarks, models, and settings, exhibiting its flexibility and universality in generating contextualized commonsense knowledge. Notably, COCONUT consistently outperforms the state-of-the-art technique by an average of 5.8%.

UR - https://www.scopus.com/pages/publications/85205290269

U2 - 10.18653/v1/2024.findings-acl.346

DO - 10.18653/v1/2024.findings-acl.346

M3 - Conference contribution

AN - SCOPUS:85205290269

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 5815

EP - 5830

BT - The 62nd Annual Meeting of the Association for Computational Linguistics

A2 - Ku, Lun-Wei

A2 - Martins, Andre

A2 - Srikumar, Vivek

PB - Association for Computational Linguistics (ACL)

Y2 - 11 August 2024 through 16 August 2024

ER -

COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this