TY - JOUR
T1 - PU-GEN
T2 - Enhancing generative commonsense reasoning for language models with human-centered knowledge
AU - Seo, Jaehyung
AU - Oh, Dongsuk
AU - Eo, Sugyeong
AU - Park, Chanjun
AU - Yang, Kisu
AU - Moon, Hyeonseok
AU - Park, Kinam
AU - Lim, Heuiseok
N1 - Funding Information:
This research was supported by the Ministry of Science and ICT (MSIT), Korea , under the ICT Creative Consilience Program ( IITP-2022-2020-0-01819 ) supervised by the Institute for Information & Communications Technology Planning & Evaluation (IITP). This work was supported by an IITP grant funded by the Korean government (MSIT) (No. 2020-0-00368 , A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques). This research was also supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) , funded by the Ministry of Education ( NRF-2021R1A6A1A03045425 ).
Publisher Copyright:
© 2022 Elsevier B.V.
PY - 2022/11/28
Y1 - 2022/11/28
N2 - Generative commonsense reasoning refers to the ability of a language model to generate a sentence with a given concept-set based on compositional generalization and commonsense reasoning. In the CommonGen challenge, which evaluates the capability of generative commonsense reasoning, language models continue to exhibit low performances and struggle to leverage knowledge representation from humans. Therefore, we propose PU-GEN to leverage human-centered knowledge in language models to enhance compositional generalization and commonsense reasoning considering the human language generation process. To incorporate human-centered knowledge, PU-GEN reinterprets two linguistic philosophies from Wittgenstein: picture theory and use theory. First, we retrieve scene knowledge to reflect picture theory such that a model can describe a general situation as if it were being painted. Second, we extend relational knowledge to consider use theory for understanding various contexts. PU-GEN demonstrates superior performance in qualitative and quantitative evaluations over baseline models in CommonGen and generates convincing evidence for CommonsenseQA. Moreover, it outperforms the state-of-the-art model used in the previous CommonGen challenge.
AB - Generative commonsense reasoning refers to the ability of a language model to generate a sentence with a given concept-set based on compositional generalization and commonsense reasoning. In the CommonGen challenge, which evaluates the capability of generative commonsense reasoning, language models continue to exhibit low performances and struggle to leverage knowledge representation from humans. Therefore, we propose PU-GEN to leverage human-centered knowledge in language models to enhance compositional generalization and commonsense reasoning considering the human language generation process. To incorporate human-centered knowledge, PU-GEN reinterprets two linguistic philosophies from Wittgenstein: picture theory and use theory. First, we retrieve scene knowledge to reflect picture theory such that a model can describe a general situation as if it were being painted. Second, we extend relational knowledge to consider use theory for understanding various contexts. PU-GEN demonstrates superior performance in qualitative and quantitative evaluations over baseline models in CommonGen and generates convincing evidence for CommonsenseQA. Moreover, it outperforms the state-of-the-art model used in the previous CommonGen challenge.
KW - Commonsense reasoning
KW - Human-centered knowledge
KW - Language model
KW - Text generation
UR - http://www.scopus.com/inward/record.url?scp=85138031965&partnerID=8YFLogxK
U2 - 10.1016/j.knosys.2022.109861
DO - 10.1016/j.knosys.2022.109861
M3 - Article
AN - SCOPUS:85138031965
SN - 0950-7051
VL - 256
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 109861
ER -