Few-shot is enough: exploring ChatGPT prompt engineering method for automatic question generation in english education

Unggi Lee, Haewon Jung, Younghoon Jeon, Younghoon Sohn, Wonhee Hwang, Jewoong Moon, Hyeoncheol Kim

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)

Abstract

Through design and development research (DDR), we aimed to create a validated automatic question generation (AQG) system using large language models (LLMs) like ChatGPT, enhanced by prompting engineering techniques. While AQG has become increasingly integral to online learning for its efficiency in generating questions, issues such as inconsistent question quality and the absence of transparent and validated evaluation methods persist. Our research focused on creating a prompt engineering protocol tailored for AQG. This protocol underwent several iterations of refinement and validation to improve its performance. By gathering validation scores and qualitative feedback on the produced questions and the system’s framework, we examined the effectiveness of the system. The study findings indicate that our combined use of LLMs and prompt engineering in AQG produces questions with statistically significant validity. Our research further illuminates academic and design considerations for AQG design in English education: (a) certain question types might not be optimal for generation via ChatGPT, (b) ChatGPT sheds light on the potential for collaborative AI-teacher efforts in question generation, especially within English education.

Original languageEnglish
JournalEducation and Information Technologies
DOIs
Publication statusAccepted/In press - 2023

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

  • Automatic Question Generation
  • ChatGPT
  • English Education
  • Large Language Model
  • Prompt Engineering

ASJC Scopus subject areas

  • Education
  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'Few-shot is enough: exploring ChatGPT prompt engineering method for automatic question generation in english education'. Together they form a unique fingerprint.

Cite this