Developing an AI literacy diagnostic tool for elementary school students

As artificial intelligence(AI) is utilised throughout society, the need to improve AI literacy as an essential competency, not only for specific experts but also for general citizens, is increasing. Therefore, several studies are being conducted on AI education, and attempts are being made to introduce it into the regular education curriculum. Although AI literacy has been emphasised as a direction for AI education, few assessment tools have been developed to measure the level of literacy to be achieved in stages. In this study, we developed and validated an AI literacy measurement tool for elementary school students. Based on AI education literature, we constructed an AI literacy content system for each school level and conducted two expert Delphi validations (n = 15). Based on this, we developed a set of 24 modular questions, conducted two rounds of expert content validation (n = 15), and preliminarily applied them to 287 sixth-grade students. The results were analysed for reliability based on the KR-20 (Kuder-Richardson Formula 20), difficulty, discrimination based on the item response theory, and correct percentages. Considering the discrimination and correctness results, we modified some items and applied them to an additional 293 sixth graders. The results showed improvements in the discrimination and precision rates. Thus, we concluded that the developed tool is valid and reliable for measuring AI literacy among elementary school students.