Association analysis in item response datasets

Association rule mining is a data mining technique used to find frequent patterns in a huge dataset. In this paper, we address the issues of its application to item response datasets, which is generally high multidimensional. The primary disadvantage about mining association rules in a high multidimensional dataset is the huge number of patterns that are discovered, most of which are trivial or uninteresting. In this paper, we introduce a new measure called suprisal that estimates the informativeness of transactional instances and attributes. Our approach to the item association analysis includes elimination of noisy and uninformative data using the surprisal first, and then generation of association rules of good quality. Experimental results on real datasets of national-level tests for Korean high school student show that the surprisal-based pruning improves quality of association rules in item response datasets significantly.