MonaCoBERT: Monotonic Attention Based ConvBERT for Knowledge Tracing

Knowledge tracing (KT) is a research area of predicting students’ knowledge states using their interaction data, such as concepts, questions, and responses. Most deep learning-based KT models have suffered from attributions of KT datasets such as the data sparsity, changeability of the knowledge state, and educational domain. Recently, most KT models use attention mechanisms to solve these problems. However, few studies tried to redesign the attention mechanism to restrict coverage of the local receptive field, for which the model can optimize to find the latent representation locally and globally in the students’ interaction. In this study, we propose MonaCoBERT, with a redesigned attention mechanism by combining monotonic attention (MA) and span-based dynamic convolution (SDC), in order to represent global and local features together and to apply students’ forgetting. As a result, MonaCoBERT achieves remarkable performance on most benchmark datasets. In addition, we used a classical test-theory-based embedding strategy to reflect the difficulty degree of knowledge concepts. We conducted ablation studies and further analysis to explain the remarkable performance of our model quantitatively. The analysis results demonstrate that SDC and MA complement one another. We also demonstrate that our model represents the relationship between concepts.