AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
Exercise-Enhanced Sequential Modeling for Student Performance Prediction.
THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTI..., pp.2435-2443, (2018)
- Online education systems, such as massive open online course (MOOC) and intelligent tutoring system (ITS), provide students with open access for self-learning.
- Their prevalence and convenience have attracted great attentions from both educators and general publics (Anderson et al 2014).
- In order to offer students proactive services for their self-improvement, e.g., learning remedy suggestion and personalized exercise recommendation (Kuh et al 2011), a crucial demand is to predict their performance, i.e., forecasting whether or not a student could answer the exercise (e.g., e5) correctly in the future (Baker and Yacef 2009)
- Online education systems, such as massive open online course (MOOC) and intelligent tutoring system (ITS), provide students with open access for self-learning
- The results indicate that Enhanced Recurrent Neural Network (EERNN) framework can make full use of exercising records and exercise texts, benefiting the prediction
- Models with Attention mechanism (EERNNA, LSTMA) outperform those with Markov property (EERNNM, LSTMM), which demonstrates that it is effective to track focused student embeddings based on similar exercises for the prediction
- We guess a possible reason is that all these RNN based models can capture the change of student exercise process, where the deep neural network structures are suitable for student performance prediction
- As mentioned in Section 3, we hold that EERNNA with Attention mechanism can track the focused states of students during the student exercising process to improve prediction performance, which is superior to EERNNM
- We presented a novel Exercise-Enhanced Recurrent Neural Network (EERNN) framework to predict student future performance by taking full advantage of student exercising records and the texts of exercises
- The authors conduct extensive experiments to demonstrate the effectiveness of EERNN from various aspects: (1) the prediction performance of EERNN against the baselines in both future and cold-start scenarios; (2) the attention effectiveness in EERNN; (3) meaningful visualization.
The experimental dataset supplied by iFLYTEK Co., Ltd. is collected from Zhixue1, a widely-used online learning system, which provides senior high school students with a large exercise resources for exercising.
- The authors conduct experiments on the mathematical data records because the mathematical dataset is currently the largest in the system.
- Note that most exercises contain less than 2 knowledge concepts, and 54 exercises on average are related to one specific knowledge concept.
- These observations prove that the way to represent exercises as knowledge concepts cannot distinguish differences among exercises, causing some information loss
- Models with Attention mechanism (EERNNA, LSTMA) outperform those with Markov property (EERNNM, LSTMM), which demonstrates that it is effective to track focused student embeddings based on similar exercises for the prediction.
- Both EERNNA and EERNNM generate better result than their variants (LSTMA, LSTMM) and DKT, showing the effectiveness of Exercise Embedding.
- The authors guess a possible reason is that all these RNN based models can capture the change of student exercise process, where the deep neural network structures are suitable for student performance prediction
- The authors presented a novel Exercise-Enhanced Recurrent Neural Network (EERNN) framework to predict student future performance by taking full advantage of student exercising records and the texts of exercises.
- For modeling student exercising process, the authors first designed a BiLSTM to extract exercise semantic representations from texts without any expertise and information loss.
- The authors proposed another LSTM architecture to trace student states by embedding exercise encodings.
- EERNNA can track the focused information for making prediction, which is superior to EERNNM.
- Extensive experiments on a large-scale real-world dataset demonstrated the effectiveness of EERNN framework and
- Table1: The statistics of mathematics dataset
- The related work can be classified into following categories, i,e., Cognitive Diagnosis, Knowledge Tracing, Matrix Factorization and Deep learning researches.
Cognitive Diagnosis. In the domain of educational psychology, cognitive diagnosis is a technique to predict student performance by discovering student states from their exercising records (DiBello, Roussos, and Stout 2006). Traditional cognitive diagnostic models (CDM) could be grouped into two parts: continuous ones and discrete ones. Among them, item response theory (IRT), as a typical continuous model, characterized each student by a variable from a logistic-like function (Embretson and Reise 2013). Comparatively, discrete models, such as Deterministic Inputs, Noisy-And gate model (DINA), represented each student as a binary vector which denoted whether she mastered or not the knowledge concepts required by exercises (De La Torre 2009). To improve prediction results, many variations, such as learning factors analysis (LFA) (Cen, Koedinger, and Junker 2006), performance factors analysis (PFA) (Pavlik Jr, Cen, and Koedinger 2009) and FuzzyCDM (Wu et al 2015) were proposed by combining other factors.
- This research was partially supported by grants from the National Basic Research Program of China (973 Program Grant No 2015CB351705) and the National Natural Science Foundation of China (Grants No 61572030, 61672483, U1605251 and 61403358)
- Qi Liu gratefully acknowledges the support of the Youth Innovation Promotion Association of CAS (No 2014299)
- Anderson, A.; Huttenlocher, D.; Kleinberg, J.; and Leskovec, J. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web, 687–698. ACM.
- Anzanello, M. J., and Fogliatto, F. S. 2011. Learning curve models and applications: Literature review and research directions. International Journal of Industrial Ergonomics 41(5):573–583.
- Baker, R. S., and Yacef, K. 2009. The state of educational data mining in 2009: A review and future visions. JEDM-Journal of Educational Data Mining 1(1):3–17.
- Cen, H.; Koedinger, K.; and Junker, B. 2006. Learning factors analysis-a general method for cognitive model evaluation and improvement. In Intelligent tutoring systems, volume 4053, 164–175. Springer.
- Chen, Y.; Liu, Q.; Huang, Z.; Wu, L.; Chen, E.; Wu, R.; Su, Y.; and Hu, G. 2017. Tracking knowledge proficiency of students with educational priors. In Proceedings of the 26th ACM International Conference on Conference on Information and Knowledge Management. ACM.
- Corbett, A. T., and Anderson, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction 4(4):253–278.
- De La Torre, J. 2009. Dina model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics 34(1):115–130.
- Desmarais, M.; Beheshti, B.; and Naceur, R. 2012. Item to skills mapping: deriving a conjunctive q-matrix from data. In Intelligent tutoring systems, 454–463. Springer.
- DiBello, L. V.; Roussos, L. A.; and Stout, W. 2006. 31a review of cognitively diagnostic assessment and a summary of psychometric models. Handbook of statistics 26:979–1030.
- Embretson, S. E., and Reise, S. P. 2013. Item response theory. Psychology Press.
- Fogarty, J.; Baker, R. S.; and Hudson, S. E. 2005. Case studies in the use of roc curve analysis for sensor-based estimates in human computer interaction. In Proceedings of Graphics Interface 2005, 129–136. Canadian Human-Computer Communications Society.
- Graves, A.; Mohamed, A.-r.; and Hinton, G. 2013. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, 6645–6649. IEEE.
- Hochreiter, S., and Schmidhuber, J. 1997. Long short-term memory. Neural computation 9(8):1735–1780.
- Huang, Z.; Liu, Q.; Chen, E.; Zhao, H.; Gao, M.; Wei, S.; Su, Y.; and Hu, G. 2017. Question difficulty prediction for reading problems in standard tests. In AAAI, 1352–1359.
- Khajah, M.; Wing, R. M.; Lindsey, R. V.; and Mozer, M. C. 2014a. Incorporating latent factors into knowledge tracing to predict individual differences in learning. In Proceedings of the 7th International Conference on Educational Data Mining, 99– 106.
- Khajah, M. M.; Huang, Y.; Gonzalez-Brenes, J. P.; Mozer, M. C.; and Brusilovsky, P. 2014b. Integrating knowledge tracing and item response theory: A tale of two frameworks. In Proceedings of Workshop on Personalization Approaches in Learning Environments (PALE 2014) at the 22th International Conference on User Modeling, Adaptation, and Personalization, 7–12. University of Pittsburgh.
- Kingma, D., and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
- Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, 1097–1105.
- Kuh, G. D.; Kinzie, J.; Buckley, J. A.; Bridges, B. K.; and Hayek, J. C. 2011. Piecing together the student success puzzle: research, propositions, and recommendations: ASHE Higher Education Report, volume 116. John Wiley & Sons.
- Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, 3111–3119.
- Orr, G. B., and Muller, K.-R. 2003. Neural networks: tricks of the trade. Springer.
- Pardos, Z., and Heffernan, N. 2011. Kt-idem: introducing item difficulty to the knowledge tracing model. User Modeling, Adaption and Personalization 243–254.
- Pavlik Jr, P. I.; Cen, H.; and Koedinger, K. R. 2009. Performance factors analysis–a new alternative to knowledge tracing. Online Submission.
- Piech, C.; Bassen, J.; Huang, J.; Ganguli, S.; Sahami, M.; Guibas, L. J.; and Sohl-Dickstein, J. 2015. Deep knowledge tracing. In Advances in Neural Information Processing Systems, 505–513.
- Rabiner, L., and Juang, B. 1986. An introduction to hidden markov models. ieee assp magazine 3(1):4–16.
- Shang, S.; Chen, L.; Jensen, C. S.; Wen, J.-R.; and Kalnis, P. 2017. Searching trajectories by regions of interest. IEEE Transactions on Knowledge and Data Engineering 29(7):1549–1562.
- Tan, M.; Santos, C. d.; Xiang, B.; and Zhou, B. 2015. Lstmbased deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108.
- Thai-Nghe, N., and Schmidt-Thieme, L. 2015. Multi-relational factorization models for student modeling in intelligent tutoring systems. In Knowledge and Systems Engineering (KSE), 2015 Seventh International Conference on, 61–66. IEEE.
- Thai-Nghe, N.; Drumond, L.; Krohn-Grimberghe, A.; and Schmidt-Thieme, L. 2010. Recommender system for predicting student performance. Procedia Computer Science 1(2):2811– 2819.
- Thai-Nghe, N.; Drumond, L.; Horvath, T.; Krohn-Grimberghe, A.; Nanopoulos, A.; and Schmidt-Thieme, L. 2011. Factorization techniques for predicting student performance. Educational recommender systems and technologies: Practices and challenges 129–153.
- Toscher, A., and Jahrer, M. 2010. Collaborative filtering applied to educational data mining. KDD cup.
- Von Foerster, H. 2007. Understanding understanding: Essays on cybernetics and cognition. Springer Science & Business Media.
- Wilson, K. H.; Xiong, X.; Khajah, M.; Lindsey, R. V.; Zhao, S.; Karklin, Y.; Van Inwegen, E. G.; Han, B.; Ekanadham, C.; Beck, J. E.; et al. 2016. Estimating student proficiency: Deep learning is not the panacea. In In Neural Information Processing Systems, Workshop on Machine Learning for Education.
- Wu, R.-z.; Liu, Q.; Liu, Y.; Chen, E.; Su, Y.; Chen, Z.; and Hu, G. 2015. Cognitive modelling for predicting examinee performance. In IJCAI, 1017–1024.
- Wu, R.; Xu, G.; Chen, E.; Liu, Q.; and Ng, W. 2017. Knowledge or gaming?: Cognitive modelling based on multiple-attempt response. In Proceedings of the 26th International Conference on World Wide Web Companion, 321–329. International World Wide Web Conferences Steering Committee.
- Xu, Y., and Mostow, J. 2010. Using logistic regression to trace multiple sub-skills in a dynamic bayes net. In Educational Data Mining 2011.
- Yudelson, M. V.; Koedinger, K. R.; and Gordon, G. J. 2013. Individualized bayesian knowledge tracing models. In International Conference on Artificial Intelligence in Education, 171– 180. Springer.
- Zhang, J.; Shi, X.; King, I.; and Yeung, D.-Y. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th International Conference on World Wide Web, 765–774. International World Wide Web Conferences Steering Committee.