AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We guess a possible reason is that all these RNN based models can capture the change of student exercise process, where the deep neural network structures are suitable for student performance prediction

Exercise-Enhanced Sequential Modeling for Student Performance Prediction.

THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTI..., pp.2435-2443, (2018)

Cited by: 79|Views152
EI
Full Text
Bibtex
Weibo

Abstract

In online education systems, for offering proactive services to students (e.g., personalized exercise recommendation), a crucial demand is to predict student performance (e.g., scores) on future exercising activities. Existing prediction methods mainly exploit the historical exercising records of students, where each exercise is usually r...More

Code:

Data:

0
Introduction
  • Online education systems, such as massive open online course (MOOC) and intelligent tutoring system (ITS), provide students with open access for self-learning.
  • Their prevalence and convenience have attracted great attentions from both educators and general publics (Anderson et al 2014).
  • In order to offer students proactive services for their self-improvement, e.g., learning remedy suggestion and personalized exercise recommendation (Kuh et al 2011), a crucial demand is to predict their performance, i.e., forecasting whether or not a student could answer the exercise (e.g., e5) correctly in the future (Baker and Yacef 2009)
Highlights
  • Online education systems, such as massive open online course (MOOC) and intelligent tutoring system (ITS), provide students with open access for self-learning
  • The results indicate that Enhanced Recurrent Neural Network (EERNN) framework can make full use of exercising records and exercise texts, benefiting the prediction
  • Models with Attention mechanism (EERNNA, LSTMA) outperform those with Markov property (EERNNM, LSTMM), which demonstrates that it is effective to track focused student embeddings based on similar exercises for the prediction
  • We guess a possible reason is that all these RNN based models can capture the change of student exercise process, where the deep neural network structures are suitable for student performance prediction
  • As mentioned in Section 3, we hold that EERNNA with Attention mechanism can track the focused states of students during the student exercising process to improve prediction performance, which is superior to EERNNM
  • We presented a novel Exercise-Enhanced Recurrent Neural Network (EERNN) framework to predict student future performance by taking full advantage of student exercising records and the texts of exercises
Methods
  • The authors conduct extensive experiments to demonstrate the effectiveness of EERNN from various aspects: (1) the prediction performance of EERNN against the baselines in both future and cold-start scenarios; (2) the attention effectiveness in EERNN; (3) meaningful visualization.

    Experimental Dataset

    The experimental dataset supplied by iFLYTEK Co., Ltd. is collected from Zhixue1, a widely-used online learning system, which provides senior high school students with a large exercise resources for exercising.
  • The authors conduct experiments on the mathematical data records because the mathematical dataset is currently the largest in the system.
  • Note that most exercises contain less than 2 knowledge concepts, and 54 exercises on average are related to one specific knowledge concept.
  • These observations prove that the way to represent exercises as knowledge concepts cannot distinguish differences among exercises, causing some information loss
Results
  • Models with Attention mechanism (EERNNA, LSTMA) outperform those with Markov property (EERNNM, LSTMM), which demonstrates that it is effective to track focused student embeddings based on similar exercises for the prediction.
  • Both EERNNA and EERNNM generate better result than their variants (LSTMA, LSTMM) and DKT, showing the effectiveness of Exercise Embedding.
  • The authors guess a possible reason is that all these RNN based models can capture the change of student exercise process, where the deep neural network structures are suitable for student performance prediction
Conclusion
  • The authors presented a novel Exercise-Enhanced Recurrent Neural Network (EERNN) framework to predict student future performance by taking full advantage of student exercising records and the texts of exercises.
  • For modeling student exercising process, the authors first designed a BiLSTM to extract exercise semantic representations from texts without any expertise and information loss.
  • The authors proposed another LSTM architecture to trace student states by embedding exercise encodings.
  • EERNNA can track the focused information for making prediction, which is superior to EERNNM.
  • Extensive experiments on a large-scale real-world dataset demonstrated the effectiveness of EERNN framework and
Tables
  • Table1: The statistics of mathematics dataset
Download tables as Excel
Related work
  • The related work can be classified into following categories, i,e., Cognitive Diagnosis, Knowledge Tracing, Matrix Factorization and Deep learning researches.

    Cognitive Diagnosis. In the domain of educational psychology, cognitive diagnosis is a technique to predict student performance by discovering student states from their exercising records (DiBello, Roussos, and Stout 2006). Traditional cognitive diagnostic models (CDM) could be grouped into two parts: continuous ones and discrete ones. Among them, item response theory (IRT), as a typical continuous model, characterized each student by a variable from a logistic-like function (Embretson and Reise 2013). Comparatively, discrete models, such as Deterministic Inputs, Noisy-And gate model (DINA), represented each student as a binary vector which denoted whether she mastered or not the knowledge concepts required by exercises (De La Torre 2009). To improve prediction results, many variations, such as learning factors analysis (LFA) (Cen, Koedinger, and Junker 2006), performance factors analysis (PFA) (Pavlik Jr, Cen, and Koedinger 2009) and FuzzyCDM (Wu et al 2015) were proposed by combining other factors.
Funding
  • This research was partially supported by grants from the National Basic Research Program of China (973 Program Grant No 2015CB351705) and the National Natural Science Foundation of China (Grants No 61572030, 61672483, U1605251 and 61403358)
  • Qi Liu gratefully acknowledges the support of the Youth Innovation Promotion Association of CAS (No 2014299)
Study subjects and analysis
exercise records: 19
The attention effects in fitting process for testing. Attention visualization in EERNNA of a student. We predict her performance on e20 based on her past 19 exercise records (we only show the first 5 exercises for better illustration). Right bars show the attention scores of all exercises based on e20.

Reference
  • Anderson, A.; Huttenlocher, D.; Kleinberg, J.; and Leskovec, J. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web, 687–698. ACM.
    Google ScholarLocate open access versionFindings
  • Anzanello, M. J., and Fogliatto, F. S. 2011. Learning curve models and applications: Literature review and research directions. International Journal of Industrial Ergonomics 41(5):573–583.
    Google ScholarLocate open access versionFindings
  • Baker, R. S., and Yacef, K. 2009. The state of educational data mining in 2009: A review and future visions. JEDM-Journal of Educational Data Mining 1(1):3–17.
    Google ScholarLocate open access versionFindings
  • Cen, H.; Koedinger, K.; and Junker, B. 2006. Learning factors analysis-a general method for cognitive model evaluation and improvement. In Intelligent tutoring systems, volume 4053, 164–175. Springer.
    Google ScholarLocate open access versionFindings
  • Chen, Y.; Liu, Q.; Huang, Z.; Wu, L.; Chen, E.; Wu, R.; Su, Y.; and Hu, G. 2017. Tracking knowledge proficiency of students with educational priors. In Proceedings of the 26th ACM International Conference on Conference on Information and Knowledge Management. ACM.
    Google ScholarLocate open access versionFindings
  • Corbett, A. T., and Anderson, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction 4(4):253–278.
    Google ScholarLocate open access versionFindings
  • De La Torre, J. 2009. Dina model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics 34(1):115–130.
    Google ScholarLocate open access versionFindings
  • Desmarais, M.; Beheshti, B.; and Naceur, R. 2012. Item to skills mapping: deriving a conjunctive q-matrix from data. In Intelligent tutoring systems, 454–463. Springer.
    Google ScholarFindings
  • DiBello, L. V.; Roussos, L. A.; and Stout, W. 2006. 31a review of cognitively diagnostic assessment and a summary of psychometric models. Handbook of statistics 26:979–1030.
    Google ScholarLocate open access versionFindings
  • Embretson, S. E., and Reise, S. P. 2013. Item response theory. Psychology Press.
    Google ScholarFindings
  • Fogarty, J.; Baker, R. S.; and Hudson, S. E. 2005. Case studies in the use of roc curve analysis for sensor-based estimates in human computer interaction. In Proceedings of Graphics Interface 2005, 129–136. Canadian Human-Computer Communications Society.
    Google ScholarLocate open access versionFindings
  • Graves, A.; Mohamed, A.-r.; and Hinton, G. 2013. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, 6645–6649. IEEE.
    Google ScholarFindings
  • Hochreiter, S., and Schmidhuber, J. 1997. Long short-term memory. Neural computation 9(8):1735–1780.
    Google ScholarLocate open access versionFindings
  • Huang, Z.; Liu, Q.; Chen, E.; Zhao, H.; Gao, M.; Wei, S.; Su, Y.; and Hu, G. 2017. Question difficulty prediction for reading problems in standard tests. In AAAI, 1352–1359.
    Google ScholarFindings
  • Khajah, M.; Wing, R. M.; Lindsey, R. V.; and Mozer, M. C. 2014a. Incorporating latent factors into knowledge tracing to predict individual differences in learning. In Proceedings of the 7th International Conference on Educational Data Mining, 99– 106.
    Google ScholarLocate open access versionFindings
  • Khajah, M. M.; Huang, Y.; Gonzalez-Brenes, J. P.; Mozer, M. C.; and Brusilovsky, P. 2014b. Integrating knowledge tracing and item response theory: A tale of two frameworks. In Proceedings of Workshop on Personalization Approaches in Learning Environments (PALE 2014) at the 22th International Conference on User Modeling, Adaptation, and Personalization, 7–12. University of Pittsburgh.
    Google ScholarLocate open access versionFindings
  • Kingma, D., and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
    Findings
  • Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, 1097–1105.
    Google ScholarLocate open access versionFindings
  • Kuh, G. D.; Kinzie, J.; Buckley, J. A.; Bridges, B. K.; and Hayek, J. C. 2011. Piecing together the student success puzzle: research, propositions, and recommendations: ASHE Higher Education Report, volume 116. John Wiley & Sons.
    Google ScholarFindings
  • Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, 3111–3119.
    Google ScholarLocate open access versionFindings
  • Orr, G. B., and Muller, K.-R. 2003. Neural networks: tricks of the trade. Springer.
    Google ScholarFindings
  • Pardos, Z., and Heffernan, N. 2011. Kt-idem: introducing item difficulty to the knowledge tracing model. User Modeling, Adaption and Personalization 243–254.
    Google ScholarFindings
  • Pavlik Jr, P. I.; Cen, H.; and Koedinger, K. R. 2009. Performance factors analysis–a new alternative to knowledge tracing. Online Submission.
    Google ScholarFindings
  • Piech, C.; Bassen, J.; Huang, J.; Ganguli, S.; Sahami, M.; Guibas, L. J.; and Sohl-Dickstein, J. 2015. Deep knowledge tracing. In Advances in Neural Information Processing Systems, 505–513.
    Google ScholarLocate open access versionFindings
  • Rabiner, L., and Juang, B. 1986. An introduction to hidden markov models. ieee assp magazine 3(1):4–16.
    Google ScholarLocate open access versionFindings
  • Shang, S.; Chen, L.; Jensen, C. S.; Wen, J.-R.; and Kalnis, P. 2017. Searching trajectories by regions of interest. IEEE Transactions on Knowledge and Data Engineering 29(7):1549–1562.
    Google ScholarLocate open access versionFindings
  • Tan, M.; Santos, C. d.; Xiang, B.; and Zhou, B. 2015. Lstmbased deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108.
    Findings
  • Thai-Nghe, N., and Schmidt-Thieme, L. 2015. Multi-relational factorization models for student modeling in intelligent tutoring systems. In Knowledge and Systems Engineering (KSE), 2015 Seventh International Conference on, 61–66. IEEE.
    Google ScholarLocate open access versionFindings
  • Thai-Nghe, N.; Drumond, L.; Krohn-Grimberghe, A.; and Schmidt-Thieme, L. 2010. Recommender system for predicting student performance. Procedia Computer Science 1(2):2811– 2819.
    Google ScholarLocate open access versionFindings
  • Thai-Nghe, N.; Drumond, L.; Horvath, T.; Krohn-Grimberghe, A.; Nanopoulos, A.; and Schmidt-Thieme, L. 2011. Factorization techniques for predicting student performance. Educational recommender systems and technologies: Practices and challenges 129–153.
    Google ScholarFindings
  • Toscher, A., and Jahrer, M. 2010. Collaborative filtering applied to educational data mining. KDD cup.
    Google ScholarFindings
  • Von Foerster, H. 2007. Understanding understanding: Essays on cybernetics and cognition. Springer Science & Business Media.
    Google ScholarFindings
  • Wilson, K. H.; Xiong, X.; Khajah, M.; Lindsey, R. V.; Zhao, S.; Karklin, Y.; Van Inwegen, E. G.; Han, B.; Ekanadham, C.; Beck, J. E.; et al. 2016. Estimating student proficiency: Deep learning is not the panacea. In In Neural Information Processing Systems, Workshop on Machine Learning for Education.
    Google ScholarLocate open access versionFindings
  • Wu, R.-z.; Liu, Q.; Liu, Y.; Chen, E.; Su, Y.; Chen, Z.; and Hu, G. 2015. Cognitive modelling for predicting examinee performance. In IJCAI, 1017–1024.
    Google ScholarLocate open access versionFindings
  • Wu, R.; Xu, G.; Chen, E.; Liu, Q.; and Ng, W. 2017. Knowledge or gaming?: Cognitive modelling based on multiple-attempt response. In Proceedings of the 26th International Conference on World Wide Web Companion, 321–329. International World Wide Web Conferences Steering Committee.
    Google ScholarLocate open access versionFindings
  • Xu, Y., and Mostow, J. 2010. Using logistic regression to trace multiple sub-skills in a dynamic bayes net. In Educational Data Mining 2011.
    Google ScholarLocate open access versionFindings
  • Yudelson, M. V.; Koedinger, K. R.; and Gordon, G. J. 2013. Individualized bayesian knowledge tracing models. In International Conference on Artificial Intelligence in Education, 171– 180. Springer.
    Google ScholarLocate open access versionFindings
  • Zhang, J.; Shi, X.; King, I.; and Yeung, D.-Y. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th International Conference on World Wide Web, 765–774. International World Wide Web Conferences Steering Committee.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科