This research paper explores the use of learning analytics to predict student performance in the Moodle Learning Management System (LMS). It examines the accuracy of various predictive models in determining student success or failure and in predicting final grades. Ten models were evaluated using Moodle student records, which included engagement patterns and academic outcomes. The results showed significant variation in model accuracy. The Gradient Boosting Classifier achieved an accuracy of 96.296% in predicting course outcomes, while Random Forest model demonstrated 92.593% accuracy. For predicting final grades, the Random Forest, Decision Tree, and Gradient Boosting Classifiers all achieved an accuracy of 92.593%, with other models also showing good accuracy. These findings offer valuable insights for teachers and students to identify at-risk students and enhance academic outcomes. Further research is recommended to explore the factors and features that contribute to predictive modeling, including gender-based variations in student success, and to refine models for improved performance.

The quality of an artificial intelligence-based tutoring system is its ability to observe and interpret student behaviour to infer the preferences and needs of an individual student. The student model enables a comprehensive representation of student knowledge and affects the quality of the other intelligent tutoring system’s (ITS) components. The Bayesian knowledge tracing model (BKT) is one of the first machine learning-based and widely investigated student models due to its interpretability and ability to infer student knowledge. The past Twenty-five Years have seen increasingly rapid advances in the field, so this systematic review deals with the BKT model enhancements by using the PRISMA guidelines and a unique set of criteria, including 13 aspects of enhancements and computational methods. Also, the study reveals two types of evaluation approaches found in the literature, including the prediction of student answers and the ability to estimate knowledge mastery. Overall, the most frequently investigated enhancements extended the vanilla BKT model by including student characteristics and tutor interventions. The educational context-based enhancements of domain knowledge properties, question difficulty and architectural prior knowledge were also frequently investigated enhancements. The expectation–maximization algorithm practically became the standard in estimating BKT parameters. While the enhanced BKT models generally overperformed the vanilla model in predicting the student answer by using the measures such as RMSE (root mean square error), AUC–ROC (area under curve, receiver operating characteristics curve) and accuracy, only a few studies further investigated the systems’ estimations of knowledge mastery by correlating it to knowledge on post-tests. The most frequently used educational platforms included ITSs, Massive Open Online Courses (MOOCs) and simulated environments.

This paper describes and evaluates the performance of a semi-automatic authoring tool (SAAT) for knowledge extraction in the AC & NL Tutor, highlighting its strengths and weaknesses. We assessed the accuracy of automatic annotation tasks (Part-of-Speech tagging, Name Entity Recognition, Dependency parsing, and Coreference Resolution) performed on a dataset of 160 sentences from unstructured Wikipedia text on a computer. We compared the automatic annotations to the gold standard, created after human post-editing and validation. Human-error analysis included 3769 words, 582 subsentences, 1129 questions, 917 propositions, 1020 concepts, and 667 relations. It resulted in the error type classification and the set of custom rules further used for automatic error identification and correction. The results showed that an average of 68.7% of the error corrections referred to CoreNLP performance and 31.3% to the SAAT extraction algorithms. Our main contributions include an integrated approach to the comprehensive pre-processing of the text, knowledge extraction and visualization; the consolidated evaluation of natural language processing tasks and knowledge extraction output (sentences, subsentences, questions, concept maps) and the newly developed reference dataset.

We present a rule-based approach to automatic factual question generation implemented in the Adaptive Courseware and Natural Language Tutor, a natural language-based intelligent tutoring system. Since machine-generated questions are intended for adaptive teaching, learning and assessment, their accuracy is of the utmost importance. However, the generation of high-quality questions is still challenging. The proposed approach relies on pre-processing techniques and syntactic and semantic feature extraction to transform declarative sentences and their segments into questions. The quality of questions, generated from domain specific texts, was evaluated by using mixed evaluation strategies: (1) human evaluation, (2) qualitative error analysis, (3) automatic evaluation, (4) human and automatic evaluation of machine-generated questions from paraphrases compared to a set of human-authored questions, (5) preliminary comparison to other approaches. The human evaluation involved two teachers of English as a foreign language who set up evaluation criteria (grammaticality, semantic accuracy, and answerability) and a group of 30 English language graduates. Student-generated questions were validated and used as reference questions for automatic evaluation based on similarity metrics (BLEU-4, METEOR, CHRF, NIST and ROUGE-L). Human and automatic evaluation results were satisfactory but improved significantly with the paraphrasing strategy. The preliminary comparison to other approaches showed that the proposed rule-based approach performed equally well despite its limitations.

In this article we present an knowledge extraction approach that can be used in systems that implement teaching in a fully automated manner. These systems are called Intelligent Tutoring Systems (ITS) and are conceived around the idea of one-to-one teaching. Many such systems use natural language processing to improve the communication interface between student and the system. These techniques can be also used on the content creator side to semi-automate or fully automate the task of teaching content creation. In such systems the knowledge representation plays a crucial role to successfully implement teaching and encourage learning. The output of the knowledge extraction phase is a knowledge in the form of a hyper graph that can be used for adaption to the students current knowledge level. We present a deep neural network architecture for precise POS tagging of words written in languages that are morphologically rich. Using sparse representations for words in this task increases the vector space and makes learning more complex. This problem can be solved to some extent by using traditional vector representations but there is also the problem with representing words that are ambiguous. Proposed architecture uses a Bidirectional Encoder Representations from Transformers (BERT) model that is pre-trained on Croatian language to achieve state-of-the-art accuracy for POS tagging.

The research investigates how note-taking practice affects the learning process in Tutomat, an intelligent tutoring system. The complete analysis includes (i) the identification of learning analytics variables to describe student-Tutomat interaction; (ii) the description of experimental student groups using learning analytics variables; (iii) data-driven clustering and (iv) the comparison of the experimental groups and revealed clusters. The results show that there is a difference in how a student interacts with Tutomat based on note-taking practice. It is revealed that the note-taking practice can be detected using the proposed learning analytics variables with the prediction accuracy of the clustering approach of 85 %.

E-Learning environment implies self-motivation and perseverance in study and completion of learning tasks. However, the more autonomy students have in managing their e-Learning, the harder they cope with distractions and remaining focused and engaged. This research study aims to assess the level of student engagement in four e-Learning platforms (CoLaB Tutor, AC-ware Tutor, CM Tutor and Moodle) in higher education. A model for Tracking Student Learning and Knowledge (TSLAK) is developed and based on two sets of variables: variables tracking student's learning activities (VTL) and variables tracking student's knowledge (VTK). This study aims to provide answers on how a model for tracking student online learning and knowledge can be formalized for the four e-Learning platforms and how can student learning and knowledge acquisition processes be described and measured by VTL and VTK. The results obtained by VTL and VTK indicate a significant decline in students' engagement. Out of 218 the most engaged students, 77 (35%) of them used the CoLaB Tutor, 41 (19%) used the AC-ware Tutor, 52 (24%) used the CM Tutor, and 48 (22%) used the Moodle. The research showed that out of the total number of students only 88 (13%) of them were the most engaged and the most successful or more precisely, 63 (71%) graduates and 25 (29%) undergraduates. Such student engagement and success measured by VTL and VTK indicate the necessity of increasing students' motivation in blended learning environments, strengthening their preparation and introduction to e-Learning platforms, and observing their feedback during a research study.

The idea of clustering students according to their online learning behavior has the potential of providing more adaptive scaffolding by the intelligent tutoring system itself or by a human teacher. With the aim of identifying student groups who would benefit from the same intervention in AC-ware Tutor, this research examined online learning behavior using 8 tracking variables: the total number of content pages seen in the learning process; the total number of concepts; the total online score; the total time spent online; the total number of logins; the stereotype after the initial test, the final stereotype, and the mean stereotype variability. The previous measures were used in a four-step analysis that consisted of data preprocessing, dimensionality reduction, the clustering, and the analysis of a posttest performance on a content proficiency exam. The results were also used to construct the decision tree in order to get a human-readable description of student clusters.

The reasoning process about the level of student’s knowledge can be challenging even for experienced human tutors. The Bayesian networks are a formalism for reasoning under uncertainty, which has been successfully used for various artificial intelligence applications, including student modeling. While Bayesian networks are a highly flexible graphical and probabilistic modeling framework, its main challenges are related to the structural design and the definition of “a priori” and conditional probabilities. Since the AC&NL Tutor’s authoring tool automatically generates tutoring elements of different linguistic complexity, the generated sentences and questions fall into three difficulty levels. Based on these levels, the probability-based Bayesian student model is proposed for mastery-based learning in intelligent tutoring system. The Bayesian network structure is defined by generated questions related to the node representing knowledge in a sentence. Also, there are relations between inverse questions at the same difficulty level. After the structure is defined, the process of assigning “a priori” and conditional probabilities is automated using several heuristic expert-based rules.

Automatic knowledge acquisition is a rather complex and challenging task. This paper focuses on the description and evaluation of a semi-automatic authoring tool (SAAT) that has been developed as a part of the Adaptive Courseware based on Natural Language AC&NL Tutor project. The SAAT analyzes a natural language text and, as a result of the declarative knowledge extraction process, it generates domain knowledge that is presented in a form of natural language sentences, questions and domain knowledge graphs. Generated domain knowledge presents expert knowledge in the intelligent tutoring system Tutomat. The natural language processing techniques are applied and the tool’s functionalities are thoroughly explained. This tool is, to our knowledge, the only one that enables natural language question and sentence generation of different levels of complexity. Using an unstructured and unprocessed Wikipedia text in computer science, evaluation of domain knowledge extraction algorithm, i.e. the correctness of extraction outcomes and the effectiveness of extraction methods, was performed. The SAAT outputs were compared with the gold standard, manually developed by two experts. The results showed that 68.7% of detected errors referred to the performance of the integrated linguistic resources, such as CoreNLP, Senna, WordNet, whereas 31.3% of errors referred to the proposed extraction algorithms.

The idea of clustering students according to their online learning behavior has the potential of providing more adaptive scaffolding by the intelligent tutoring system itself or by a human teacher. With the aim of identifying groups of students who would benefit from the same intervention, in this paper, we study a set of 104 weekly behaviors observed for 26 students in a blended learning environment with AC-ware Tutor, an ontology-based intelligent tutoring system. Online learning behavior in AC-ware Tutor is described using 8 tracking variables: (i) the total number of content pages seen in the learning process; (ii) the total number of concepts seen in the learning process; (iii) the total content proficiency score gained online; (iv) the total time spent online; (v) the total number of student logins to AC-ware Tutor; (vi) the stereotype value after the initial test in AC-ware Tutor, (vii) the final stereotype value in the learning process, and (viii) the mean stereotype variability in the learning process. The previous measures are used in a four-step analysis process that includes the following elements: data preprocessing (Z-score normalization), dimensionality reduction (Principal component analysis), the clustering (Kmeans), and the analysis of a posttest performance on a content proficiency exam. By using the Euclidean distance in K-means clustering, we identified 4 distinct online learning behavior clusters, which we designate by the following names: Engaged Pre-knowers, Pre-knowers Non-finishers, Hard-workers, and Non-engagers. The posttest proficiency exam scores were compared among the aforementioned clusters using the Mann-Whitney U test.

This paper presents the current state of research in the fields of intelligent tutoring systems and intelligent agents. The architecture of intelligent tutoring systems typically consists of four major components: the domain knowledge module, the learner diagnosis module, the tutoring module, and the communication module. An agent-based approach to intelligent tutoring systems research and development is applied as the set of agents that constitute the general architecture, and as animated characters - pedagogical agents. Therefore, the paper focuses on the components of intelligent tutoring systems that can be implemented as agents, as well as on functions that these intelligent agents can perform in each intelligent tutoring system component. Also, we present the recent endeavours in the intelligent tutoring systems community that moves from standalone systems focused on a single instructional domain to adaptive instructional systems that comprise learners, intelligent tutoring systems, and external (non-adaptive) environment.

Over the last few decades, researchers put efforts to improve intelligent tutoring systems' abilities with the aim to get them as close as possible to the ultimate goal of one-to-one tutoring. CoLaB Tutor and AC-ware Tutor are intelligent tutoring systems based on conceptual knowledge learning and are notable due to the fact they are relatively easy to generalize to multiple knowledge domains. CoLaB Tutor's forte lies in teacher-learner communication in controlled natural language, while AC-ware Tutor focuses on the automatic and dynamic generation of adaptive courseware. In order to compare various intelligent tutoring system supported education environments, in this chapter, the authors summarize several empirical evaluations of CoLaB Tutor and AC-ware Tutor. The results of intelligent tutoring systems' effectiveness in these environments offer the possibility to observe the specific intelligent tutoring system across various education levels, as well as to compare the intelligent tutoring systems' supported education environments.

This paper discusses a study conducted in a blended learning environment for an introductory computer programming course. The experimental environment is a type of flipped classroom model, in which students initially learn the basic concepts online and then learn traditional lecture in the subsequent class. During the design phase of a blended learning course, it is important to determine the time frame a teacher expects from students regarding spending time online. Therefore, the aim of this study of AC-ware supported education is to identify online learning time frames. After the students use AC-ware Tutor for four experimental weeks, we will investigate their weekly online learning behavior. The online learning behavior will be represented using the knowledge tracking variables complemented by additional stereotype tracking variables. By using K-means algorithm, we will highlight four groups of specific online learning behaviors. The performance of each student group will be described using weekly paper-based concept map posttests.

CoLaB Tutor and AC-ware Tutor are Intelligent Tutoring Systems (ITSs) that are based on concept-based learning and are notable due to the fact they are relatively easy to generalize to multiple knowledge domains. In this research study we investigate the performance of CoLaB Tutor, AC-ware Tutor, and Moodle in a blended learning environment for an introductory computer programming course. In our study, regular face-to-face lectures and laboratory exercises were complemented with online learning at the students' own pace, time and location. Our study revealed that CoLaB Tutor students had moderately higher knowledge gains than those students in the AC-ware and Moodle groups. The prediction of student success (pass/fail) for a basic knowledge post-test revealed an overall classification rate of 73,5% for the CoLaB Tutor group (completed knowledge and online score as predictors), 71,4% for the AC-ware group (completed knowledge as predictor) and 70% for the Moodle group (time spent online as predictor). Additionally, students that used ITSs on average passed through more knowledge online than students that used LMS, while students that used LMS on average spent more time online.

In this research we propose a comprehensive set of knowledge indicators aimed to enhance learners' self-reflection and awareness in the learning and testing process. Since examined intelligent tutoring systems do not include additional messaging features, the introduction of common set of knowledge indicators differentiates our approach from the previous studies. In order to investigate the relation between proposed knowledge indicators and learner performance, the correlation and regression analysis were performed for 3 different courses and each examined intelligent tutoring system. The results of correlation and regression analysis, as well as learners' feedback, guided us in discussion about the introduction of knowledge indicators in dashboard-like visualizations of integrated intelligent tutoring system.

We present results of empirical evaluation of intelligent tutoring systems (ITS) with ontological domain knowledge representation. This research was done as a first step in the process of developing a new model of intelligent tutoring system that will include all the characteristics of evaluated systems: adaptive content, communication based on controlled natural language, graphical presentation of ontological domain knowledge representation. The case study results revealed extraordinary effectiveness of evaluated adaptive intelligent tutoring systems when compared with traditional learning and teaching process.