Massive Open Online Courses (MOOCs) have been proliferating, offering free or low-cost content for learners. Nevertheless, the certification rate of both free and paid courses has been low (between 4.5% - 13% and 1% - 3%, respectively). Thus, this study aims to survey MOOCs certification predictive models, synthesise results for a comprehensive and deep understanding of this field and explore how these models contributed to addressing the very low certification level. We adopted the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) for transparently conducting the present review and reporting the results from the works reviewed. Additionally, this SLR highlights several trends and limitations within the present predictive models, including some methodological concerns: the extent to which the present models are generalisable, the excessive filtration of the experimental population, the incompatibility of some experiments with real-time scenarios (nonrealistic modelling), and the shallow reporting of model performances. We have also discussed the replicability of the present models and ongoing efforts towards building a state-of-the-art predictive model. Finally, we highlight future research opportunities in the field of MOOC certification prediction that either deal with the limitations of the present models or address unanswered questions.

Massive Open Online Courses (MOOCs) have revolutionised the landscape of education, generating unprecedented arrays of diverse data about, from and by learners. However, there are few publicly accessible large-scale text datasets that several stakeholders, including course designers and instructors, can use. This paper presents MOOCRev, a groundbreaking data repository aggregating MOOC reviews from various platforms, leveraging cutting-edge data collection and processing techniques. The dataset encompasses many review characteristics (over 1,250,000 course reviews and ratings), making it, to the best of our knowledge, the largest publicly accessible MOOC review dataset. MOOCRev targets the following challenges (1) thorough, research-based anonymisation of the data to ensure user privacy and data integrity, (2) dealing with class (rating) bias, (3) addressing language diversity and platform-specific rating systems, (4) processing emojis and emoticons, (5) providing text weights (6) and tagging the reviews with the corresponding Parts of Speech (POS). The repository provides researchers, educators, and platform developers with invaluable opportunities for various Natural Language Processing (NLP) tasks, including learners' sentiments and feedback analysis, aiding future learners in selecting courses aligned with their interests and learning styles based on previous learners' experiences.

Urgent intervention in learner forum posts have recently occupied a very important role in research in Massive Open Online Course (MOOC) environments. Intervening in time may make the difference between a learner dropping out or staying on a course. However, due to the typical extremely high learner-to-instructor ratio in MOOCs, it is very challenging – if not sometimes impossible - for the instructor to monitor all the existing posts and identify which need immediate intervention, to encourage retention. Current approaches are based on shallow machine learning and deep learning. Whilst deep learning methods have been shown to be most accurate in many domains, the exact architecture can be very domain-dependent. In spite of their sheer size and representation power, deep neural networks are known to perform better when a problem is divided into the right sub-problems. These sub-problems can be further assembled together, to answer to the original problem, in what we intuitively call a ‘plug & play’-like fashion, similarly to puzzles – via hybrid (deep) neural networks. Hence, in this paper, we address this problem by proposing a classification model for identifying when a given post needs intervention from an instructor, based on hybrid neural networks. We represent words using two different methods; word2vec: that capture the word's semantic and syntactic characteristics; and transformer model (BERT): which represents each word according to its context. Then we construct different architectures, integrating various deep neural networks (DNNs) -‘word-based’ or ‘word-character based’, as we expected that adding additional character-sequence information may increase performance. For word-based, we apply convolutional neural network (CNN) and/or different types of recurrent neural networks (RNN); in some scenarios we added attention. This is to present a comprehensive answer to the character-sequence question in particular, as well as to the urgency of intervention need prediction in MOOC forums, in general. Experimental results demonstrate that using BERT rather than word2vec as a word embedding enhances performance in different models (the optimal result is the CNN + LSTM + Attention model based on BERT at word-level). Interestingly, adding word-character input does not improve the performance, as it does for word2vec.

Discussion forums on MOOCs are developing as a major tool for communication between learners and instructors, generating large amounts of posts, exchanged as unstructured text content. Thus, it is a major challenge for instructors to find (and respond to) urgent posts, amongst the vast amount of posts. Learners also may inadvertently pose posts that may seem more urgent than they are. However, the current literature lacks research on analysing posts from an urgency language perspective, i.e., extracting the language used for urgent expression . This paper explores for the first time the urgent language that learners use to express their need for immediate intervention, via an automatised approach. It describes our analysis of 5181 text posts from a course from the Stanford MOOCPosts dataset, selected for its good representation of urgent posts. We use topic modelling, here, via the widely used latent Dirichlet allocation (LDA). Moreover, we demonstrate a correlation between specific topics and urgent posts. Also, we show that most urgent posts start new threads. Additionally, constructing a visual interface for instructors or learners may support them understand the urgent language and improve intervention.

In MOOCs, identifying urgent comments on discussion forums is an ongoing challenge. Whilst urgent comments require immediate reactions from instructors, to improve interaction with their learners, and potentially reducing drop-out rates—the task is difficult, as truly urgent comments are rare. From a data analytics perspective, this represents a highly unbalanced (sparse) dataset . Here, we aim to automate the urgent comments identification process, based on fine-grained learner modelling —to be used for automatic recommendations to instructors. To showcase and compare these models, we apply them to the first gold standard dataset for U rgent i N structor I n TE rvention (UNITE) , which we created by labelling FutureLearn MOOC data. We implement both benchmark shallow classifiers and deep learning. Importantly, we not only compare, for the first time for the unbalanced problem, several data balancing techniques , comprising text augmentation, text augmentation with undersampling, and undersampling, but also propose several new pipelines for combining different augmenters for text augmentation . Results show that models with undersampling can predict most urgent cases; and 3X augmentation + undersampling usually attains the best performance. We additionally validate the best models via a generic benchmark dataset (Stanford). As a case study, we showcase how the naïve Bayes with count vector can adaptively support instructors in answering learner questions/comments, potentially saving time or increasing efficiency in supporting learners. Finally, we show that the errors from the classifier mirrors the disagreements between annotators. Thus, our proposed algorithms perform at least as well as a ‘super-diligent’ human instructor (with the time to consider all comments).

Determining when instructor intervention is needed, based on learners’ comments and their urgency in massive open online course (MOOC) environments, is a known challenge. To solve this challenge, prior art used autonomous machine learning (ML) models. These models are described as having a "black-box" nature, and their output is incomprehensible to humans. This paper shows how to apply eXplainable Artificial Intelligence (XAI) techniques to interpret a MOOC intervention model for urgent comments detection. As comments were selected from the MOOC course and annotated using human experts, we additionally study the confidence between annotators (annotator agreement confidence), versus an estimate of the class score of making a decision via ML, to support intervention decision. Serendipitously, we show, for the first time, that XAI can be further used to support annotators creating high-quality, gold standard datasets for urgent intervention.

Deciding upon instructor intervention based on learners' comments that need an urgent response in MOOC environments is a known challenge. The best solutions proposed used automatic machine learning (ML) models to predict the urgency. These are 'black-box'-es, with results opaque to humans. EXplainable artificial intelligence (XAI) is aiming to understand these, to enhance trust in artificial intelligence (AI)-based decision-making. We propose to apply XAI techniques to interpret a MOOC intervention model, by analysing learner comments. We show how pairing a good predictor with XAI results and especially colour-coded visualisation could be used to support instructors making decisions on urgent intervention.

Automatically identifying the learner gender, which serves as this paper's focus, can provide valuable information to personalised learners' experiences in MOOCs. However, extracting the gender from learner-generated data (discussion forum) is a challenging task, which is understudied in literature. Using syntactic features is still the state-of-the-art for gender identification in social media. Instead we propose here a novel approach based on Recursive Neural Networks (RecNN), to learn advanced syntactic knowledge extracted from learners' comments, as an NLP-based predictor for their gender identity. We propose a bi-directional composition function, added to NLP state-of-the-art candidate RecNN models. We evaluate different combinations of semantic level encoding and syntactic level encoding functions, exploring their performances, with respect to the task of learner gender profiling in MOOCs.

High learner dropout rates in MOOC-based education contexts have encouraged researchers to explore and propose different intervention models. In discussion forums, intervention is critical, not only to identify comments that require replies but also to consider learners who may require intervention in the form of staff support. There is a lack of research on the role of intervention based on learner comments to prevent learner dropout in MOOC-based settings. To fill this research gap, we propose an intervention model that detects when staff intervention is required to prevent learner dropout using a dataset from FutureLearn. Our proposed model was based on learners' comments history by integrating the most-recent sequence of comments written by learners to identify if an intervention was necessary to prevent dropout. We aimed to find both the proper classifier and the number of comments representing the appropriate most recent sequence of comments. We developed several intervention models by utilising two forms of supervised multi-input machine learning (ML) classification models (deep learning and transformer). For the transformer model, specifically, we propose the siamese and dual temporal multi-input, which we term the multi-siamese BERT and multiple BERT. We further experimented with clustering learners based on their respective number of comments to analyse if grouping as a pre-processing step improved the results. The results show that, whilst multi-input for deep learning can be useful, a better overall effect is achieved by using the transformer model, which has better performance in detecting learners who require intervention. Contrary to our expectations, however, clustering before prediction can have negative consequences on prediction outcomes, especially in the underrepresented class.

Recently, the growing number of learners in Massive Open Online Course (MOOC) environments generate a vast amount of online comments via social interactions, general discussions, expressing feelings or asking for help. Concomitantly, learner dropout, at any time during MOOC courses, is very high, whilst the number of learners completing (completers) is low. Urgent intervention and attention may alleviate this problem. Analysing and mining learner comments is a fundamental step towards understanding their need for intervention from instructors. Here, we explore a dataset from a FutureLearn MOOC course. We find that (1) learners who write many comments that need urgent intervention tend to write many comments, in general. (2) The motivation to access more steps (i.e., learning resources) is higher in learners without many comments needing intervention, than that of learners needing intervention. (3) Learners who have many comments that need intervention are less likely to complete the course (13%). Therefore, we propose a new priority model for the urgency of intervention built on learner histories – past urgency, sentiment analysis and step access.

One key type of Massive Open Online Course (MOOC) data is the learners’ social interaction (forum). While several studies have analysed MOOC forums to predict learning outcomes, analysing learners’ sentiments in education and, specifically, in MOOCs, remains limited. Moreover, most studies focus on one platform only. Here, we propose a cross-platform MOOCs sentiment classifier using almost 1.5 million human-annotated learners’ comments obtained from 633 MOOCs delivered via the Stanford University platform and Coursera -the largest dataset collected for sentiment analysis (SA). We explore not only various state-of-the-art SA tools, but also their confidence level distributions and evaluate their performance. Our results show that the Lexicon and Rulebased (LRB) and Convolutional Neural Network (CNN)-based sentiment tools, trained mainly on social media platforms, may not be suitable for the educational domain. We further introduce MOOCSent1, a BERT-based model for predicting MOOC learners’ sentiments from their comments, which almost doubles the accuracy of the classification results, outperforming the state-of-the-art with a 95% accuracy.

Massive Open Online Courses (MOOCs) have become a popular choice for e-learning thanks to their great flexibility. However, due to large numbers of learners and their diverse backgrounds, it is taxing to offer real-time support. Learners may post their feelings of confusion and struggle in the respective MOOC forums, but with the large volume of posts and high workloads for MOOC instructors, it is unlikely that the instructors can identify all learners requiring intervention. This problem has been studied as a Natural Language Processing (NLP) problem recently, and is known to be challenging, due to the imbalance of the data and the complex nature of the task. In this paper, we explore for the first time Bayesian deep learning on learner-based text posts with two methods: Monte Carlo Dropout and Variational Inference, as a new solution to assessing the need of instructor interventions for a learner’s post. We compare models based on our proposed methods with probabilistic modelling to its baseline non-Bayesian models under similar circumstances, for different cases of applying prediction. The results suggest that Bayesian deep learning offers a critical uncertainty measure that is not supplied by traditional neural networks. This adds more explainability, trust and robustness to AI, which is crucial in education-based applications. Additionally, it can achieve similar or better performance compared to non-probabilistic neural networks, as well as grant lower variance.

Welfare and economic development is directly dependent on the availability of highly skilled and educated individuals in society. In the UK, higher education is accessed by a large percentage of high school graduates (50% in 2017). Still, in Brazil, a limited number of pupils leaving high schools continue their education (up to 20%). Initial pioneering efforts of universities and companies to support pupils from underprivileged backgrounds, to be able to succeed in being accepted by universities include personalised learning solutions. However, initial findings show that typical distance learning problems occur with the pupil population: isolation, demotivation, and lack of engagement. Thus, researchers and companies proposed gamification. However, gamification design is traditionally exclusively based on theory-driven approaches and usually ignore the data itself. This paper takes a different approach, presenting a large-scale study that analysed, statistically and via machine learning (deep and shallow), the first batch of students trained with a Brazilian gamified intelligent learning software (called CamaleOn), to establish, via a grassroots method based on learning analytics, how gamification elements impact on student engagement. The exercise results in a novel proposal for realtime measurement on Massive Open Online Courses (MOOCs), potentially leading to iterative improvements of student support. It also specifically analyses the engagement patterns of an underserved community.

In recent years, massive open online courses (MOOCs) have become one of the most exciting innovations in e-learning environments. Thousands of learners around the world enroll on these online platforms to satisfy their learning needs (mostly) free of charge. However, despite the advantages MOOCs offer learners, dropout rates are high. Struggling learners often describe their feelings of confusion and need for help via forum posts. However, the often-huge numbers of posts on forums make it unlikely that instructors can respond to all learners and many of these urgent posts are overlooked or discarded. To overcome this, mining raw data for learners' posts may provide a helpful way of classifying posts where learners require urgent intervention from instructors, to help learners and reduce the current high dropout rates. In this paper we propose, a method based on correlations of different dimensions of learners' posts to determine the need for urgent intervention. Our initial statistical analysis found some interesting significant correlations between posts expressing sentiment, confusion, opinion, questions, and answers and the need for urgent intervention. Thus, we have developed a multidimensional deep learner model combining these features with natural language processing (NLP). To illustrate our method, we used a benchmark dataset of 29598 posts, from three different academic subject areas. The findings highlight that the combined, multi-dimensional features model is more effective than the text-only (NLP) analysis, showing that future models need to be optimised based on all these dimensions, when classifying urgent posts.

In adaptive e-learning systems attention is focused on adjusting the learning material to the needs of the individual student. There are differences in the properties of each individual such as differences in skills, prior knowledge, ... etc. These characteristics must be measured to display the content that is suitable to individual needs of student. There are different e-assessment models that are appropriate for discovering the prior knowledge and skills. The main objective of this research is to build an e-assessment model for Personal and Intelligent Assessment using Test (PIAT) to facilitate the evaluation process and measure the students' proficiency with more accuracy and store it in the student's profile for later use in the process of adapting content material to individual student needs. On one hand the model can help the instructors to develop the test by determining the objectives, creation and reuse of the test items. On the other hand the model can provide a personal test environment to the student. It generates the appropriate questions in an adaptive way, based on the student's model. The PIAT model is based on two theories. The first theory is Revised Bloom's Taxonomy (RBT) and the second theory is Felder-Silverman Learning Style Model (FSLSM).