Early-warning models in distance education must distinguish future withdrawal from events that have already occurred and make the information available at prediction time explicit. Using the Open University Learning Analytics Dataset (OULAD), we constructed risk sets at days 28, 56 and 84 containing 27,515, 26,506 and 25,713 eligible enrollments, with future-withdrawal rates of 18.55%, 15.43% and 12.81%. Logistic regression, random forest and XGBoost were evaluated on the later 2014J presentation after student-grouped development and probability calibration. Logistic regression achieved ROC-AUCs of 0.687, 0.692 and 0.710. Specifically in the day-84 temporal holdout, its ROC-AUC was 0.7102 versus 0.7104 for random forest; the paired difference was 0.0002 (95% CI −0.0105 to 0.0119). A simultaneous regression identified independent associations for assessment completion, weighted performance and lateness, whereas submission inactivity added weaker evidence after mutual adjustment. Day-84 discrimination remained 0.708 in a student-disjoint test subset and 0.682 when score-derived predictors were removed. Adding basic virtual-learning-environment indicators improved earlier discrimination but provided no clear day-84 gain. At the day-84 F2-selected threshold, 427 alerts per 1000 enrollments included approximately 93 future withdrawals and 333 false positives. Subgroup error differences and imperfect calibration support capacity-aware human review, local recalibration and prospective evaluation of intervention effects.
High-quality, annotated datasets are crucial for enhancing the application of natural language processing tasks. Nevertheless, in the educational domain, they are limited compared to the vast amount of learning content. In the context of smart learning environments, game-based learning and artificial intelligence technologies enable the development of interactive and engaging learning experiences. In this paper, we provide an annotated dataset for sentence classification from educational problem sets to automate educational content structuring. The initial collection was extracted from the Vaia platform, including 13 disciplines with 91,590 problem sets. A subset of 1678 problem sets was manually labeled using the Doccano tool. Our annotation schema includes four sentence categories: Background, Given Data, Instruction, and Question, which represent the required components of educational problem sets and support escape game design applications. To validate our dataset, deep learning models are tested on the proposed dataset to provide a comparative study. The results demonstrate that a simple transformer architecture, the BERT classifier, achieved the best overall performance with a Macro F1-score of 0.94.
Chikungunya is a mosquito-borne viral disease and an important public health challenge, particularly in regions where competent Aedes vectors are established. High-quality question-answering resources grounded in authoritative clinical guidance are needed to support the development and evaluation of Artificial Intelligence (AI) applications for this disease. ChikungunyaQA is a Brazilian Portuguese (PT-BR) question-answering dataset designed to support the development and evaluation of language-model applications grounded in authoritative Chikungunya guidance. The dataset was constructed from six Brazilian clinical and technical documents, which were semantically segmented and enriched with provenance and clinical metadata. GPT-4o mini generated multi-persona question–answer pairs for physicians, patients, and caregivers using an inductive, source-grounded pipeline. All instances were independently evaluated with Claude Sonnet 4.5 using G-Eval across faithfulness, completeness, persona appropriateness, and clinical relevance, then pairs below a threshold underwent feedback-guided repair and re-evaluation. The final dataset contains 1262 retained QA pairs generated from 112 productive source fragments, among 141 fragments extracted from six documents. Mean G-Eval score was 97.12/100, with 99.7% of generated pairs retained. Human evaluation of 200 pairs by a physician and a physical therapist showed high ratings for clinical accuracy, completeness, and persona adherence, with Gwet’s AC2 ranging from 0.944 to 0.961 in the shared subset. ChikungunyaQA provides a reusable resource intended to support research on fine-tuning and evaluating language models, developing RAG-based systems, and investigating factual consistency and hallucinations in healthcare AI.
Small tabular datasets can produce uncertain performance estimates because limited sample information affects both model fitting and evaluation. This study developed a procedure calibrated through simulation to estimate the mean absolute discrepancy between performance on out-of-bag observations and performance on an independent reference sample across models fitted to bootstrap samples. The procedure uses quantities that can be observed during bootstrap evaluation. The primary experiment included 60 simulation scenarios, 3000 independent small datasets, four classification algorithms, and 100 bootstrap repetitions per dataset. Large independent reference samples were used during simulation to define the calibration targets. Gradient Boosting calibrators were developed using 40 scenarios and evaluated on 20 reserved scenarios. In reserved evaluation, AUC error estimation achieved an MAE of 0.0164, an R2 of 0.7217, and a Spearman correlation of 0.8855. For Brier Score error, the corresponding values were 0.0090, 0.7848, and 0.9143. The complete calibration model also showed better error estimation than reduced configurations based on bootstrap variability alone. Under the three alternative data-generating mechanisms examined, the calibration models maintained similar performance, while Monte Carlo assessment showed small uncertainty in the main stress-test estimates. A methodological case study with 50 participants illustrated application of the procedure when independent reference performance was unavailable. The procedure complements conventional AUC and Brier Score reporting by estimating the mean absolute discrepancy between out-of-bag and independent reference performance from information available in a small observed dataset.
The widespread adoption of smart TVs has made content overload in television programming increasingly severe, and high-quality recommender systems are critical for improving user experience. However, the broadcast-television-recommendation field has long lacked large-scale public datasets that realistically reflect its scenario-specific characteristics, which has constrained the development and validation of recommendation algorithms for broadcast television. To address this gap, this paper constructs and releases BroadCastRec, a large-scale real-world dataset designed specifically for broadcast television recommendation. The dataset is derived from 12 months of IPTV viewing records from a province in China. After data cleaning and anonymization, it contains more than 360,000 users, 44,000 programs, and 113 million user–program interaction records. BroadCastRec provides timestamps, user payment-tier information, and rich program metadata, thereby supporting large-scale program-level analysis of television viewing while leaving complete channel-level EPG reconstruction to a future release. Finally, experiments on benchmark models for click-through rate prediction, sequential recommendation, and multi-task learning show that existing models achieve substantially lower performance on BroadCastRec than on general-domain datasets. These results not only confirm the unique challenges of broadcast television recommendation, but also demonstrate the reliability of BroadCastRec as a new benchmark dataset.
