Creating sufficient motivation for unit testing is often a challenge in software engineering education. As well, it is often difficult for students to decide whether they have written sufficient tests. Classical coverage measures require thorough explanation and thus introduce great complexity, which risks to overwhelm programming novices. Instead, we started using mutation testing early on in programming courses, as mutations are an easy concept to understand. In addition, integrating mutation testing into projects is nowadays straightforward. Furthermore, hunting mutations can create a sense of gaming challenge, and thus, fun for novice programmers. In this paper, we describe experiences with mutation testing in the first and second semester of an introductory course on software development in a Bachelor's degree program on Computer Science. In order to critically evaluate our approach, we analyze in detail the meta data on testing activities generated in the students' repositories. Specifically, we analyze their work performance and identify typical performance patterns in terms of programming activities over time, as well as in terms of improvement of code quality during this process. Furthermore, we correlate these performance data to the exam performance. Finally, we classify leftover mutants in terms of skill levels according to Bloom's revised taxonomy of learning objectives. This provides an insight into students' abilities, and identifies where we need to adapt our teaching and exercises in the next iteration of teaching these courses, to better support our students along their learning path.
Computer science is essential in our daily lives, and many students enter higher education each year. Institutions must ensure and improve these students' academic success. One option is to design optional learning opportunities and e-learning strategies, often in combination with incentives. However, putting these strategies in action comes with heavy resource costs and is highly course-specific. For instructors planning those strategies, experience reports from similar courses can help to guide their decision-making process. This work presents the results of a five-year case study on the influence of incentives and continuous feedback in a software engineering course combining modeling and programming education. It explains the course structure, exams, and e-learning environment and compares the exam results of engaged and non-engaged students. For the investigated software engineering course, a statistical analysis evaluates a data set of five years of exam results and the corresponding results of a parallel incentives program. In this work, we present the benefit of incentives on exam results, a comparison between the size of the incentive program and the potential benefit, and the impact of e-learning on such a program.
Eye tracking technology offers valuable insights into how developers and users interact with software artifacts, tools, and interfaces. However, conducting empirical eye tracking research comes with a number of challenges. To assist researchers and students new to the field, this article provides a concise summary of the background as well as the key considerations. As an additional resource, we present a detailed checklist along with its application. Note that both the outline and the checklist are specifically tailored to, but not limited to, the context of software engineering research.
The web-based tool eyenalyzer supports prospective researchers in learning and performing statistical data analyses. To further improve the tool, we conducted a controlled experiment investigating its usability. In the experiment, a total of 41 participants complete common data analysis tasks with eyenalyzer while being recorded by an eye tracker and fill out a questionnaire on demographics, prior experience, difficulty of the tasks, and opinion of the tool. The results confirm a good overall usability of eyenalyzer, e.g., achieving an average SUS score of 82. Moreover, the data gives deeper insights into the use of certain elements of the tool.
The user interface (UI) and user experience (UX) design is of crucial importance for human-computer interaction (HCI), particularly in the context of web applications. In light of the high expectations of users and the competitive nature of the market, it is imperative to employ usability measurement techniques to avoid losing users. Heuristic evaluation (HE) is a cost- and resource-efficient method for evaluating the usability of websites in which evaluators are guided by heuristics. However, the level of expertise of the evaluators has a significant impact on the results, with experts identifying up to 50% more usability issues than novices. To address this gap, this paper proposes Eye Movement Modeling Examples (EMMEs) to demonstrate Jakob Nielsen's ten usability heuristics in an easy-to-understand format for all levels of experience while also incorporating expert knowledge. In particular, the eye movements and verbal feedback of a usability expert are recorded as the expert analyses the usability of a simple website application in terms of Jakob Nielsen's ten usability heuristics. This reveals the strategies and cognitive processes of the expert when assessing the usability of a website and makes them more tangible for non-experts or novices. The findings of a questionnaire-based assessment indicate that EMMEs are perceived as beneficial and supportive during the learning process. Ultimately, this comprehensive analysis not only enables a deeper understanding of heuristics for usability novices, but could also lead to EMMEs being applied more efficiently in diverse domains.
Large programs often contain duplicate parts, known as code clones. Programs riddled with code clones become difficult to reason about and modify. To avoid code clones and enable code reuse, programmers introduce abstractions such as functions and classes. Because abstraction is so important, it should be explicitly taught in programming courses and appropriately supported by tools. Unfortunately, development environments, including some for novices, do not always encourage abstraction. Instead, they facilitate the creation of code clones and ultimately hinder code reuse. This paper presents the Toolbox of Functions, an approach for teaching code reuse to beginner programmers in schools. This approach helps students to develop, collect, and reuse their own functions, as a simple form of abstraction. Learners are guided in creating and using their own library, without the complexity found in other environments. We implemented the approach in a publicly-available web platform for programming in Python. We collaborated with high school teachers who adopted the approach in their mandatory programming courses. Over the course of a year, more than 800 users executed over 30 000 programs that use the Toolbox of Functions. This initial experience suggests the potential of the approach to instill the principle of code reuse effectively.
Frequent student complaints about the complexity of traditional debuggers and the lack of visualization for key data structures motivated us to develop BugVis, a debugger designed for ease of use and customizability. BugVis allows users to focus on essential data structures and visualize them in a blackboard-like manner. In addition to standard debugging features, it supports both customizable visualizations and time-travel debugging, enabling users to navigate through execution history for deeper program analysis.
While successful software engineering students must demonstrate robust technical skills, the ability to acquire and master soft skills is just as crucial to their long-term professional success. These abilities are essential as they compliment technical expertise. Currently, research on soft skills primarily focuses on areas such as communication, teamwork and leadership. However, there is limited research exploring how confidence, adaptability and time management correlate with academic performance and contribute to building better prepared students for the job market. Time management, adaptation and building confidence are crucial non-technical abilities that allow software engineering students to succeed in a holistic way in academia and in their careers. In addition, the belief of a student in their own abilities to accomplish tasks and understand the concepts or modules learning outcomes presented to them forms the foundation of their growth. This study investigated whether soft skills such as confidence, adaptation and time management can be a predictor of success for software engineering students. The analysis of student self-reported data identified four principal clusters: high achievers (46.4%), mid-high performers (33.3%), mid-performers (13%) and developing performers (7.2%). Performance indicators suggest high reliability in grade prediction with an R2 value of 0.837 indicating that approximately 83.7% of the variance in student grades can be explained by the model results. Furthermore, an RMSE of 3.63 indicates that predictions deviate by less than 4 points on a 100 point scale. A standard deviation of RMSE (0.98) suggests consistent prediction accuracy across different student data. This study shows that software engineering curricula should also focus on equipping students with not only cutting edge technological tools and technical skills but also the most effective approaches to education by incorporating these soft skills.
The way software is developed is changing rapidly due to the general availability of generative AI tools. As a result, the software engineering education that is part of every computer science program needs to change. Especially in software engineering courses, such AI tools need to be used and practiced in a meaningful and useful way. The programming project is one such course at our university, and the curriculum will be expanded accordingly in the future. In this paper we describe our approach and a user study among the participants of the last programming project, in which we collected experiences with the use of current AI tools, in particular highlighting their usefulness and limitations. Our study focuses on identifying which aspects of the course students used AI tools for, evaluating successful applications, and uncovering remaining challenges.
The Adaptive Digital Learning Spaces (AdLer) project focuses on creating motivating, interactive, and gamified learning experiences through a 3D learning world. This paper investigates the integration of storytelling into AdLer and its impact on the students' learning experience quality. Based on existing research demonstrating the effectiveness of storytelling in education and the promising empirical results from the field of Computer Science within the context of AdLer, a "story element" has been introduced in AdLer. This allows lecturers to embed narrative frameworks into their courses via non-player characters (NPCs). The study evaluates whether the inclusion of storytelling in AdLer improves students' learning experiences and engagement compared to learning spaces without a narrative component. A quasi-experimental design was implemented, comparing three groups of students-two using non-narrative 3D learning worlds and one with an integrated story. Data collection utilized the "Game Experience Questionnaire" (GEQ), and both descriptive and inferential statistical methods were applied. Results revealed that students in the storytelling group experienced significantly higher quality of learning experience than those in the non-storytelling groups, confirming the hypothesis that storytelling enhances student experiences in a 3D learning world. These findings highlight the potential of storytelling to transform digital education by fostering deeper engagement.
Curricular specifications play an important role in the Higher Education sector and the domain of Computer Science and Software Engineering is characterized by a wide range of education programs with a broad range of topic. Therefore, recommendation frameworks play an important role and their usage is beneficial for a unification of education profiles in a systematic way. This research is contributing to this development by exploring how a recommendation for the domain of Business Informatics in German speaking countries can be improved by formalizing the recommendations in a semantic model that relies on sophisticated European ontologies in the domain like the European Learning Model (ELM) and related data models. It employs Generative Artificial Intelligence Systems to create semantic models in an experimental way and evaluates the resulting model quality. The results show that a formalization using GenAI has a high potential, but currently also shows deficits in the correctness of the resulting models, requiring human oversight during the model creation.
Software processes are the game plan to develop comprehensive software systems, and there are many ways to teach software processes. Nowadays, the normal approach is to teach students to use software processes correctly, usually by explaining what agile methods are about or by introducing tools that already implement selected processes, and then letting students apply such selected tools and methods in smaller projects or project courses. However, while this approach addresses the application of, e.g., Scrum or Unit Testing, the question of how a company develops its own software process is not answered. With this paper, we address this issue and share our experiences from more than 15 years of teaching the analysis, design, realization, and improvement of software processes in our joint Software Process Engineering course, which is based on a structured Software Process Improvement model. We contribute insights into the process improvement model that builds the foundation of the course, an overview of the course content, and we share our experiences, lessons learned, and recommendations for teachers.
Learning analytics can provide valuable insight to improve the learning experience for students. We are successfully using the ATTEST test system in our Real-Time Operating Systems course for two semesters. Besides the direct advantage of providing low-latency feedback to students, analysis of usage data provides the foundation for adaptation decisions to improve the course in future semesters. This paper answers 11 concrete questions, including the preferred working day and time. It analyzes the number of test cases passed as the deadline approaches and correlates the starting time to the achieved results. Finally, a technical analysis shows that the test system response time is sufficiently short despite the constraint of testing on the actual target MSP430 microcontroller. Thus, students can receive multiple feedback iterations within an average working session length without interrupting their workflow.
This paper explores the automation of generating and dispatching Experience API (xAPI) statements for comprehensive tracking of user interactions in e-learning environments. It introduces the react-xapi-wrapper library, an extension of the xAPI JavaScript library designed for use in web applications. Key aspects discussed include the library's features, its integration into a web-based adaptive learning system (ALS) for software engineering, and the custom verbs used. The goal is to reduce implementation effort for tutors and developers while taking advantage of xAPI's interoperability, scalability, and ability to track student learning activities and behaviors, laying the foundation for more responsive and personalized learning experiences.
To engage a broader spectrum of computer science (CS) students, particularly underrepresented groups such as novices and female students, this study introduces a unique interdisciplinary online course. This course merges the technical rigour of artificial intelligence (AI) with the aesthetic allure of music, underscoring the importance of beauty and arts in technology. It demonstrates AI's applications in music through song recommendation algorithms and composition generation, captivating students and enhancing their interest in CS. Findings from a pre-test-post-test survey study involving two school classes (24 students, female: 17, male: 7, mean age: 17.63) indicate a significant rise in students' engagement for CS with large effect sizes observed. These results highlight the potential of integrating arts into technology education, not only as an aesthetic enhancement but as a means to broaden the appeal and understanding of CS. This approach aligns with the growing emphasis on STEAM (Science, Technology, Engineering, Arts, Mathematics) education, aiming to make CS more accessible and prepare students for the diverse demands of the 21st-century workplace.
Adaptive learning environments aim to enhance student engagement and learning efficiency by tailoring educational content to individual needs. This paper presents a modular tool landscape designed to extend a learning management system like Moodle with adaptive learning paths based on learner profiles. To this end, the proposed architecture integrates multiple AI-driven tools - like Bayesian networks and Markov models - to generate personalized learning paths. The framework is implemented as a Moodle plugin named Pythia, which facilitates the selection and sequencing of learning elements by analyzing learning styles, completion data, and learning analytics. The main goal of this work is to provide lecturers a modular architecture for an adaptive learning management system that can be used and expanded the way they want it. Therefore, the present work has the following contributions: The architecture with its tools has to be discussed on the basis of flexibility in tool integration. They should be open for extensions, the functionality of the used algorithms should be transparent, and they should offer the possibility of psychological questionnaire integration.
Eye tracking is a powerful tool for investigating cognitive processes, yet the classification performance of state-of-the-art eye movement classifiers often relies on manually defined thresholds, making them vulnerable when labeling data with varying noise levels. To address this limitation, deep learning approaches promise to replace such manual thresholds using automated and adaptive methods. In recent years, several studies have proposed deep learning models for eye movement classification. However, comparisons of these algorithms are rare. This study fills the research gap by comparing two publicly available neural networks for eye movement classification with the Tobii I-VT algorithm used in the Tobii Pro Lab software. The comparison evaluates classification performance using metrics such as F1-score, Cohen's Kappa, and Matthew's Correlation Coefficient. The analysis leverages datasets recorded with Tobii Pro Spectrum and Tobii Pro Fusion eye trackers from two separate studies with sampling frequencies of 300 Hz and 250 Hz, respectively. Together, these datasets include approximately 5000 eye movements, comprising fixations, saccades, and post-saccadic oscillations. The results provide interesting insights: On the noisier 250 Hz dataset, the Tobii I-VT algorithm excels in sample-level evaluation, while deep learning approaches outperform it in event-level saccade detection. In contrast, on the 300 Hz dataset, one deep learning model achieves superior performance on the event-level. This work aims to assist researchers in choosing the most suitable eye movement classifier for their datasets, making a significant contribution to the field of eye movement classifier comparisons.
Despite being a well-structured domain, software engineering lacks standardized definitions, metrics, and theories for analyzing eye movements in domain-specific tasks. This gap can be addressed by adapting models from other fields, such as radiology and psychology. In particular, the holistic models of image perception provide a suitable framework for software engineering applications. This paper introduces a unified model of visual perception focused on eye movements during code reviews. It is based on prior research, findings from other studies, and cross-domain theories. Empirical studies on C and C++ code reviews confirm a phase-based process, where experts switch between global scanning and focal viewing. In addition, significant differences in the fixation rate, fixation duration, number of saccades, and AOI-specific metrics highlight the role of expertise in visual processing. The proposed model offers a structured framework for eye-tracking analysis in software engineering, defining relevant metrics and supporting future refinements across various software engineering tasks.
Learning Management Systems have become fundamental in higher education for delivering and managing educational content. However, traditional implementations often lack the ability to provide personalized learning experiences and detailed insights into learner behavior. A new approach addresses these limitations by enabling more detailed Learning Analytics through the integration of interactive H5P content and the implementation of Moodle's LogStore xAPI plugin to send Experience API-based statements within a Moodle Learning Management System. By extending this plugin, detailed user interactions, including activity outcomes, scores, durations and completion status, are captured as Learning Records and stored in a Learning Record Store for further analysis. The enriched Learning Records enable more advanced Learning Analytics that provide deeper insights into student behavior, such as identifying learning preferences, activity patterns, and knowledge levels. Future work will involve developing a recommendation system that uses the Learning Analytics data to identify the next activity best suited to fill learning gaps. The system should monitor learner preferences to maintain engagement, enable adaptive learning paths and offer personalized suggestions. Further efforts will focus on refining the system and evaluating its effectiveness in improving educational outcomes.
Eye tracking provides insight into human cognition and attention, enabling a variety of use cases in different areas of research. However, traditional eye tracking research software suffers from several limitations. First, it lacks native web integration, making it difficult to integrate eye tracking into web applications. Second, due to the first limitation, mapping gaze data to evolving HTML elements in dynamic web environments is challenging. Third, collecting gaze data based on high-precision eye tracking hardware requires the installation of dedicated software. To address these challenges, we present WebGazeTrack, a lightweight, web-based eye tracking tool implemented as a Chrome extension that aims to radically simplify eye tracking research on the web. By using CSS selectors for dynamic AOI definitions and WebUSB for direct hardware communication, WebGazeTrack offers plug-and-play deployment without direct software installation on the host system and native integration into web pages. Currently, the extension supports the Tobii Pro Fusion eye tracker by wrapping the Tobii Pro SDK. It can be used as a standalone Chrome extension or, based on its API, as a foundation for developing custom applications. In this paper, we also propose three specific applications to demonstrate the versatility and potential of our approach.
