This study analyzes the impact of ChatGPT on usability and cognitive load in programming education for undergraduate students in an Information Technology Management program. The goal is to evaluate whether ChatGPT improves learning outcomes in programming topics. A comparative research design was used with two groups: a traditional instruction control group and an experimental group using ChatGPT as a support tool. Data were collected using the System Usability Scale (SUS), a custom usability questionnaire, cognitive load surveys, and programming performance evaluations. The results indicate that the students using ChatGPT reported greater usability and a lower cognitive load. They also showed improved comprehension and problem-solving skills. However, improper use of the tool can lead to superficial learning, highlighting the need for structured guidance. The findings suggest that, when integrated appropriately, ChatGPT can improve the learning experience by reducing mental effort and enhancing participation in programming education. Recommendations are offered to help educators incorporate AI tools effectively and responsibly.
This paper focuses on the topic of the Internet of Things (abbr. IoT) in the context of higher education and academic understanding of it. When briefly looking at the IoT course curriculum at our department, we suspected that the curriculum contents are not adhering to the definition of IoT. The goal of our work was to pinpoint the correct definition of IoT, which can be used to bring contents of the IoT courses as close to the truth as possible. Secondarily, we reviewed available articles and reviews of formerly and currently taught IoT or related courses and evaluated whether their approach and contents were correct when considering the definition of IoT. We summarise the issues present in existing works and identify which specific parts are problematic, according to our assessment. Improving IoT courses is crucial since it shapes a student's understanding of the IoT paradigm and allows them to use it or even develop it in the future. Provisioning our students with a needed set of skills will make them more suitable for research, development, and industry-related futures.
Generative language models present significant advancements in artificial intelligence with increasing applications in software engineering education. This paper explores the potential of customized generative dialogue models for automated assessment of programming assignments. The research introduces KP Assistant, a tailored implementation based on GPT-4o developed for a Component Programming university course. The research evaluated the effectiveness of this approach in generating relevant questions about source code, assessing student understanding, and providing objective feedback through a series of experiments with various game implementations and student testing. The findings demonstrate the feasibility of integrating such models into educational workflows while also acknowledging their present limitations. The study provides a framework for implementing similar systems in programming education, showing how generative AI can augment traditional assessment methods while maintaining pedagogical integrity.
As Large Language Models (LLMs) become increasingly integrated into secure software development workflows, a critical question remains unanswered: can these models not only detect insecure code but also reliably classify vulnerabilities according to standardized taxonomies? In this work, we conduct a systematic evaluation of three state-of-the-art LLMs - Llama3, Codestral, and Deepseek R1 - using a carefully filtered subset of the Big-Vul dataset annotated with eight representative Common Weakness Enumeration categories. Adopting a closed-world classification setup, we assess each model's performance in both identifying the presence of vulnerabilities and mapping them to the correct CWE label. Our findings reveal a sharp contrast between high detection rates and markedly poor classification accuracy, with frequent overgeneralization and misclassification. Moreover, we analyze model-specific biases and common failure modes, shedding light on the limitations of current LLMs in performing fine-grained security reasoning.These insights are especially relevant in educational contexts, where LLMs are being adopted as learning aids despite their limitations. A nuanced understanding of their behaviour is essential to prevent the propagation of misconceptions among students. Our results expose key challenges that must be addressed before LLMs can be reliably deployed in security-sensitive environments.
While server-side assessment of programming exercises, with its ease of installing diverse compilers and execution environments, is common, it presents three key limitations: the necessity of a constant Internet connection, increased bandwidth consumption, and centralized execution load. The alternative is to rely on JavaScript, the single programming language supported by all standard web browsers. This paper introduces Osiris, a pure JavaScript multi-language transpiler designed to enable the execution of diverse programming languages within web browsers. Targeted primarily at Virtual Learning Environments (VLE) for language programming education, Osiris employs a parser generator to translate small student programs into JavaScript based on language-specific grammars with semantic rules. It also includes a comprehensive, though not exhaustive, JavaScript library that emulates the standard libraries of its supported languages. Validation of Osiris indicates the pedagogical effectiveness of browser-based transpilation for introductory programming education.
The rapid adoption of GenAI for code generation presents unprecedented opportunities and significant security challenges. Raising awareness about secure coding is critical for preventing software vulnerabilities. To investigate how Generative AI can best support secure coding, we built an AI Secure Coding platform, an interactive training environment that embeds a GPT-4 based chatbot directly into a structured challenge workflow. The platform comprises a landing page, a challenges page with three AI-generated tasks, and a challenge page where participants work with code snippets. In each challenge, developers (1) identify vulnerabilities by reviewing code and adding comments, (2) ask the AI for help via a chat based interface, (3) review and refine comments based on AI feedback, and (4) fix vulnerabilities by submitting secure patches. The study involved 18 industry developers tackling three challenges. Participants used the AI Secure Coding Platform to detect and remediate vulnerabilities and then completed a survey to capture their opinions and comfort level with AI assisted platform for secure coding. Results show that AI assistance can boost productivity, reduce errors, and uncover more defects when treated as a "second pair of eyes," but it can also foster over-reliance. This study introduces the AI Secure Coding platform, presents preliminary results from a initial study, and shows that embedding GenAI into a structured secure-coding workflow can both enable and challenge developers. This work also opens the door to a new research field: leveraging GenAI to enable secure software development.
This paper proposes the use of introductory programming assignments based on mutation testing where students are asked to write tests rather than code. We believe such exercises can be used to teach code reading skills before students could write the corresponding programs on their own. Furthermore, feedback for such exercises can be automatically generated using testing tools. We have extended an existing web-based system for programming exercises with such mutation testing assignments and show some example use cases. This is on-going work that has yet to be validated in the classroom.
This paper presents an ongoing work on the application of concept maps to teaching an introductory Object-Oriented Analysis and Design course for Computer Science students. There exist previous works that introduce the concept map model in these object-oriented courses. However, these works do not usually go deeply enough into the transition from concept maps to static class diagrams. Although concept maps present some clear advantages when defining the abstractions present in object-oriented software modeling, some drawbacks may also appear if the transformation from these maps to class diagrams when the task is carried out through simplistic rules. In this paper we propose an approach, which is illustrated through a use case, to transition from a concept map to a static class diagram in a more realistic way.
Undergraduate data science education faces a scalability challenge: addressing a high volume of diverse student questions stemming from varying levels of prior knowledge, technical skills, and learning styles - while ensuring timely and accurate responses. Traditional solutions like manual replies or generic chatbots often fall short in terms of contextual relevance, speed, and efficiency. To tackle this, we introduce RAGent, a Retrieval-Augmented Generation (RAG) agent tailored for a university-level data science course at the University of Maryland. RAGent integrates course-specific materials - lecture notes, assignments, and syllabi - to deliver fast, context-aware answers while maintaining low computational overhead. A central innovation of RAGent is its query classification system, which categorizes student questions into: (i) directly answerable, (ii) relevant but unresolved (requiring instructor input), and (iii) irrelevant or out-of-scope. This system uses semantic similarity, keyword relevance, and dynamic thresholds to drive a targeted prompting strategy, enhancing response accuracy. Another key feature is RAGent's self-learning loop, which continuously improves performance by integrating resolved queries into its knowledge base and flagging unresolved ones for review and retraining. This dual mechanism ensures both immediate adaptability and long-term scalability. We evaluate RAGent using standard NLP metrics (accuracy, precision, recall, F1-score) and report strong performance in filtering and answering student queries. In a user study with 125 students, over 94% expressed a desire to keep RAGent in the course, citing improved clarity and helpfulness. These results suggest that RAGent significantly enhances support in data science education by providing accurate, contextual responses and reducing instructor workload - offering a scalable, adaptive alternative to conventional support methods. Future work will explore deployment across additional courses and institutions to further validate the RAGent's adaptability.
Artificial Intelligence (AI) has assumed an increasingly prominent role in education, transforming the dynamics of teaching and learning while introducing new pedagogical opportunities and challenges. In computer programming education, generative AI tools have had a particularly profound impact. Historically, computer programming education has emphasized problem-solving skills, syntax accuracy, and code efficiency. However, the emergence of generative AI models capable of supporting automatic code generation, producing high-quality code snippets and entire programs, personalized explanations and tutoring, and real-time debugging, has triggered a paradigm shift. These tools make learning processes and assessment less effective and less clear about students' true knowledge. In this context, the paper explores three key dimensions: the broader impact of AI in education, the new challenges that AI presents in teaching and learning computer programming in higher education, and the implications for student assessment, an essential element of the educational process. To investigate these topics, we conducted an online survey targeting Portuguese higher education instructors teaching programming-related courses. Our primary objective was to understand the changes introduced in evaluation methods and criteria due to the growing use of generative AI tools, particularly those focused on code generation.
Educational content creation is being transformed by artificial intelligence (AI) tools that can generate, process, and deliver learning materials across different environments. This paper presents a novel approach for converting AI-generated podcasts into immersive virtual reality (VR) learning experiences. We use the NotebookLM platform to create educational podcast audio recordings, which are then transformed into VR learning objects through automatic transcription, speaker diarization, and integration into a VR environment. The resulting VR learning objects address multiple learning preferences - supporting auditory learners through audio and visual learners through virtual presenters, and kinesthetic learners through immersive interaction. We evaluated this approach within a software engineering course, demonstrating its practical applicability and educational effectiveness.
This paper presents an innovative strategy for assessing programming exercises in higher education, leveraging generative artificial intelligence (GAI) to support automated grading while ensuring transparency, fairness, and pedagogical relevance. The proposed approach is framed within the TechTeach paradigm and integrates multiple tools - HackerRank for code development, Google Forms and Sheets for submission and prompt generation, and the ChatGPT API for intelligent evaluation. The correction process is personalised using student-specific variables (e.g., student ID, birth date, performance in group work), which are dynamically embedded into the statement and prompt. The GAI algorithm evaluates the code and performs authorship verification using peer-assessed effort data, enabling the detection of potential plagiarism or misuse of AI tools. A case study was conducted in the 2023/2024 edition of the Web Programming course at the University of Minho, which involved 118 students. Results indicate that the method produced consistent and meaningful grades, reflecting a balanced perception of difficulty from students. The system also includes a gamification mechanism (Grade Rescue) for managing contested cases. The achieved findings (>90% of students approved the exercise model) support the viability of GAI-based evaluation as a scalable and effective solution for programming education, while maintaining academic integrity and enhancing the student experience.
This paper outlines the methods employed by several instructors within a single department to implement standards-based assessments. The authors began integrating standards across multiple courses in their computer science, cybersecurity, data science, and mathematics programs. This shift was driven by a desire to promote equity in grading and to address the growing influence of artificial intelligence, which can obscure a student's true understanding. In this work, the authors examine the supporting research that guided their motivation and informed their implementation of various grading techniques. With an emphasis on courses involving technology, they also detail the processes they use to manage the new assessments, provide examples of assessment questions, and share key lessons learned in making this transition successful for both instructors and students. This work addresses a significant gap in the literature, as there appears to be a notable lack of resources on the application of standards-based grading in technical disciplines.
With the increasing integration of large language models (LLMs) into software development and programming education, concerns have emerged about the security of AI-generated code. This study investigates the security of three open source code generation models. Codestral, DeepSeek R1, and LLaMA 3.3 70B using structured prompts in Python, C, and Java. Some prompts were designed to explicitly trigger known vulnerability patterns, such as unsanitized input handling or unsafe memory operations, in order to assess how each model responds to security-sensitive tasks. The findings reveal recurring issues, including command execution vulnerabilities, insecure memory handling, and insufficient input validation. In response, we propose a set of recommendations for integrating secure prompt design and code auditing practices into developer training. These guidelines aim to help future developers generate safer code and better identify flaws in GenAIgenerated output. This work offers an initial analysis of the limitations of GenAI-assisted code generation and provides actionable strategies to support the more secure and responsible use of these tools in professional and educational contexts.
Gamification in educational context involves applying game design elements and principles to enhance the learning experience. By incorporating the motivational features of games, it aims to engage students and support educational goals. The work presented in this article is part of ThinkGame Erasmus+ project. The project's goal is to encourage the use of Learning Management System (LMS) tools, such as lessons, wikis, and online tests to create gamified experiences in programming classes. These innovative strategies are intended to boost student motivation and creativity by incorporating compelling narratives, adaptable challenges, collaborative tasks, and continuous feedback. Another important challenge addressed in the project was fostering creativity among teachers, encouraging them to transform conventional, non-gamified resources into engaging and thought-provoking activities for students. A case study composed of ten gamified experiences was developed at Polytechnic Institute of Braganca, one of the project partners, during one semester in Imperative Programming subject.
The combination of storytelling and gamification in educational settings has emerged as a method to enhance student engagement and learning outcomes. Through an overarching narrative, course content can be connected while providing context for gamified exercises, creating a motivating and competitive learning experience. However, a narrative that resonates with one student may not interest others. The presented solution to this problem is to offer multiple narratives for students to choose from. This enables the students to engage with the material in ways that align with their interests and motivations. Yet, managing multiple narratives presents several challenges. Each narrative must cover all syllabus topics equally, and every exercise must be available across all narratives while maintaining consistent difficulty levels and learning objectives. This paper presents a systematic approach for creating gamified courses with multiple narratives. The methodology includes the development of a base course template and its narrative variations, along with transformation processes to generate exercises in the FGPE Ecosystem, namely AuthorKit and FGPE PLE. The final output is a single Moodle MBZ file that can be imported into Moodle, a widely adopted learning management system.
Questions about Learners' Code (QLCs) assess programming students' program comprehension skills by providing personalised questions targeting the students' own program code. We conducted a preliminary, experimental implementation of integrating QLCs in the Automated Assessment System (AAS) used in an introductory programming course using Java. QLCs targeted some of the code assignments which students had to complete during the course. We collected 889 answers to QLCs, answered by 13 students over five course modules. We found that as the complexity of exercises increases, the success rate of the same type of QLC may not improve, and even exhibit a decline over time. We further analysed incorrect answers individually to relate them to possible misconceptions.
