Programming Error Messages (PEMs) are often misunderstood by students who then miss the opportunity to benefit from them. We have developed and used 'error hunt' lessons using Java, Python, and C PEMs. The activities are supported by videos (images shown below). The materials are targeted at novice programmers.

Computer Science Curricula 2023 (CS2023) was released in early 2024 by the Joint Task Force of the ACM, IEEE-Computer Society, and AAAI. Viewed from an educational perspective, CS2023 reflects the decennial changes in computer science (CS) since the release of the previous curricular guidelines, Computer Science Curricula 2013. CS2023 comprises curricular content and curricular practices: the former includes updates to the computer science knowledge areas and a competency model framework while curricular practices cover other aspects such as program design and delivery issues. As CS2023 has proposed several changes to curricular content and emphases, this special session examines efforts that have already started to align computer science to reflect the CS2023 curricular guidelines. The presenters will discuss how their programs were reexamined in detail and changes are being proposed to align with CS2023 better. The session targets educators, administrators, and professionals interested in understanding computer science curricular issues and aligning their programs for the future.

Adversarial Thinking (AT) is essential in cybersecurity, fostering strategic problem-solving by anticipating worst-case scenarios. However, its integration into early computing education, especially in the first two years, remains underexplored. Introductory courses like CS 0 build foundational skills but are challenging to implement in resource-limited community colleges. Strengthening AT in these students can enhance their workforce readiness and support transfers to four-year programs. Servingness describes how Hispanic-Serving Institutions (HSIs) go beyond merely enrolling Latinx students to address their specific cultural, academic, and social needs. It involves creating inclusive environments where students feel a strong sense of belonging, supported by culturally relevant practices, leadership opportunities, and community engagement. These environments generate feelings of being accepted, valued, and included within a community or group, also called a sense of belonging. This work presents research focused on the incorporation of AT within the first two years of computer science education at community colleges, in particular to a course CS 0. By examining the integration of AT principles in early coursework, the study identifies key characteristics and a sense of belonging that are instrumental in cultivating AT capabilities among students. The findings aim to offer actionable insights for educators in community colleges, enabling them to more effectively prepare students for the complexities of modern computing careers and to address the broader needs of the cybersecurity field. Through this targeted educational approach, students can develop a more robust understanding of adversarial strategies, enhancing their overall computational thinking and problem-solving skills.

Making errors is part of the programming process – even for the most seasoned professionals. Novices in particular are bound to make many errors while learning. It is well known that traditional (compiler/interpreter) programming error messages have been less than helpful for many novices and can have effects such as being frustrating, containing confusing jargon, and being downright misleading. Recent work has found that large language models (LLMs) can generate excellent error explanations, but that the effectiveness of these error messages heavily depends on whether the LLM has been provided with context – typically the original source code where the problem occurred. Knowing that programming error messages can be misleading and/or contain that serves little-to-no use (particularly for novices) we explore the reverse: what happens when GPT-3.5 is prompted for error explanations on just the erroneous source code itself – original compiler/interpreter produced error message excluded. We utilized various strategies to make more effective error explanations, including one-shot prompting and fine-tuning. We report the baseline results of how effective the error explanations are at providing feedback, as well as how various prompting strategies might improve the explanations' effectiveness. Our results can help educators by understanding how LLMs respond to such prompts that novices are bound to make, and hopefully lead to more effective use of Generative AI in the classroom.

Sense of belonging in education encompasses an individual's experiences of feeling accepted, valued, and included within educational environments. Not all students experience the same level of sense of belonging, and those from marginalised or under-represented groups are shown to report lower levels of belonging compared to others in their cohorts. Fostering a stronger sense of belonging amongst these groups to address this unevenness is beneficial for retaining diversity and closing representation gaps, as students persist when they feel they belong. However, student belonging is complex and challenging to measure. For the development of meaningful interventions to level sense of belonging across various groups by improving belonging for those with lower belonging, there is a need for clarity on how 1) sense of belonging is theorised in research into undergraduate students, 2) how it is measured, and 3) how belonging has been operationalised to improve diversity and student retention. This review focuses specifically on computing, as a discipline with persistent representation gaps limiting its development, synthesising existing studies investigating sense of belonging among undergraduate computing students. The PRISMA extension for scoping reviews (PRISMA-Scr) was utilised to source and review 22 relevant peer-reviewed articles published between 2013 and 2022. The review shows that most existing research follows a quantitative study. The research reveals how multiple barriers to student belonging are experienced differently across different student populations. Belongingness has significant disparities relating to gender, race, and sexuality that may explain representation gaps, but attention needs to be paid to how these identities intersect to shape student experiences. The research also suggests various social and academic interventions that can raise sense of belonging. These findings provide valuable insights into undergraduate student belonging that may inform the development of targeted interventions aimed at retaining diverse student cohorts and closing representation gaps, as well as informing future research aimed at fostering and improving sense of belonging among undergraduate computing students.

Code reading and comprehension skills are essential for novices learning programming, and explain-in-plain-English tasks (EiPE) are a well-established approach for assessing these skills. However, manual grading of EiPE tasks is time-consuming and this has limited their use in practice. To address this, we explore an approach where students explain code samples to a large language model (LLM) which generates code based on their explanations. This generated code is then evaluated using test suites, and shown to students along with the test results. We are interested in understanding how automated formative feedback from an LLM guides students' subsequent prompts towards solving EiPE tasks. We analyzed 177 unique attempts on four EiPE exercises from 21 students, looking at what kinds of mistakes they made and how they fixed them. We found that when students made mistakes, they identified and corrected them using either a combination of the LLM-generated code and test case results, or they switched from describing the purpose of the code to describing the sample code line-by-line until the LLM-generated code exactly matched the obfuscated sample code. Our findings suggest both optimism and caution with the use of LLMs for unmonitored formative feedback. We identified false positive and negative cases, helpful variable naming, and clues of direct code recitation by students. For most students, this approach represents an efficient way to demonstrate and assess their code comprehension skills. However, we also found evidence of misconceptions being reinforced, suggesting the need for further work to identify and guide students more effectively.

In the rapidly evolving Generative AI (GenAI) landscape, source code and natural language are being mixed and used in new ways. This presents opportunities for rethinking teaching practice in Introductory Programming (CS1) courses that includes, but goes beyond, assessment. In this paper we examine the reasons why and how instructors who are early adopters of GenAI are using it in their teaching, and why others are not. We also explore the changes and adaptations that are currently being made to practice. This is achieved by synthesizing insights from several recent studies that have collected primary data from introductory programming instructors who are teaching with, considering teaching with, or actively not teaching with GenAI. Due to the fast pace of GenAI development and adoption, the fixed-pace and cyclical nature of education, and the relatively slow pace of research (including ethical approvals) and publication cycles, research with primary data from instructors is only being published relatively recently. In computing education, there is not yet enough published research with primary data from CS1 instructors to warrant a systematic literature review, although in the next year this will likely be possible. Based on an analysis of the nascent research that has been published, we propose emerging and flexible guidelines on how CS1 instructors could adapt their practice based on what others have done so far. These guidelines highlight important factors to consider when integrating GenAI in CS1 courses, which for many is only beginning.

The sudden emergence of large language models (LLMs) such as ChatGPT has had a disruptive impact throughout the computing education community. LLMs have been shown to excel at producing correct code to CS1 and CS2 problems, and can even act as friendly assistants to students learning how to code. Recent work shows that LLMs demonstrate unequivocally superior results in being able to explain and resolve compiler error messages—for decades, one of the most frustrating parts of learning how to code. However, LLM-generated error message explanations have only been assessed by expert programmers in artificial conditions. This work sought to understand how novice programmers resolve programming error messages (PEMs) in a more realistic scenario. We ran a within-subjects study with n = 106 participants in which students were tasked to fix six buggy C programs. For each program, participants were randomly assigned to fix the problem using either a stock compiler error message, an expert-handwritten error message, or an error message explanation generated by GPT-4. Despite promising evidence on synthetic benchmarks, we found that GPT-4 generated error messages outperformed conventional compiler error messages in only 1 of the 6 tasks, measured by students’ time-to-fix each problem. Handwritten explanations still outperform LLM and conventional error messages, both on objective and subjective measures.

Model curricula for baccalaureate computer science (CS) have been published regularly from 1968 through 2013. In early 2021, the ACM, IEEE-Computer Society, and the Association for the Advancement of Artificial Intelligence (AAAI) constituted a task force to revise these curricula, which have now been released as Computer Science 2023 Curricula (CS2023). The CS2023 curricular guidelines inform educators and administrators on the what, why, and how to cover undergraduate CS over the next decade. Like past guidelines, CS2023 provides curricular content - a knowledge model largely backward compatible with CS2013, supplemented by a competency framework influenced by Computing Curricula 2020 (CC2020) - and complementary curricular practices, which include articles by international experts on program design and delivery. Ongoing drafts of CS2023 were disseminated via the CS2023 website, along with regular publications or presentations at various computing education venues. This panel focuses on three among the 17 CS2023 knowledge areas: Society, Ethics, and the Profession (SEP), Artificial Intelligence (AI), and Security (SEC). While the other 14 knowledge areas remain important in CS education, these three have been in the news due to inadequacies in current CS education. The panelists, who served on the CS2023 steering committee, will discuss how CS2023 addresses these challenges. Attendees will appreciate the approach taken by CS2023 toward these three hot-button items of CS education, especially constraints on curriculum design, and how CS2023 may be used to educate the next generation of CS graduates to rise to these three challenges.

This study collected insights from post primary Computer Science teachers regarding the integration of Artificial Intelligence (AI) and Machine Learning (ML) into the Leaving Certificate Computer Science (LCCS) curriculum in Ireland. By surveying teachers, the research evaluates current practices, identifies educational needs and challenges, and seeks to inform future developments to enhance AI education for students. Preliminary findings suggest a range of perspectives, highlighting the importance of professional development and the growing interest in AI within the curriculum.

In a previous Birds of a Feather discussion, we delved into the nascent applications of generative AI, contemplating its potential and speculating on future trajectories. Since then, the landscape has continued to evolve revealing the capabilities and limitations of these models. Despite this progress, the computing education research community still faces uncertainty around pivotal aspects such as (1) academic integrity and assessments, (2) curricular adaptations, (3) pedagogical strategies, and (4) the competencies students require to instill responsible use of these tools. The goal of this Birds of a Feather discussion is to unravel these pressing and persistent issues with computing educators and researchers, fostering a collaborative exploration of strategies to navigate the educational implications of advancing generative AI technologies. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed leaders to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.

Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how novices interact with these tools in practice. We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical introductory programming (CS1) assignment. Through observations and interviews we explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students. We consider design implications of these findings, specifically in terms of how tools like Copilot can better support and scaffold the novice programming experience.

Sense of belonging, or belongingness in academia, is an individual’s personal conviction as to their acceptance as a valued member of an academic community. The importance of belongingness lies in correlations with motivation, persistence, mental health and well-being. Prior work has shown that belongingness can be lower in students who are minoritised due to their gender, race, nationality, socio-economic status, religion and/or disability. However, there is limited research on how a student’s university entry route impacts their sense of belonging in Computer Science and Other Science disciplines. In Ireland’s higher education system, university entry is primarily managed through the Central Applications Office (CAO) process, which predominantly serves students transitioning directly from secondary school to university. Additionally, alternative access routes are available for students from socio-economically disadvantaged backgrounds, those with disabilities, and mature students. At University College Dublin (UCD), entry is facilitated through both the traditional direct-entry school leaving route and combined access routes, each serving distinct student demographics. To investigate how these varying entry routes influence students’ sense of belonging, we utilised a survey adapted from the validated ‘Math Sense of Belonging Scale’ to examine the belongingness of undergraduate Science students, including Computer Science students. We examined how the belongingness of students varies by university entry route, and how the belongingness of students in Computer Science varies by university entry route compared to Other Science students. Our results reveal a significant difference in belongingness between students entering university through the direct-entry school leaving route, compared to those who entered via combined access routes (e.g. university admissions schemes for school leavers from socio-economically disadvantaged backgrounds, with disabilities, and mature students). Specifically, within Computer Science, students entering university through the direct-entry school leaving route had significantly higher belongingness than combined access route students. These results provide insight that may help others improve the belongingness of undergraduate Computer Science students.

The interaction of computing and society, the requirement for ethical development and use of computing technology, and the responsibility our profession has to society, have never been greater. This poster provides an in-depth picture of how the "Society, Ethics, and The Profession" (SEP) knowledge area (KA) in the CS2023 curricular guidelines was designed, spanning the evolution of the KA, and how the CS2023 SEP KA was designed so that future CS graduates will have the SEP knowledge, skills, dispositions, and competencies to effectively and responsibly carry out their work as professional computer scientists. The ultimate goal is to help educators in integrating these new guidelines into their curricula.

Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming, such as forming incorrect conceptual models of the problem or having a false sense of progress after testing their solution. Novices are typically unaware of how these difficulties are hindering their progress. Meanwhile, many novices are now programming with generative AI (GenAI), which can provide complete solutions to most introductory programming problems, code suggestions, hints for next steps when stuck, and explain cryptic error messages. Its impact on novice metacognition has only started to be explored. Here we replicate a previous study that examined novice programming problem solving behavior and extend it by incorporating GenAI tools. Through 21 lab sessions consisting of participant observation, interview, and eye tracking, we explore how novices are coding with GenAI tools. Although 20 of 21 students completed the assigned programming problem, our findings show an unfortunate divide in the use of GenAI tools between students who did and did not struggle. Some students who did not struggle were able to use GenAI to accelerate, creating code they already intended to make, and were able to ignore unhelpful or incorrect inline code suggestions. But for students who struggled, our findings indicate that previously known metacognitive difficulties persist, and that GenAI unfortunately can compound them and even introduce new metacognitive difficulties. Furthermore, struggling students often expressed cognitive dissonance about their problem solving ability, thought they performed better than they did, and finished with an illusion of competence. Based on our observations from both groups, we propose ways to scaffold the novice GenAI experience and make suggestions for future work.

Challenges and opportunities faced by computing educators and students adapting to LLMs capable of generating accurate source code from natural-language problem descriptions.

Generative AI (GenAI) and large language models in particular, are disrupting Computer Science Education. They are proving increasingly capable at more and more challenges. Some educators argue that they pose a serious threat to computing education, and that we should ban their use in the classroom. While there are serious GenAI issues that remain unsolved, it may be useful in the present moment to step back and examine the overall trajectory of Computer Science writ large. Since the very beginning, our discipline has sought to increase the level of abstraction in each new representation. We have progressed from hardware dip switches, through special purpose languages and visual representations like flow charts, all the way now to “natural language.” With the advent of GenAI, students can finally change the abstraction level of a problem to the “language” they've been “problem solving” with all their lives. In this paper, we argue that our programming abstractions were always headed here – to natural language. Now is the time to adopt a “Prompts First” approach to Computer Science Education.

There is a significant gap in Computing Education Research (CER) concerning the impact of Large Language Models (LLMs) in advanced stages of degree programmes. This study aims to address this gap by investigating the effectiveness of LLMs in answering exam questions within an applied machine learning final -year undergraduate course. The research examines the performance of LLMs in responding to a range of exam questions, including proctored closed -book and open -book questions spanning various levels of Bloom's Taxonomy. Question formats encompassed open -ended, tabular data -based, and figure -based inquiries. To achieve this aim, the study has the following objectives: Comparative Analysis: To compare LLM-generated exam answers with actual student submissions to assess LLM performance. Detector Evaluation: To evaluate the efficacy of LLM detectors by directly inputting LLM-generated responses into these detectors. Additionally, assess detector performance on tampered LLM outputs designed to conceal their AIgenerated origin. The research methodology used for this paper incorporates a staff -student partnership model involving eight academic staff and six students. Students play integral roles in shaping the project's direction, particularly in areas unfamiliar to academic staff, such as specific tools to avoid LLM detection. This study contributes to the understanding of LLMs' role in advanced education settings, with implications for future curriculum design and assessment methodologies.

This ITiCSE Working Group (WG) has two goals: first, to increase awareness of computing education research in the African countries, and second, to create and strengthen connections between computing education researchers in African countries and those in the larger computing education research community. To achieve these goals, the WG will focus on two tasks: writing a literature review of publications related to computing education in African countries and building a set of contextually relevant materials -- introductory programming examples and assignments at university level -- designed for students in African countries. The final report will include an overview of past computing education research related to Africa, a current project (the new course materials), and suggestions for future work.

Large language models (LLMs) are revolutionizing the field of computing education with their powerful code-generating capabilities. Traditional pedagogical practices have focused on code writing tasks, but there is now a shift in importance towards reading, comprehending and evaluating LLM-generated code. Alongside this shift, an important new skill is emerging - the ability to solve programming tasks by constructing good prompts for code-generating models. In this work we introduce a new type of programming exercise to hone this nascent skill: 'Prompt Problems'. Prompt Problems are designed to help students learn how to write effective prompts for AI code generators. A student solves a Prompt Problem by crafting a natural language prompt which, when provided as input to an LLM, outputs code that successfully solves a specified programming task. We also present a new web-based tool called Promptly which hosts a repository of Prompt Problems and supports the automated evaluation of prompt-generated code. We deploy Promptly in one CS1 and one CS2 course and describe our experiences, which include student perceptions of this new type of activity and their interactions with the tool. We find that students are enthusiastic about Prompt Problems, and appreciate how the problems engage their computational thinking skills and expose them to new programming constructs. We discuss ideas for the future development of new variations of Prompt Problems, and the need to carefully study their integration into classroom practice.